Is there a way to run lima machines on an isolated network

[jaepetto]

We are currently testing DHCP and PXE servers for our infrastructure. For the time being I am using qemu and vde_switch to create the infrastructure.

Typically, I do have a pfsense vm with two links. The WAN link is connected to a user network and the LAN link is connected to the vde switch. Pfsense is nice to use because it has a DHCP server than can be configured to send a next server in the DHCP offer and it includes tcpdump directly.

Then I have my TFTP server running on a VM connected only to the vde switch. The last machine is a simple VM trying to boot up.

This setup works pretty well aside from some slowness related to the use of x86_64 architecture on a M1 mac.

I would like to move this infrastructure to lima. Unfortunately, I could not find any reference on how to create an isolated network (a custom switch).

Does anyone know if it's feasible?

[jandubois]

Please check https://github.com/lima-vm/lima/blob/master/docs/network.md#managed-vmnet-networks-192168105024 for more information on how to create virtual networks between Lima VMs.

Look on slide 22 from our talk for a visualization of how networking is configured: https://static.sched.com/hosted_files/kccnceu2022/5f/lima.pdf.

[jaepetto]

@jandubois Thank you very much for the resources. The only point I'm not really clear about is the way to declare a new network in the $LIMA_HOME/_config/networks.yaml.

Would you happen to have an example?

[jandubois]

Just copy any of the 3 pre-defined networks (host, bridged, shared) and give them a different name (and subnet, for non-bridged networks).

These 3 networks are just predefined, but not special in any other way. You can define more, but there is a limit (from Apple's VMNET framework) on how many you can actually instantiate at the same time at runtime.

[jaepetto]

I just tried to create an isolated network (without any configuration - which is my intent):

paths:
  vdeSwitch: /opt/homebrew/Cellar/vde/2.3.3/bin/vde_switch
  vdeVMNet: /opt/vde/bin/vde_vmnet
  varRun: /private/var/run/lima
  sudoers: /private/etc/sudoers.d/lima

group: everyone

networks:
  shared:
    mode: shared
    gateway: 192.168.105.1
    dhcpEnd: 192.168.105.254
    netmask: 255.255.255.0
  bridged:
    mode: bridged
    interface: en0
    # bridged mode doesn't have a gateway; dhcp is managed by outside network
  host:
    mode: host
    gateway: 192.168.106.1
    dhcpEnd: 192.168.106.254
    netmask: 255.255.255.0
  isolated:

After failing because of sudoers rights (which was fixed as per documentation with limactl sudoers | sudo tee /etc/sudoers.d/lima), it is now failing with the below error message:
FATA[0000] networks.yaml field paths.vdeSwitch error: file "/opt/homebrew/Cellar/vde/2.3.3/bin/vde_switch" is not owned by "root" (uid: 0), but by uid 501

This is too in the documentation. Unfortunately, it is kind of hard to change the rights because vde_switch was installed by brew and I don't want to mess the rights in the folders belonging to it.

[andrewcrook]

sudo brew services start xxxx

Some of these tools you have to run as root. Besides slip userspace networking has limitations, slower and doesn’t support all network types for examples lookup rootless docker..