Even though I stopped using Proxmox on my dedicated servers, there were still some stateful containers that I needed to host that couldn't be hosted via Docker, such as "VPSes" that I give out to my friends for them to host their own stuff.
Enter LXD: A virtual machine and system containers manager developed by Canonical. LXD is included in all Ubuntu Server 20.04 (and newer versions), and can be easily set up by using the lxd init command. Just like how Proxmox can manage LXC containers, LXD can also manage LXC containers. Despite their similar names, LXD is not a "successor" to LXC; rather, it is a management tool for LXC containers. They do know that this is very confusing.
Keep in mind that LXD does not provide a GUI like Proxmox. If you prefer managing your containers through a GUI, you may find LXD less appealing. But for me? I rarely used Proxmox's GUI anyway and always managed my containers via the terminal.
Peter Shaw has already written an excellent tutorial on this topic, and his tutorial rocks! But I wanted to write my own tutorial with my own findings and discoveries, such as how to fix network issues after migrating the container, since that was left out from his tutorial because "it is a little beyond the scope of this article, that’s a topic for another post."
The source server is running Proxmox 7.1-12, the target server is running Ubuntu Server 22.04. The LXC container we plan to migrate is running Ubuntu 22.04.
First, you need to dump the container with vzdump ContainerIdHere -compress gzip -dumpdir /tmp.
root@doge-reborn:~# vzdump 40015 -compress gzip -dumpdir /tmp
INFO: starting new backup job: vzdump 40015 --compress gzip --dumpdir /tmp
INFO: filesystem type on dumpdir is 'zfs' -using /var/tmp/vzdumptmp3397965_40015 for temporary files
INFO: Starting Backup of VM 40015 (lxc)
INFO: Backup started at 2023-03-22 19:33:06
INFO: status = stopped
INFO: backup mode: stop
INFO: ionice priority: 7
INFO: CT Name: PeterKuraminha
INFO: including mount point rootfs ('/') in backup
INFO: creating vzdump archive '/tmp/vzdump-lxc-40015-2023_03_22-19_33_06.tar.gz'
INFO: Total bytes written: 5164144640 (4.9GiB, 26MiB/s)
INFO: archive file size: 1.51GB
INFO: Finished Backup of VM 40015 (00:03:13)
INFO: Backup finished at 2023-03-22 19:36:19
INFO: Backup job finished successfully
After dumping the container, you will have a file on the /tmp folder called /tmp/vzdump-lxc-ContainerIdHere-DateHere.tar.gz, along with the dump log.
root@doge-reborn:~# cd /tmp
root@doge-reborn:/tmp# ls -lh
total 1.5G
-rw-r--r-- 1 root root 607 Mar 22 19:36 vzdump-lxc-40015-2023_03_22-19_33_06.log
-rw-r--r-- 1 root root 1.6G Mar 22 19:36 vzdump-lxc-40015-2023_03_22-19_33_06.tar.gz
Next, copy the vzdump-lxc-ContainerIdHere-DateHere.tar.gz file to your new server. To import the container into LXD, we need to create a metadata.yaml indicating the architecture and the creation date of the container.
x86_64/amd64. If you want to be sure, use pct config ContainerIdHere in your Proxmox server to get the container's configuration.creation_date timestamp doesn't matter. It is a timestamp in epoch seconds.root@magnificent-brunette:/srv/peter# nano metadata.yaml
architecture: x86_64
creation_date: 1424284563
With the metadata.yaml file ready, you need to compress it with tar -czvf metadata.tar.gz metadata.yamlbefore importing it to LXD... Yeah, I'm not sure why it needs to be compressed.
root@magnificent-brunette:/srv/peter# tar -czvf metadata.tar.gz metadata.yaml
metadata.yaml
root@magnificent-brunette:/srv/peter# ls
metadata.tar.gz metadata.yaml vzdump-lxc-40015-2023_03_22-19_33_06.tar.gz
And finally, you can now import the container to LXD with the lxc image import metadata.tar.gz vzdump-lxc-ContainerIdHere-DateHere.tar.gz command.
root@magnificent-brunette:/srv/peter# lxc image import metadata.tar.gz vzdump-lxc-40015-2023_03_22-19_33_06.tar.gz
Image imported with fingerprint: 3084015d1954255d157855be9bbeebeffa6fa6f745ecfdaaa1eb726d01e7d68b
You can see all of your imported images with lxc image list.
root@magnificent-brunette:/srv/peter# lxc image list
+-------+--------------+--------+-------------+--------------+-----------+------------+-------------------------------+
| ALIAS | FINGERPRINT | PUBLIC | DESCRIPTION | ARCHITECTURE | TYPE | SIZE | UPLOAD DATE |
+-------+--------------+--------+-------------+--------------+-----------+------------+-------------------------------+
| | 3084015d1954 | no | | x86_64 | CONTAINER | 1541.43MB | Mar 22, 2023 at 10:47pm (UTC) |
+-------+--------------+--------+-------------+--------------+-----------+------------+-------------------------------+
And now you can launch the container with lxc launch ImageFingerprintHere NameOfTheContainerHere!
root@magnificent-brunette:/srv/peter# lxc launch 3084015d1954255d157855be9bbeebeffa6fa6f745ecfdaaa1eb726d01e7d68b peterkuraminha
Creating peterkuraminha
Starting peterkuraminha
You can attach to the container (like how you would with lxc-attach in Proxmox) with lxc exec ContainerName /bin/bash
root@magnificent-brunette:/srv/peter# lxc exec peterkuraminha /bin/bash
root@PeterKuraminha:~#
One thing that you will notice is that the container won't have any internet connectivity, this is because we need to make some changes to the network settings from within the container to fix it.
root@PeterKuraminha:~# ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
From 10.31.253.16 icmp_seq=1 Destination Host Unreachable
From 10.31.253.16 icmp_seq=2 Destination Host Unreachable
From 10.31.253.16 icmp_seq=3 Destination Host Unreachable
^C
--- 1.1.1.1 ping statistics ---
5 packets transmitted, 0 received, +3 errors, 100% packet loss, time 4080ms
pipe 4
Now, this step varies depending on what operating system you are using for your container, and how your network configuration is set up, but here's how to fix it for an Ubuntu Server 22.04 container with the default LXD network configuration. If your container is running a different distro, one thing that I recommend is running a new LXC container via LXD, checking if internet works correctly within it, tracking down what files make the network connectivity "tick" and then copying the files over to your migrated container.
First, open your network configuration file. Keep in mind that the eth0 filename may be different if you setup the network interface with an different name in Proxmox.
root@PeterKuraminha:~# nano /etc/systemd/network/eth0.network
[Match]
Name = eth0
[Network]
Description = Interface eth0 autoconfigured by PVE
Address = 10.31.253.16/8
Gateway = 10.0.0.1
DHCP = no
IPv6AcceptRA = false
Delete everything and replace the contents within with
[Match]
Name = eth0
[Network]
DHCP = yes
This will indicate that the container will get their IP via DHCP. The IP is automatically provided by LXD, so you don't need to worry about it.
Now reboot the system with reboot, reattach to the system and...
root@PeterKuraminha:~# ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
64 bytes from 1.1.1.1: icmp_seq=1 ttl=57 time=7.91 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=57 time=7.93 ms
64 bytes from 1.1.1.1: icmp_seq=3 ttl=57 time=7.90 ms
^C
--- 1.1.1.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2004ms
rtt min/avg/max/mdev = 7.896/7.913/7.932/0.014 ms
Now internet connectivity on the container works, yay!
And that's it! If you want to limit the container's resources usage (memory, CPU, etc) like how you would in Proxmox, check out this tutorial.