Not only Docker - how LXCs work

Today we're going to talk about LXC, a minimalist Linux container system that we use in our production servers.

When you type the word "container" into a search engine, your first result will probably be Docker. Somewhere below that you'll probably find Kubernetes (also often used with Docker), some cloud solutions, and other commercial solutions. Somewhere at the bottom, perhaps an offering of true shipping containers or other large-scale transport will break through, if they manage to break through the popularity of OS-level virtualization solutions at all.

However, the topic of Linux containers neither begins nor ends with Docker. In this article, we'll look at LXC (short for Linux Containers). LXC is a simple and minimalistic container system. It provides a complete set of tools needed to support containers, while unlike Docker, it does not impose a way of working. It works similarly to classic virtual machines, i.e. by default it runs the entire operating system inside the container (except of course the kernel). LXC can run any Linux installation in a container as long as we give it the path to the directory/disk containing it. Interestingly, for a long time Docker itself was based on LXC until libcontainer was created, using liblxc to activate and manage containers.

What is LXC?

  • Based on the Namespaces and Control Groups mechanisms present in the standard Linux kernel (similar to Docker).
  • Each container is practically a full-fledged operating system just like a normal virtual machine (it only shares the kernel with the host).
  • Like other container solutions, unlike standard virtualization, programs under LXC run just as fast as when running "natively".
  • Configuration is based on simple text files, and to manage containers we have a few simple commands (lxc-create, lxc-start, etc.).
  • Allows a very wide integration between the host and the container, easily sharing not only the files, but also the whole devices, and even sockets (eg, we can easily run an application in a container and pass it a socket X11 so that its GUI is displayed on the host screen).
  • It provides a high level of security, supporting mechanisms such as Apparmor or SELinux (however, it does not require them to operate). Of course, like any container solution, it is slightly less secure than VMs, because it shares the kernel with the host.
  • The policy of the project is to allow the user maximum configurability, including getting rid of some of the containerization (such as sharing the network with the host).
  • The whole thing is completely free and released under the LGPL license.

Let's move on to practice. Let's try to create a simple LXC container. We will be working under the Debian distribution "buster", but LXC is available under practically any popular distribution.

Before the whole thing, let's get root permissions (we can also precede all subsequent commands with sudo command)

> sudo -i

Let's start by installing the appropriate packages.

> apt install lxc lxc-templates

We already have LXC installed and can create our first simple container.

> lxc-create -n container-name -t download

After executing this command, LXC will ask you what Linux distribution you want to install in your container. We type debian, buster, amd64 one by one. LXC has created a new container and installed our chosen system in it. Let's see what actually happened.

> cd /var/lib/lxc/container-name > ls

In the folder we will find two elements: a file named config and rootfs folder. In rootfs folder we will find nothing else but file system of our container. Yes, by default LXC keeps all operating system and other files of our container simply in one folder. Of course we can use different way of storage (like LVM or ZFS), but for our needs this option is perfect. Thanks to this, we will be able to move or backup a deactivated (!) container just like any other file.

The second thing is config file. It contains configuration of our container. We can edit it with our favorite text editor. Full config file description we will get after typing command:

> man lxc.container.conf

We already have our first container. Let's try to get it up and running.

> lxc-start -n container-name

At this point our container should already be starting, we can check its status with the command:

> lxc-info -n container-name

Or

> lxc-ls -f # displays the status of all containers

Now let's try connecting to our container.

> lxc-attach -n container-name

At this point, you should see bash or another shell system running in a container with root privileges. What can we do? Pretty much everything you would do with a normal virtual machine! (just remember that you share one kernel with the host system, and things like direct access to the disk (or other things from /dev) won't work unless you explicitly allow it in config (e.g. by mounting the appropriate file/device in the container with the bind option). Exit the container.

> exit

However, most likely (and certainly under debian) our container will not have network access by default. In our container's config you will find a line like this:

lxc.net.0.type = empty

We can connect the container to the network in several ways. Unfortunately, LXC due to its minimalism will not do it for us. The two most popular methods are:

  • NAT - containers will use the IP address of our computer to go out into the world, however to connect to them from outside we will need to set up port forwarding on our computer. This is the default option for e.g. Docker.
  • Bridge - we are "splicing" the container network with the network to which our computer is connected. The container will be directly available on our LAN

We use the latter, but if you use LXC at home, you will probably find the NAT option more beneficial. As the network configuration is strongly dependent on the distribution you are using, I would refer you to its documentation. In many you can even do it from the GUI e.g. via nm-connection-editor. If you are also using Docker or Libvirt, you probably already have NAT configured and after typing:

> brctl show

We will be shown the name of the network bridge that will provide us with access to the Internet. We can also connect a physical network card directly under the container or use one of the many other solutions available in LXC.

Using one method, we were able to get a network bridge that gives us access to the internet. Let's connect it under the container. We open the config of our container and replace the previous network configuration with:

lxc.net.0.type = veth lxc.net.0.link = name-network-bridge lxc.net.0.name = eth0

We can also directly set the container's IP address in config by adding (with the appropriate addresses, of course):

lxc.net.0.ipv4.address = 10.20.10.120/16 lxc.net.0.ipv4.gateway = 10.20.0.1 lxc.net.0.flags = up

Then we can restart our container:

> lxc-stop -n container-name > lxc-start -n container-name > lxc-attach -n container-name

We should now be able to access the network. If we set a static IP address, we'll probably also need to update the DNS server address in /etc/resolv.conf in the container.

We can already create containers that can easily replace most virtual machines. In particular, using LXC, we can also turn an existing VM into an LXC container - all we have to do is copy its files into a folder, write our own config, and put everything in /var/lib/lxc. We don't have to do anything else, LXC will find the new container itself and let us start it by lxc-start.

What next? We can further secure our system by creating unprivileged containers. This mechanism allows us to ensure that if an attacker manages to get out of an LXC container it will have zero privileges. We just need to add the following line to /etc/subuid and /etc/subgid (our distribution may have done this for us):

root:10000:1000000000

And to /etc/lxc/default.conf add:

lxc.idmap = u 0 10000 1000000000 lxc.idmap = g 0 10000 1000000000

And the next container we create will run unprivileged. We encourage you to do this - it's a very small change, and it significantly increases the security of the whole thing.

If your distribution uses AppArmor or SELinux you can use them to make your containers even more secure. Ubuntu, for example, does this by default with AppArmor, which can cause some problems with some applications in the container. Of course, in that case, it's best to tweak the AppArmor settings. As a last resort, we can also add to config

lxc.apparmor.profile = unconfined

which will disable apparmor in the container.

The capabilities of LXC are far greater than described in this article. It can also work in a similar way to docker - containerize individual processes (lxc-execute), create ephemeral containers (removed/cleaned to bootable state after every run) or be accessible to a regular user (not root, as we did in this tutorial).

LXC is not limited to the tools presented. Its creators themselves have written a high-level frontend called LXD. It allows automated management of multiple containers, including automatic migrations between machines or network handling. Also worth noting is Proxmox Virtual Environment, a modified Linux distribution specifically for virtualization. It supports both standard virtual machines (QEMU/KVM) and LXC containers, allowing for their easy management from the browser level. However, for the time being we stick to "pure" LXC, it is completely sufficient for our needs.

For more information about LXC, I refer you to linuxcontainers/ and, as always, the indispensable ArchWiki.

The original article in Polish can be found here.

01 May 2021