⬢ Distrobox: An In-depth Guide for Developers - Part 1

Motivation & Background

I generally use Ubuntu LTS at work, mainly due to how widely used and supported it is. However, there are many situations where it's helpful to use a different distro on the job.

Maybe I need to:

• test against an older version of Ubuntu
• reproduce a bug reported on another distro
• test something on several distros
• replicate a very specific environment
• develop with an init system I don't have
• acquire specific packages or versions from a different package repository

There are a few solutions to this problem.

We can use:

• a virtual machine (hardware virtualization)
• containers (docker / podman / etc.) (os-level virtualization)

Let's briefly consider each of these options.

Virtual Machines

Hardware virtualization is an easy and classic solution to this problem. This is the virtualization paradigm of virtual machines run by hypervisors & virtual machine monitors like qemu, virtualbox, VMware Workstation, and so forth. However, virtual machines come with some issues.

Firstly, the setup time can be quite long - downloading an iso, configuring the VM settings, installing the OS, setting up your environment with your preferred editor or IDE, shell, window manager or DE, and populating the system with libraries, compilers, language servers, and other tools. Then there can be additional time required to do things like enable clipboard sharing, which may not even work all that well for some more obscure distros. After all this, the VM itself both on disk and at runtime is a bit heavy. The hypervisor emulates hardware resources, including the entire storage device, and the bootloader and kernel are virtualized. This takes up quite a large chunk of persistent storage and typically has a relatively high memory footprint, often hindering the performance of applications, especially considering there is overhead for each resource that is not passed through to the host hardware.

It can be quite useful despite these shortcomings, but as the number of virtual environments grows, the more these issues are multiplied. Moreover, certain processes, like installing Qt from the online installer, are simply repeated over and over unnecessarily.

Containers

Containers are also a choice solution, as they circumvent these shortcomings. They virtualize at the OS userland (see OS-level virtualization) or application level. One way to think about this is the existence of several distinct userspaces, including the file system, on the same running kernel. Using the host's kernel eliminates most resource overhead and makes bootloader virtualization irrelevant. Clipboard sharing works out of the box, as the container's standard I/O is routed through a host terminal session when run interactively.

There are some key shortcomings here as well though: without further integration with the host machine, there is still a need to do significant setup to obtain tooling and libraries. Furthermore, it will not inherently have a graphical environment -- in order to open a graphical application, like a text editor, or perhaps a Qt project that you are intending to test in the first place, you will need to pass through sockets for X11 or Wayland from the host. It's quite powerful that the display doesn't need to be virtualized (and for instance a separate Xorg server doesn't need to run like in a virtual machine), but the setup for this can be cumbersome with many command line arguments. Containers are meant to be sandboxed from the host system, and share only explicitly specified resources from the host. However, this is again powerful -- we can mount parts of the host filesystem to pass through libraries, binaries, git configuration, ssh keys, project files, and so forth to share these resources between the host and any number of guests. The issue is again the tedium of setting these capabilities up for every container, both when installing and running.

Distrobox As A Solution To Shortcomings

Distrobox provides a solution to this. Due to the flexibility of containers, the aforementioned cumbersome tasks can be done automatically such that the workflow is massively simplified for the purposes of passing through graphical sessions, installing packages, and sharing parts of the host filesystem. Additional tooling provided by distrobox enables tighter integration between host and guest, like running host commands from within the guest without mounting, and exporting programs from the guest to host, so that they can be recognized by the host's application launcher for example. Distrobox automatically mounts a home directory from the host along with removable devices, journal, dbus, /dev, and many others, which can all be further configured. It can be used with several container management backends, namely docker, podman, and lilipod (a tool the authors of distrobox created).

Let's see an example of how much simpler this is. We can use the distrobox flag --dry-run to see what command is actually run on the backend when we issue a distrobox command on the host. Here I use podman as my container manager.

Doing so, we can see that the following distrobox command:

distrobox create -n tempalpine -i alpine -H /home/sus/temphome --init

translates to this podman command:

podman create
--hostname "tempalpine.sus"
--name "tempalpine"
--privileged
--security-opt label=disable
--security-opt apparmor=unconfined
--pids-limit=-1
--user root:root
--ipc host
--network host
--label "manager=distrobox"
--label "distrobox.unshare_groups=1"
--env "SHELL=bash"
--env "HOME=/home/sus"
--env "container=podman"
--env "TERMINFO_DIRS=/usr/share/terminfo:/run/host/usr/share/terminfo"
--volume /:/run/host:rslave
--volume /tmp:/tmp:rslave
--volume "/usr/bin/distrobox-init":/usr/bin/entrypoint:ro
--volume "/usr/bin/distrobox-export":/usr/bin/distrobox-export:ro
--volume "/usr/bin/distrobox-host-exec":/usr/bin/distrobox-host-exec:ro
--volume "/home/sus":"/home/sus":rslave
--volume /dev:/dev:rslave
--volume /sys:/sys:rslave
--volume /dev/pts
--volume /dev/null:/dev/ptmx
--volume /var/log/journal
--env "HOME=/home/sus/temphome"
--env "DISTROBOX_HOST_HOME=/home/sus"
--volume "/home/sus/temphome:/home/sus/temphome:rslave"
--volume /etc/hosts:/etc/hosts:ro
--volume /etc/resolv.conf:/etc/resolv.conf:ro
--annotation run.oci.keep_original_groups=1
--ulimit host
--systemd=always
--userns keep-id

--entrypoint /usr/bin/entrypoint
alpine
--verbose
--name "sus"
--user 1000
--group 1000
--home "/home/sus/temphome"
--init "1"
--nvidia "0"
--pre-init-hooks ""
--additional-packages ""
-- ''

which you can see mounts additional distrobox binaries like distrobox-init, distrobox-export, and distrobox-host-exec.

These tools themselves provide additional helpful functionality. distrobox-init, for example, is generally run automatically when a new container is initialized. It mounts several more directories, sets up sudo if needed, passes through xorg or wayland, creates a user and group for the user, and so on. This would all need to be done manually without that binary.

Thus, using distrobox saves a tremendous amount of time and effort, and wraps several reused arguments and additional commands to get such a container running quickly and without much thought.

Additional Background: Containers & OCI

For a fantastic introduction to containers and their usefulness, see this whitepaper: https://www.kdab.com/containers-in-embedded/

Further information can be found here: https://www.kdab.com/the-developers-guide-to-containers/

Some of the below may summarize, add to, or just repeat concepts in the above links.

Linux containerization is made possible by the cgroups & namespaces kernel features.

You have likely heard of, or used, docker before. Docker is the premier suite of tools for creating and deploying containers. Major contributors to the project include Microsoft, Cisco, Google, Huawei, IBM, and Red Hat.

Podman is a similar system, developed by Red Hat, that can act as a drop-in replacement for docker, as its API is identical.

Some advantages of podman over docker include:

• lack of a daemon -- dockerd can become a single point of failure for all containers
• rootless by default, while running rootless dockerd and docker-managed containers requires additional configuration

These tools are designed to be OCI-compliant. OCI (Open Container Initiative) is a set of open standards for containers. The specifications provide a common container image format and runtime model that enables using container images across multiple container management tools, and distribution of containers in a standardized packaging format. Tools can thus be made compliant with the OCI spec, and therefore should support any OCI-compliant container images. This is powerful, as someone can distribute a container image that should then work as a regular container spawned by docker or podman, and also work with distrobox, and other tooling around containers.

From Open Containers Initiative's About Page:

The OCI currently contains three specifications: the Runtime Specification (runtime-spec), the Image Specification (image-spec) and the Distribution Specification (distribution-spec). The Runtime Specification outlines how to run a “filesystem bundle” that is unpacked on disk. At a high-level an OCI implementation would download an OCI Image then unpack that image into an OCI Runtime filesystem bundle. At this point the OCI Runtime Bundle would be run by an OCI Runtime.
This entire workflow should support the UX that users have come to expect from container engines like Docker and rkt: primarily, the ability to run an image with no additional arguments:
docker run example.com/org/app:v1.0.0
rkt run example.com/org/app,version=v1.0.0

OCI Images more or less consist of metadata in json files, and a number of image layers. The image layers are essentially tar archives containing changesets on one another back to a base filesystem, which is, at its most simple, just an empty directory. The unpacked archives can be thought of like a diff format for an entire filesystem, and the manifest describes how these diffs are applied on top of one another to result in a filesystem that is runnable.

For more information, see the official specification pages.

Distrobox's Purpose In Terms of These Background Concepts

Distrobox's website provides a very concise summary of the tool that ties together all the content above:

The distrobox environment is based on an OCI image. This image is used to create a container that seamlessly integrates with the rest of the operating system by providing access to the user’s home directory, the Wayland and X11 sockets, networking, removable devices (like USB sticks), systemd journal, SSH agent, D-Bus, ulimits, /dev and the udev database, etc.

It also provides some compelling use cases:

• Provide a mutable environment on an immutable OS, like ChromeOS, Endless OS, Fedora Silverblue, OpenSUSE Aeon/Kalpa, Vanilla OS, or SteamOS3
• Provide a locally privileged environment for sudoless setups (eg. company-provided laptops, security reasons, etc…)
• To mix and match a stable base system (eg. Debian Stable, Ubuntu LTS, RedHat) with a bleeding-edge environment for development or gaming (eg. Arch, OpenSUSE Tumbleweed, or Fedora with the latest Mesa)
• Leverage a high abundance of curated distro images for docker/podman to manage multiple environments.

Some notable goals and features mentioned on the site include:

This project aims to bring any distro userland to any other distro supporting podman, docker, or lilipod. It has been written in POSIX shell to be as portable as possible and it does not have problems with dependencies and glibc version’s compatibility.
Refer HERE for a list of supported container managers and minimum supported versions.
It also aims to enter the container as fast as possible, every millisecond adds up if you use the container as your default environment for your terminal.

Intro to Using Distrobox

With the background out of the way, let's see how distrobox is used.

Note:
I'm using starship as my prompt, which will be relevant later. In my host environment, the prompt will look something like this:

~ 
🎄

or

~/exampledir 
🎄

instead of something like this:

~ $

or

~/exampledir $

You will notice that different containers often have different default prompts, and we will import my host's prompt into a container later in the article.

Creating Containers

A basic command to create a distrobox container for Alpine Linux looks like so:

~ 
🎄 distrobox create --image alpine

alpine is a shortname defined in /etc/containers/registries.conf.d/shortnames.conf. on my machine, it corresponds to docker.io/library/alpine.

We can also use -i instead of --image.

The container will automatically be named alpine here, automatically generated from the repository path docker.io/library/alpine. Attaching a tag to the path also affects the name. For instance, docker.io/library/alpine:latest or simply alpine:latest would result in the name being alpine-latest

The container's name is the way we interact with the container, to start, stop, enter, or delete it.

Let's say we want a gentoo image. There is no "gentoo" shortname in shortnames.conf, so we will provide the full path and tag:

~ 
🎄 distrobox create -i docker.io/gentoo/stage3:latest

This works fine, but it names the container stage3-latest. I might forget this name, so it's easier if i just call it gentoo. We specify a name for it with --name or -n:

~ 
🎄 distrobox create -i docker.io/gentoo/stage3:latest -n gentoo

Here is a list of supported container images that are officially tested. Other images can work too, but these are tested explicitly by the distrobox developers.

Entering Containers

Let's actually use a container and show what it looks like and how it can be useful.

~ 
🎄 distrobox enter alpine
Starting container...                   	 [ OK ]
Installing basic packages...            	 [ OK ]
Setting up devpts mounts...             	 [ OK ]
Setting up read-only mounts...          	 [ OK ]
Setting up read-write mounts...         	 [ OK ]
Setting up host's sockets integration...	 [ OK ]
Integrating host's themes, icons, fonts...	 [ OK ]
Setting up package manager exceptions...	 [ OK ]
Setting up distrobox profile...         	 [ OK ]
Setting up sudo...                      	 [ OK ]
Setting up user groups...               	 [ OK ]
Setting up kerberos integration...      	 [ OK ]
Setting up user's group list...         	 [ OK ]
Setting up existing user...             	 [ OK ]
Setting up user home...                 	 [ OK ]
Ensuring user's access...               	 [ OK ]

Container Setup Complete!
sus@alpine:~$

If we look at /lib in the container, we get our alpine image's lib files:

sus@alpine:~$ ls /lib
apk       ld-linux-x86-64.so.2  libc.musl-x86_64.so.1  libcrypt.so.1    libm.so.6        libresolv.so.2  libutil.so.1  modules-load.d
firmware  ld-musl-x86_64.so.1   libc.so.6              libgcompat.so.0  libpthread.so.0  librt.so.1      mdev          sysctl.d

This should be different from the /lib directory on my host machine and feel like its own machine, almost like a VM. But, distrobox also mounted my host's $HOME as a read-write directory. If we ls in ~ we will see the files from our host's ~.

We can use alpine's package manager to get programs we want inside the container:

sus@alpine:~$ vim
bash: vim: command not found
sus@alpine:~$ sudo apk add vim
(1/4) Installing vim-common (9.1.1012-r0)
(2/4) Installing xxd (9.1.1012-r0)
(3/4) Installing vim (9.1.1012-r0)
(4/4) Installing vim-doc (9.1.1012-r0)
Executing busybox-1.37.0-r9.trigger
OK: 546 MiB in 367 packages
sus@alpine:~$ vim

If we don't want it to specifically be an alpine package, and we already have the program on the host, we can also mount the host's installation files when we create the container. This grants us the flexibility to use what we already have where we can, and use the container distribution to get other packages.

Our installed programs also persist when we restart the container.

If we install something in a running container that we don't have on the host system:

sus@alpine:~$ sudo apk add qutebrowser
...
...
sus@alpine:~$ qutebrowser

This will open qutebrowser, which is a graphical application, using the host's X session (assuming you are running an Xorg session).

If we leave the container (Ctrl+D, exit, logout, etc.) and try to use qutebrowser:

sus@alpine:~$ 
logout

~ took 13m1s 
🎄 qutebrowser
Command 'qutebrowser' not found, but can be installed with:
sudo apt install qutebrowser

~ 
🎄

... we clearly can't use it, as the container has the program, not our host system. However, we can run it from the host system without entering an interactive container session by specifying a single command to run in the container, like so:

~ 
🎄 distrobox enter alpine -- qutebrowser

We can also export it from the container so we can invoke the command in distrobox from the host, and even make the program locatable by our application launcher.

Next Steps

In the next part of the series, we'll explore some more involved examples, including:

• using additional flags to customize how our container is created and integrated with the host
• further integrating the host with the guest

The post appeared first on KDAB.