Debian Installer internals

The installer supports a lot of different installation methods and in some cases installation methods can be creatively combined. The definition of an installation method is based on the following questions.

The table below shows the answers to these questions for the most common installation methods.

^b) Whether packages for the base system and tasks are retrieved from CD (image) or the network depends on the type of CD used in combination with these boot methods.

The boot process for the installer is similar to the boot of a regular system. A bootloader (in some cases the system's firmware) is responsible for loading the kernel and loading the initrd after which init is started. The boot process can be debugged by adding the BOOT_DEBUG parameter.

It is possible to pass additional kernel and boot parameters. Kernel parameters are sometimes needed to get non-conformant hardware supported, or to install from serial console instead of an attached keyboard/display.

Boot parameters can also be used to influence the installer itself. More about this in the section on preseeding.

The boot process is currently quite complex as it needs to support several generations of linux:

The following enumeration gives an overview of the main steps in the boot process.

Scripts in /lib/debian-installer(-startup).d can be architecture specific.

The component main-menu drives the rest of the installation. It is responsible both for dynamically assembling the menu and for executing components when they are selected. Note that the menu is only actually visible when installing at low or medium debconf priority. At higher priorities it is still there, but it will automatically execute the next component without showing the menu to the user.

In some situations the debconf priority is automatically changed. It is reduced when the execution of a component fails, or when the user uses the <Go Back> button to back out all the way to the menu. In these cases it will also be increased back to the original level when the next component executed finishes successfully.

An important characteristic of udebs is that execution of their postinst scripts is delayed. On installation udebs are only unpacked; the execution of the postinst is done by main-menu and is actually what happens when a component is selected in the menu. In other words, main-menu can be said to be responsible for “configuring” the udeb.

For a component to be included in the menu, it needs to have an Installer-Menu-Item line in the dpkg status file (/var/lib/dpkg/status). The order of components in the menu is determined primarily by its dependencies; the menu item number is used only where the order for two or more components cannot be resolved by dependencies alone.

Provides can be used to define virtual states which other components can depend on, as shown in the next example.

Package: netcfg
Status: install ok installed
Version: 1.23
Provides: configured-network
Depends: libc6 (>= 2.3.5-1), libdebconfclient0, libdebian-installer4 (>= 0.37),
         dhcp-client-udeb | dhcp3-client-udeb | pump-udeb, libiw28-udeb,
         cdebconf-udeb, ethernet-card-detection
Description: Configure the network
Installer-Menu-Item: 18

Package: choose-mirror
Status: install ok unpacked
Version: 1.19
Depends: libc6 (>= 2.3.5-1), libdebconfclient0, libdebian-installer4 (>= 0.38),
         configured-network
Description: Choose mirror to install from
Installer-Menu-Item: 23

This example also shows that netcfg has been run successfully (“installed”) while mirror selection has not yet taken place (“unpacked”).

Some components are included in the menu but are not run by default. These components have a menu number higher than finish-install. Examples are components that allow to change the debconf priority, to save debug logs, to check the integrity of a CD-ROM or to abort the installation.

At certain points in the installation, components provide run-parts like execution of scripts. This allows other components to just drop scripts there so that commands can be run at that point in the installation, even though the postinst for the component itself is run earlier or later (if the component even has a postinst).

The main hooks are:

Other, more special purpose hooks are /usr/lib/apt-setup/generators, /lib/main-menu.d and /lib/rescue.d.

Besides these general hooks, partman is basically structured as one big collection of hooks where partman components all drop their own scriptlets to extend partman's functionality. The most noteworthy of these hooks is /lib/partman/finish.d as some bootloaders drop scripts there to add checks that ensure the selected partitioning scheme conforms to their requirements.

The installer has some special commands which can be used in postinst or preseeding scripts:

There are three preconfiguration methods that are currently supported:

Which of these methods is available depends on the installation method.

The main purpose of preseeding is to set default values for debconf questions. Note that it makes no sense when using file or network preseeding to preseed values for questions that are asked before the preconfiguration file is parsed.

Besides offering the option to set default values for debconf questions, preseeding also makes it possible to run scripts at two distinct moments using preseed/early_command and preseed/late_command. The early_command is executed immediately after the preconfiguration file is parsed (only for file and network preseeding); the late_command is executed as part of the finish-install component.

These scripts can be made as complex as you like. One option is to use them to install additional packages on the target system (using apt-install). The early_command could be used to install additional d-i components (using anna-install) or to install scripts in the various hook script directories (if commands need to be executed at a specific point in the installation).

Additional information about preseeding is available in an appendix of the Installation Guide.

Because most of the installer is scripted, it is fairly easy to debug most problems by adding a set -x in the correct place. The obvious place to start is the postinst of the component you want to debug. The output will appear in /var/log/syslog, which can most easily be viewed by starting the internal webserver from the “Save debug logs” menu option (after the network has been configured).

For components written in C debugging is a bit harder. Options are to use the strace udeb (add it to a custom image if needed) or to create a custom version of a component with added debug statements in the source and use that.

For debugging, you will probably want to control when components are started. Booting the installer with install debconf/priority=medium is a good way to achieve that. It will make sure the menu is displayed before a new component is started.

If you boot the installer at medium priority, you will also be able to load the optional component open-ssh-client^[4]. Loading this component will allow you to use scp to copy files from the system being installed to another computer and vice versa. Also useful for testing new versions of scripts or commands without rebuilding the udeb and image. The command nc from the netcat package is available by default in the installer.

For components that are included in the main menu, the udeb will at least contain:

Other things like additional debconf templates, C programs, hook scripts can be added as needed.

A special type of control file worth mentioning is the .isinstallable file. If a script with this extension is present in /var/lib/dpkg/info for a component, the main menu will run this script and only include the component in the menu if the script exits 0.

Creating a udeb is not all that hard, especially if you use debhelper. debhelper knows the special properties of a udeb and will do the right thing by default at build time. That is, if you don't forget to tell it you are creating a udeb.

The example below shows the debian/control file for a udeb that is supposed to be included in the main menu. Note the special “debian-installer” section.

Source: kbd-chooser
Section: debian-installer
Priority: optional
Maintainer: Debian Install System Team <debian-boot@lists.debian.org>
Uploaders: [...]
Build-Depends: debhelper (>= 5.0.22), libdebian-installer4-dev (>= 0.41), po-debconf (>= 0.5.0), flex | flex-old , bison, libdebconfclient0-dev (>= 0.49)

Package: kbd-chooser
Architecture: i386 amd64 powerpc alpha hppa sparc [...]
XC-Package-Type: udeb
Depends: ${shlibs:Depends}, ${misc:Depends}, console-keymaps
Description: Detect a keyboard and select layout
XB-Installer-Menu-Item: 12

The line XC-Package-Type tells debhelper to treat the package as a udeb. The XB-Installer-Menu-Item is added in the control file for the udeb and will eventually end up in the dpkg status file to help main-menu figure out that this udeb should be included in the menu and in what order^[6]. Packaging a udeb becomes a bit harder if it is derived from a regular package but needs to be compiled with different compiler options (e.g. some features disabled or a different optimization).

The main thing to always keep in mind when creating a udeb is size. It is very important to keep size as minimal as possible. This includes using tabs instead of spaces when indenting in scripts and not being too verbose in comments.

A major recent improvement is the addition of “package type” support in shlibs files for libraries. This allows dpkg-dev and debhelper to automatically set correct dependencies on library udebs when a d-i component that depends on them is built.

For example, the regular binary package zlib1g now has the following lines in its shlibs control file:

libz 1 zlib1g (>= 1:1.2.1)
udeb: libz 1 zlib1g-udeb (>= 1:1.2.1)

The second line is specific to the package type “udeb”. This alternative line is used when dpkg-shlibdeps is called with the -tudeb option; dh_shlibdeps will automatically add this option when processing a udeb.

Generating the extra udeb: lines is supported by dh_makeshlibs if the --add-udeb="<udeb name>" option is used. For example, the debian/rules file for libusb contains the following line:

dh_makeshlibs -V -s --add-udeb="libusb-0.1-udeb"

For both release and development builds the build dependencies as listed in installer/debian/control need to be satisfied.

To build installer images from SVN trunk, your build machine needs to be running unstable or you need to set up a sid chroot to build in. (To build images from the sarge branch of the repository, the build machine needs to run Sarge.)

During the build, the needed udebs will be retrieved from a mirror. By default this mirror is based on your /etc/apt/sources.list (see the generated file build/sources.list.udeb). To use a different source, create a file sources.list.udeb.local.

To see which targets are available, run make. This will result in a list of some 130 targets, most of which are not really relevant. A more useful list can be obtained with make | grep ^build. The table below has the most often used targets for i386.

The reallyclean target is often needed when changes are made between builds because otherwise udebs or information may be retrieved from temporary or cache directories and the changes will not take effect. The rebuild_* targets clean some of this, but not always enough.

The easiest way to start is with the purpose of the subdirectories in the installer/build directory.

Two files containing important configuration info are config/dir and config/common. However, normally there should be no need to modify any of the variables defined in these files.

Both the config and pkg-lists directories have a tree structure with general configuration defined in the root and more specific configuration defined in branches and leaves. Branches are defined in directories that have the same name as a config file on the higher level. The config directory contains makefile snippets.

MEDIUM_SUPPORTED = cdrom netboot floppy hd-media

FLAVOUR_SUPPORTED = "" 2.6
MEDIA_TYPE = netboot image
EXTRATARGETS = build_netboot_2.6

#include "discover"
console-keymaps-at
console-keymaps-usb
usb-discover [2.4]
socket-modules-${kernel:Version} ?
acpi-modules-${kernel:Version} [2.6]

#include "base"
#include "kernel"

If the build is successful, you will find the images under the build/dest directory. Depending on the type of build you will also find manifest and log files there.

Before the image is created, its contents are assembled in the directory build/tmp/<target>. The tree subdirectory there contains the full contents of the initrd; other subdirectories are used for different purposes.

Library reduction (relinking a library leaving out unused symbols) is used as yet another method to minimize the size of initrds. The downside of library reduction is that this requires the dev and pic packages for the libraries to be reduced to be installed on the build system which also means that their version needs to match the version of the libraries in the udebs.

The size reduction is most significant for libc (40%) and libm (90%). Other libraries that are reduced include libresolv, libslang and libnewt. The reduction is done by calling mklibs from the main Makefile.

As only the executables that are included in an image are taken into account during the library reduction, we have provide for executables in components that are installed later as they would fail if they use symbols that have been taken out.

This is the reason that the udebs containing reduced libraries are excluded in pkg-lists/exclude which results in the udeb not being listed in the /var/lib/dpkg/status file in the intrd. If no udebs that are installed later depend on the library, all is well. If a udeb that does depend on it is installed later, anna (or rather udpkg) will see that the dependency is not satisfied, and will install the udeb so the unreduced library replaces the reduced version.

Note that library reduction is only done after unpacking udebs for inclusion in an image; the libraries included in udebs are never reduced.

The localudebs directory allows to use a different version of udebs than is available from the mirror you use. This can be used to test a new version of a udeb or to run the installer with a debug version of a udeb. It can also be used to build an image with a custom udeb.

To use a local udeb, just copy it into the directory. A Packages file will be generated automatically. Your udeb should have a version equal to or greater than the udeb currently on the mirror you use.

Note that local udebs will only be included in the image if the udeb would be included in a normal build too. So it has to be selected by the pkg-list script. Create a pkg-lists/local or pkg-lists/<image>/local to add udebs to the image that would not normally be included.

Some things to keep in mind when using localudebs.