Introduction
miso builds operating system images from a build file. The build file is
called an Imagefile and looks like a Dockerfile. It has FROM, COPY and
RUN steps, and every step becomes a cached layer.
The result comes out as standard systemd artifacts:
- disk images that boot under UEFI
- ISO images
- root file systems, with the kernel and initrd to boot them
- portable services, system extensions and configuration extensions
- the files
systemd-sysupdateconsumes
A CHECK step boots the result and runs a command in it. If the command
fails, the build fails and no output is written.
What miso does not do
miso builds images. It does not apply updates, configure machines or run anything on the target after installation. It writes standard systemd formats and leaves the rest to whoever runs the machine.
Quick start
This page builds a Debian disk image with htop in it.
You need Linux on amd64 or arm64, qemu-system-x86_64 on amd64 or
qemu-system-aarch64 on arm64, and access to /dev/kvm.
The first build downloads the builder kernel, the firmware and the Debian
base image.
The Imagefile
Make a directory and put a file named Imagefile in it:
FROM debian:sid
RUN apt-get update && apt-get install -y --no-install-recommends systemd-boot-efi htop
RUN : > /etc/fstab
CMDLINE rw console=ttyS0
PARTITION esp Type=esp Format=vfat CopyFiles=/efi:/ SizeMinBytes=256M SizeMaxBytes=256M
PARTITION root Type=root Format=ext4 CopyFiles=/:/ SizeMinBytes=3G
OUTPUT disk os.raw
CHECK command -v htop
This is examples/htop/Imagefile from the repository without its comments.
| Line | Meaning |
|---|---|
FROM debian:sid | Start from the Debian sid cloud image. |
RUN apt-get ... | Install htop and the systemd-boot files the disk needs. |
RUN : > /etc/fstab | Empty the fstab. The cloud image’s fstab names partitions the new disk does not have. |
CMDLINE | The kernel command line of the disk. |
PARTITION | A partition of the disk, written as systemd-repart settings. |
OUTPUT disk os.raw | Write the disk to a file named os.raw. |
CHECK | Boot the disk and run this command in it. |
On arm64 the serial console is ttyAMA0, so write
CMDLINE rw console=ttyAMA0 there. With ttyS0 the disk still boots, but a
failed check has no boot log to show.
Plan
miso plan lists the steps without building anything. Run it in the
directory with the Imagefile:
miso plan
Each stage starts with its FROM line. Under it, every step shows its line
number, its cache key, whether the layer is cached or would run, and the
step as written. The stage named miso tools is not in your file. miso adds
it for the tools that write the disk.
Build
miso build -o out
miso prints the output of every step. When the build succeeds, the disk is
out/os.raw. It is a GPT disk with an EFI system partition and a root
partition, and it boots under UEFI.
Build again
miso keeps the result of every step as a layer. If you change a step, miso
builds that step and all steps after it again and reuses the rest. Edit the
RUN line with htop and run miso plan to see it: the cache key of that
step and of every step after it changes, and those steps say run.
The Imagefile
An Imagefile describes how to build an image. This page lists every
instruction and the rules of the file. miso build and miso plan read a
file named Imagefile in the directory you give them, or the file named by
-f.
Example Imagefile:
FROM debian:sid
RUN apt-get update && apt-get install -y htop
OUTPUT rootfs os.ext4
File format
- One instruction per line. The keyword comes first. Keywords are not case
sensitive, so
runandRUNare the same. - Spaces and tabs before a keyword are ignored.
- Blank lines are ignored.
- A line whose first character is
#is a comment. A#later in a line is not a comment. It is part of the instruction. - A line that ends in
\continues on the next line. The backslash is removed and the lines are joined without a newline. The next line’s leading spaces stay. - A file needs at least one
FROM. Every other instruction must come after one.
An error names the line where the instruction starts, for example
Imagefile:4: COPY needs an absolute destination.
Stages
Each FROM starts a stage. The instructions after it belong to that stage
until the next FROM. A stage is a root file system that the instructions
build up step by step.
Instructions fall into three groups:
RUN,ENVandCOPYchange the root file system.PARTITIONandCMDLINEdescribe a disk and leave the root file system alone.OUTPUTandCHECKmake files and leave the root file system alone.
An OUTPUT takes the root file system as it is at that line. Instructions
below it do not change what it captured. The PARTITION and CMDLINE lines
above an OUTPUT in the same stage belong to it, and the ones below do not.
A stage can have any number of outputs. A stage without outputs is useful as
a base for another stage or as the source of a COPY --from.
FROM
FROM <base>
FROM <base> AS <name>
Starts a stage from a base. The base is one of:
scratch, an empty root file system.- The name of an earlier stage. The new stage starts where that stage ends,
including its
ENVvalues. debian:sid,debian:13ordebian:trixie. These are the Debian cloud images for the architecture the image is built for.debian:13anddebian:trixieare the same image. A name means whatever image is current at its address when it is first fetched. After that the fetched image is cached.
Any other name is an error. Paths, URLs and container images are not yet supported.
AS <name> names the stage so that a later stage can start FROM <name> or
use COPY --from=<name>. Names are case sensitive and unique in the file.
scratch cannot be a stage name.
RUN
RUN [--network=none|default] <command>
Runs a shell command in the stage’s root file system. The command is run
with /bin/sh -c, so the image needs a /bin/sh. The rest of the line is
the command, exactly as written.
Every RUN runs as root in the builder VM, in its own process, mount and
network namespaces, with / as the working directory. The environment is
PATH, HOME=/root, TARGETARCH and the values set by ENV. The
environment of the host is not passed on.
TARGETARCH is the architecture the image is built for, amd64 or arm64,
so one Imagefile can install the kernel of either:
RUN apt-get install -y linux-image-$TARGETARCH
An ENV that sets TARGETARCH replaces it.
A command that exits with a code other than 0 fails the build.
--network=none gives the command no network except loopback. The default
is --network=default, which gives it network access. Giving default is
the same as giving no option.
The result of a RUN is cached as a layer. The cache key depends on the
command and on every step before it.
ENV
ENV <key>=<value> [<key>=<value> ...]
Sets environment variables for the RUN instructions after it in the same
stage, and for stages that start FROM this stage. A later ENV with the
same key replaces the earlier value.
A value with spaces is put in double quotes:
ENV GREETING="hello world" LEVEL=2
A value is everything after the first =. Single quotes and backslash
escapes are not special.
ENV does not write anything into the image. It does not reach COPY and
it does not reach CHECK. A variable that the booted system needs has to be
put into the image by a RUN.
COPY
COPY <source>... <destination>
COPY --from=<stage> <source>... <destination>
Copies files into the root file system.
The destination must be an absolute path. With more than one source it must
end in /. Paths with spaces are put in double quotes.
From the build context
Without --from, the sources are paths relative to the build context, the
directory you pass to miso build.
- A source is a literal path. Patterns such as
*are not expanded. - A source must exist and must lie inside the context. A path with
..or a symbolic link in the middle of it is an error. A symbolic link as the last part is copied as a link. - A directory source copies its contents into the destination.
COPY app /opt/app/puts the files ofappdirectly in/opt/app/. - A file goes into the destination if the destination ends in
/or is a directory that already exists. Otherwise the destination is the new name of the file. - Missing parent directories are created with mode 0755. Files that already exist are replaced.
- Everything is owned by root. The mode of a file is its mode in the context. Times are not kept.
Changing a file in the context changes the cache key of the COPY and of
every step after it.
From another stage
With --from=<stage>, the sources are absolute paths in the root file
system of an earlier stage, as it is at the end of that stage:
FROM debian:sid AS program
RUN apt-get update && apt-get install -y busybox-static
FROM scratch
COPY --from=program /bin/busybox /usr/bin/hello
Owner, mode and extended attributes are kept. The destination rules are the same as above.
Copying a file made by an OUTPUT of another stage is not supported yet.
CMDLINE
CMDLINE <text>
Adds to the kernel command line of the disks of the stage. The text is taken
as written. Several CMDLINE lines are joined with one space, in file order.
CMDLINE is used by OUTPUT disk, OUTPUT iso and OUTPUT update. It
becomes the command line of the unified kernel image on the disk and takes
the place of /etc/kernel/cmdline in the image. Without a CMDLINE, the
image’s own /etc/kernel/cmdline is used if it has one.
A CHECK on an OUTPUT rootfs boots the kernel with this command line.
Only the lines above an OUTPUT count for it. Lines are not passed on to a
stage that starts FROM this stage. The other kinds of output ignore
CMDLINE.
PARTITION
PARTITION <name> [<Key>=<Value> ...]
Adds a partition to the disks of the stage. The settings are the ones
systemd-repart takes in a partition definition file, such as Type,
Format, CopyFiles, SizeMinBytes and SplitName. See
repart.d(5) for the list.
PARTITION esp Type=esp Format=vfat CopyFiles=/efi:/ SizeMinBytes=256M SizeMaxBytes=256M
PARTITION root Type=root Format=ext4 CopyFiles=/:/ SizeMinBytes=3G
- Settings are separated by spaces. A value cannot contain a space.
- A key can appear more than once, as
CopyFilesoften does. - miso does not check the settings.
systemd-repartreports a bad one when the disk is made. - The name is only used for the file name of the definition. It has to be a plain file name, and it is unique within a stage.
- The partitions are created in the order of the lines.
With at least one PARTITION, only those partitions are used. With none,
systemd-repart reads the definitions it finds in the image.
PARTITION is used by OUTPUT disk, OUTPUT iso and OUTPUT update. For
the other kinds of output it does nothing, and miso does not warn about it.
Only the lines above an OUTPUT count for it.
The disk needs systemd-boot and its stub in the image, so install
systemd-boot-efi, and it needs a kernel and an initrd.
OUTPUT
OUTPUT <kind> <name> [--option[=value] ...]
Writes a file from the stage. The name is a plain file name and is unique in
the whole file, over all stages. The files are written to the directory given
to miso build -o.
| Kind | Result | Options |
|---|---|---|
disk | A GPT disk image with a unified kernel image. | |
iso | The same disk with a boot catalog for optical media. | |
rootfs | The root file system as an image. | --format=ext4 (default) or --format=erofs |
portable | A portable service in a disk image with one root partition. | --format |
sysext | A system extension in a disk image with one root partition. | --format |
confext | A configuration extension in a disk image with one root partition. | --format |
update | A directory with the files systemd-sysupdate uses. | --version=<version> (required) |
kernel | The kernel of the image. | --elf unpacks an x86 kernel to an ELF file, --path=<file> names it |
initrd | The initrd of the image. | --path=<file> names it |
The kernel and the initrd are found in the image. miso uses the newest
version under /usr/lib/modules, unless --path names the file.
Disks are written as sparse files.
miso adds a stage named miso tools to the build when an output needs
tools to be made. It holds systemd-repart, ukify, the file system tools
and compressors, installed from a fixed Debian snapshot. Plain kernel and
initrd outputs do not need it.
CHECK
CHECK <command>
Boots the output above it and runs the command in the booted system. The
command is run by /bin/sh in the image. It is taken as written.
- A
CHECKbelongs to the closestOUTPUTabove it in the same stage. Several checks after one output run in order, in one boot. - The output has to be a
disk, anisoor arootfs. The other kinds are never booted, and aCHECKfor them is an error. - A
rootfsneeds a kernel and an initrd to boot. Put anOUTPUT kerneland anOUTPUT initrdabove it. It also needs aCMDLINEwith aroot=word above it. - The image needs systemd.
ENVvalues are not set.
If a command exits with a code other than 0, the build fails at that line and the output is not written. Checks after it do not run.
A check runs during miso build, even without -o, for disks and for the
kernel and initrd that a checked rootfs is booted with.
Command line
miso <command> [options] [arguments]
| Command | What it does |
|---|---|
plan | Lists the steps of a build without building. |
build | Builds the outputs of an Imagefile. |
shell | Opens a shell on the layers at a step of the build. |
prune | Removes old layers and downloads from the cache. |
resize | Makes the cache disk larger. |
version | Prints the version. |
miso --help and miso <command> --help print the options of a command.
The cache
miso keeps everything it fetches and builds in a miso directory inside the
user cache directory. On Linux that is $XDG_CACHE_HOME/miso, or
~/.cache/miso if the variable is not set.
bases/holds the downloaded base images.builder/layers.imgis the disk the builder VM keeps the layers on. It is a sparse file. It takes only the space the layers use and can grow up to 50 GiB.miso resizemakes it larger.
Only one miso uses the cache disk at a time. A second one waits and prints
miso: waiting for another build to let go of the cache.
The build context and the build file
plan, build and shell read an Imagefile. They take the build context as
an optional argument. The context is a directory and defaults to the current
one. The build file is the file named Imagefile in the context. -f names
another file. A path given to -f is used as it is and is not relative to the
context. COPY always reads from the context.
miso plan
miso plan [-f <file>] [--arch <arch>] [context]
Reads the Imagefile and lists what a build would do, without building.
--arch plans the build for an architecture, as it does for
miso build.
The first line names the version of miso. A base image that is not in the
cache yet is listed as download <name>. Then every stage is listed with its
FROM line, and under it every step with its line number, its cache key,
whether its layer is cached, and the step as written. A stage named
miso tools is added by miso when an output needs the tools to be made.
After the key, RUN, COPY and OUTPUT say cached when the layer is on
the cache disk and run when a build would make it. With no cache disk, or a
base image that is not downloaded yet, every one of them says run. To know
what is cached, miso plan starts the builder VM, so it waits while another
build uses the cache. It does not start the VM when there is nothing to ask.
Two plans with the same cache key for a step build the same layer. After a change, the keys of the changed step and of every step after it are different.
An error in the Imagefile is reported with the file name and the line, for
example Imagefile:4: COPY needs an absolute destination.
miso build
miso build [-f <file>] [-o <directory>] [--arch <arch>] [context]
Builds the Imagefile in the builder VM. The steps are run one by one and their output is printed. A step whose cache key is already in the cache is not run again.
| Option | Meaning |
|---|---|
-f, --file | The build file. Imagefile in the context if not given. |
-o, --output | The directory the outputs are written to. It is created if it is missing. |
--arch | The architecture the image is built for, amd64 or arm64. The default is the architecture miso runs on. |
An image for the other architecture is built in the same builder VM. Its
RUN steps run through QEMU in user mode, qemu-x86_64-static or
qemu-aarch64-static, which the qemu-user-static package installs. Its
checks boot without KVM. Both are much slower than a build for the
architecture miso runs on.
Without -o no output is written. Disks with a CHECK are still built and
booted, so the checks run.
The build stops at the first step that fails. An error in a step names the
line of the Imagefile, and for a step that is not a FROM it also prints a
miso shell command that opens a shell before that step.
The builder VM needs access to /dev/kvm.
miso shell
miso shell [-f <file>] [--arch <arch>] [--before <line>] [context] [-- <command>...]
Opens an interactive shell in the root file system as it is at some point of the build. Layers that are not built yet are built first.
| Option | Meaning |
|---|---|
-f, --file | The build file. Imagefile in the context if not given. |
--arch | The architecture of the layers, as for miso build. |
--before | The line of the Imagefile whose step the shell stands before. |
Without --before the shell stands at the end of the last stage. With
--before 12 it stands before the step on line 12, with the environment
that step would have and the network it would have. A line that holds no step
is an error. A FROM line is not a step.
Everything written in the shell is thrown away when it ends. It does not change the cache.
Words after -- are run instead of the interactive shell:
miso shell --before 12 -- ls /etc
The exit code of the shell, or of the command, is the exit code of miso.
miso prune
miso prune [--older-than <duration>] [--keep-storage <size>]
Removes layers and downloaded files that were not used for a while. At
least one option is needed. Without one, prune stops with an error.
| Option | Meaning |
|---|---|
--older-than | Remove what no build used for longer than this. A duration such as 720h. |
--keep-storage | Remove the least recently used until the layers take no more than this, and the downloads take no more than this. A size such as 10GiB. |
Both limits must be above zero. With both, what either of them selects is removed.
prune prints how many layers it removed, then how many downloads, each with
the bytes it freed.
miso resize
miso resize [--size <size>]
Makes the cache disk larger. The layers on it are kept. It can only grow the disk. A size smaller than the disk is an error and leaves the disk as it was.
| Option | Meaning |
|---|---|
--size | The new size, such as 80GiB. 50 GiB if not given. |
Without --size, a disk that already has 50 GiB or more is left alone.
resize needs an existing cache disk. It does not make one. The first build
makes it.
Sizes are written as a number and a unit, such as 500MB or 10GiB.
miso version
miso version
Prints the version of this build of miso.
Outputs
An OUTPUT instruction writes a file from a stage. This page lists every
kind of output with what it contains, what the image needs for it and how
to use the result. The syntax of OUTPUT is on the
Imagefile page.
| Kind | Result | Booted by a CHECK |
|---|---|---|
disk | A bootable GPT disk image. | yes |
iso | A bootable ISO image. | yes |
rootfs | The root file system as an image. | yes |
kernel | The kernel of the image. | no |
initrd | The initrd of the image. | no |
portable | A portable service. | no |
sysext | A system extension. | no |
confext | A configuration extension. | no |
update | Files for systemd-sysupdate. | no |
The output files are written to the directory given with miso build -o.
The kernel and the initrd of an image are found in the image. miso takes the
newest version under /usr/lib/modules.
disk
OUTPUT disk os.raw
A GPT disk image that boots under UEFI. The layout comes from the
PARTITION lines above the OUTPUT. The disk is made by systemd-repart.
miso builds a unified kernel image from the kernel, the initrd and the
CMDLINE of the stage, and puts it on the EFI system partition together with
systemd-boot. The image therefore needs:
- a kernel and an initrd
systemd-bootand its stub, which the Debian packagesystemd-boot-efiprovides
examples/htop builds a disk with an EFI system partition and a root
partition. The quick start walks through it.
A disk is written as a sparse file.
iso
OUTPUT iso os.iso
The same disk with a boot catalog, so that it boots from a CD drive. It is made from the same PARTITION and CMDLINE lines.
A root on a CD is not found the way a root on a disk is. The initrd needs the
drivers of the CD and a way to find a root there. examples/iso makes the
initrd again with dracut, with the loop unit and systemd-dissect added.
It uses a read-only erofs root with an overlay in memory on top.
A CHECK boots the ISO as a CD.
rootfs
OUTPUT rootfs os.ext4
OUTPUT rootfs os.erofs --format=erofs
The root file system as an image. --format is ext4, which is the default,
or erofs.
A rootfs does not contain a kernel. Write OUTPUT kernel and OUTPUT initrd
next to it to get the files to boot it with. examples/microvm does this and
the result boots in Firecracker and in QEMU.
A rootfs always has empty /dev, /proc and /sys for the kernel to mount
on, even when the stage has none.
A CHECK on a rootfs boots the kernel and the initrd of the stage with the
rootfs as a virtual disk. So the stage needs, above the rootfs:
- an
OUTPUT kernel - an
OUTPUT initrd - a
CMDLINEwith aroot=word, such asroot=/dev/vda
The initrd must be able to mount the root. Debian’s cloud initrd is made for
the disk of the cloud image, so examples/microvm makes it again with the
driver of the virtual disk.
kernel and initrd
OUTPUT kernel vmlinuz
OUTPUT kernel vmlinux --elf
OUTPUT kernel vmlinuz --path=/src/linux/arch/x86/boot/bzImage
OUTPUT initrd initrd.img
OUTPUT initrd initrd.img --path=/src/initrd.img
The kernel and the initrd of the image, copied as they are.
--elf unpacks an x86 bzImage to the ELF file it contains, which
Firecracker needs on x86. An arm64 kernel is an Image that is unpacked
already, and Firecracker takes it as it is, so an arm64 build refuses
--elf.
--path names the file in the image, for a kernel or an initrd that a RUN
built somewhere else. It must be an absolute path. It works together with
--elf.
These outputs cannot be booted, so they take no CHECK.
portable
OUTPUT portable hello.raw
OUTPUT portable hello.raw --format=erofs
A portable service: the root file system in a disk image with one root
partition. It does not boot. The image needs /usr/lib/os-release or
/etc/os-release.
On a machine with systemd:
portablectl attach ./hello.raw
systemctl start hello
--format is ext4 or erofs. examples/portable builds a service with
busybox.
sysext
OUTPUT sysext hello.raw
A system extension: files for /usr and /opt in a disk image with one root
partition. The stage should hold only what the extension adds, so it usually
starts FROM scratch and copies in what it needs.
The stage needs a release file named after the output file without .raw:
/usr/lib/extension-release.d/extension-release.hello
With ID=_any in it, the extension merges on any operating system. A real
extension names the system it was made for.
On a machine with systemd:
cp hello.raw /var/lib/extensions/
systemd-sysext merge
--format is ext4 or erofs. examples/sysext builds one.
confext
OUTPUT confext hello.raw
A configuration extension: files for /etc in a disk image with one root
partition. It is made like a sysext. The release file is
/etc/extension-release.d/extension-release.hello
On a machine with systemd:
cp hello.raw /var/lib/confexts/
systemd-confext merge
--format is ext4 or erofs. examples/confext builds one.
update
OUTPUT update updates --version=1.0
The files that systemd-sysupdate downloads, made from the partitions of the
disk. --version is required. It may use letters, digits and . _ + ^ ~ -,
and must start with a letter or a digit.
The output is a directory. Name the partitions to ship with SplitName= in
their PARTITION line. The directory contains:
<SplitName>_<version>.rawfor every partition that has aSplitNameuki_<version>.efi, the unified kernel image of the diskSHA256SUMS, the digests of those files
For SplitName=root and --version=1.0, the files are root_1.0.raw,
uki_1.0.efi and SHA256SUMS.
An update is an error when no PARTITION above it has a SplitName.
Serve the directory over HTTP, or point a transfer file at it with a
file:// URL. examples/update has two transfer files that install the
update into plain files. A target that is a partition was not tried.
Errors
An error in an OUTPUT names the line, the kind and the file name, for
example Imagefile:22: OUTPUT iso os.iso: the image did not boot within 5m0s.
It also gives the reason. These are the ones from the OUTPUT line itself:
unknown kind of output--x: unknown option--elf: option takes no value--elf: needs an x86 bzImage, other kernels are unpacked already, on an arm64 image--path=x: path is not absolute--format=x: unknown file systemnot a plain file name, when the name has a/in itfile name taken, when two outputs have the same name
Checks
A CHECK boots an output and runs a command in the booted system. If the
command fails, the build fails and the output is not written. This is how
a build proves that the image it made starts.
OUTPUT disk os.raw
CHECK command -v htop
Which outputs can be checked
A CHECK belongs to the closest OUTPUT above it in the same stage. It can
check:
- a
disk - an
iso - a
rootfs
The other kinds are never booted. A CHECK after one of them is an error.
A CHECK with no OUTPUT above it is an error too.
The image has to run systemd. miso learns that the system has booted from systemd, and it runs the commands through a systemd service.
Checking a rootfs
A rootfs has no kernel, so miso boots the kernel and the initrd of the stage
with the rootfs as a virtual disk. The stage needs, above the OUTPUT rootfs:
- an
OUTPUT kernel - an
OUTPUT initrd - a
CMDLINEwith aroot=word
CMDLINE root=/dev/vda rootfstype=ext4 rw console=ttyS0
OUTPUT kernel vmlinuz
OUTPUT initrd initrd.img
OUTPUT rootfs os.ext4
CHECK test -b /dev/vda
The kernel and the initrd are fetched for the check even if you build without
-o. See examples/microvm.
How a check runs
- miso builds the output.
- miso boots it in a virtual machine with 2 CPUs and 2 GiB of memory. A disk is booted under UEFI. An ISO is booted as a CD. A rootfs is booted with its kernel and initrd.
- When the system has booted, miso runs each
CHECKof the output in the order of the file, each one in its own shell. - If every command exits with 0, the output is written under its name.
A disk is booted from a copy. Booting does not change the file that is written.
The command is run by /bin/sh of the image. It is taken as written, so
pipes, && and quotes work as in a shell. ENV values are not set in a
check.
Several checks after one output run in one boot. A check after a second
OUTPUT belongs to that one and boots it on its own.
Checks run during miso build with or without -o.
When a check fails
The build stops. The output file is not written. The checks after the failing one do not run.
miso: Imagefile:16: CHECK echo looking for emacs && command -v emacs: exit code 127
looking for emacs
to look at the layers before it: miso shell --before 16
The message names the line of the CHECK and the exit code of the command.
It shows what the command wrote. The last line is a miso shell command that
opens a shell on the layers before the check.
A passing check prints nothing.
If the system does not boot, the build fails with the image did not boot,
and with within 5m0s when it did not reach a booted state in five minutes.
The error names a log file in the cache directory. It holds the console
output of the boot, which shows where the boot stopped. On amd64 the kernel
writes it only when the command line names the serial console,
console=ttyS0. On arm64 the kernel finds the serial console itself, but a
console= that names another one, such as ttyS0, hides it. Leave it out
there, or write console=ttyAMA0.
What the machine needs
Checks run on the machine that runs miso, not in the builder VM. They need:
qemu-system-x86_64for an amd64 image orqemu-system-aarch64for an arm64 image, and access to/dev/kvm. An image for another architecture than the machine’s boots without KVM, which is much slower- the firmware, which miso downloads and keeps in the cache
- a vsock device. miso uses it to talk to the booted system. If there is
none, the build stops with
CHECK needs a vsock of its own.
Base images
FROM starts a stage from a base. miso knows these base images by name:
| Name | Image |
|---|---|
debian:sid | The daily Debian sid cloud image. |
debian:13 | The current Debian 13 (trixie) cloud image. |
debian:trixie | The same image as debian:13. |
The images are the genericcloud qcow2 images that Debian publishes at
cloud.debian.org, for amd64 and arm64. miso fetches the image for the
architecture the image is built for, see --arch. A name that is not in the table is an
error, and so are paths, URLs and container images.
FROM scratch starts from an empty root file system, and FROM <stage> starts
from an earlier stage of the same file. See The Imagefile.
When an image is fetched
The first build that needs a name downloads the image into the bases
directory of the cache, or into bases/arm64 for arm64.
Later builds use the cached image. A name keeps pointing at the same image for
as long as that image stays in the cache, even when Debian has published a
newer one.
miso plan shows the image a stage starts from as a digest after the FROM
line:
FROM debian:sid base sha256:6a9ef70a8ecb
A base image that has not been fetched yet is listed as download debian:sid.
Getting a newer image
The digest of the base image is part of the key of the first step of the stage. When the image changes, the keys of all the steps of the stage change, and its layers are built again.
miso prune removes downloads that were not used for a while. The next build
that needs a removed image downloads it again, and it may be a newer one. See
Command line.
Packages
The base images are cloud images. Install what a build needs with RUN, for
example RUN apt-get update && apt-get install -y htop. A RUN is cached by
its text, so the packages it installs are not looked up again on the next
build. See Caching.
Caching
miso keeps the result of the steps of a build and reuses it. A build that changes only the end of an Imagefile runs only the steps from the change on.
Cache keys
Every step has a cache key. miso plan lists the key of each step, and says
whether its layer is cached:
2 3e624b25c9ad cached RUN apt-get update && apt-get install -y ...
4 71d3dcea2f22 run RUN : > /etc/fstab
run is a step that a build would run. After a change, the changed step and
every step after it say run.
The key of a step is made from:
- the key of the step before it
- the instruction as written
- what the instruction reads. For a stage’s first step this is the digest of
the base image. For
COPYit is the content, the mode and the link target of every file it copies. ForCOPY --fromit is the key of the stage it copies from. - the version of miso
- the architecture the image is built for
A step whose key is in the cache is not run again. When a step changes, its key changes, and so do the keys of all the steps after it. The steps before it keep their keys and their layers.
Some details:
- Comments, blank lines, line numbers, the case of the keyword and the names of stages are not part of a key.
RUN --network=noneandRUNhave different keys.- An upgrade of miso changes every key, so the first build with a new version builds everything again.
Layers
RUN and COPY each produce a layer, which is the set of files the step
changed. ENV produces none. The layers are kept on the cache disk of the
builder VM.
PARTITION, CMDLINE, OUTPUT and CHECK do not change the root file
system. Changing them does not build the RUN and COPY steps before them
again.
A RUN is cached by its text
miso does not know what a RUN downloads. The key depends on the text of the
command, not on the result. A step such as
RUN apt-get update && apt-get install -y htop
keeps its layer when the Debian archive has new packages. To build it again,
change the text of the step, or let miso prune remove the layer.
Space
The layers are kept on the cache disk, and downloaded files, such as base images, are kept in the cache directory. Both are described under the cache.
miso pruneremoves what was not used for a while.miso resizemakes the cache disk larger. It can grow up to 50 GiB until you resize it.
miso shell does not change the cache.
Examples
The examples directory of the repository has one build for each kind of
output. Each directory holds an Imagefile with comments, and some hold files
that it copies in. Build one from the root of the repository:
miso build -o out examples/<name>
All of them start from debian:sid and build with the steps described on the
Imagefile page.
| Example | What it builds |
|---|---|
htop | A bootable disk with htop. |
iso | A bootable ISO. |
microvm | A root file system with a kernel and an initrd. |
portable | A portable service. |
sysext | A system extension. |
confext | A configuration extension. |
update | The files of an update for systemd-sysupdate. |
htop
A Debian disk with htop in it, with an EFI system partition and a root
partition. The CHECK boots the disk and looks for htop. The
quick start builds it.
Result: out/os.raw
iso
A Debian ISO that boots from a CD. The root is a read-only erofs file system
with an overlay in memory on top. The initrd is made again with dracut, so
that it can find a root on a CD. The CHECK boots the ISO as a CD.
It installs linux-image-amd64 and writes console=ttyS0, so it is for
amd64. On arm64 install linux-image-arm64 and write console=ttyAMA0.
Result: out/os.iso
microvm
A Debian root file system for a microVM, with the kernel and the initrd to
boot it. The initrd is made again with the driver of the virtual disk. The
CHECK boots the three files together. Like iso, it is for amd64.
Result: out/os.ext4, out/vmlinuz and out/initrd.img
The Imagefile has a qemu-system-x86_64 command that boots them. Firecracker
on x86 needs the kernel as an ELF file, so write OUTPUT kernel vmlinux --elf
instead of OUTPUT kernel vmlinuz. The Imagefile also has a Firecracker
configuration that boots the result to a login prompt.
portable
A portable service. It holds busybox and a unit named hello that prints a
line and sleeps.
Result: out/hello.raw
On a machine with systemd:
portablectl attach ./out/hello.raw
systemctl start hello
sysext
A system extension that adds one program, /usr/bin/hello. The stage starts
FROM scratch and copies busybox out of a stage that installs it, so the
image holds only what the extension adds. The release file is copied in from
the example directory.
Result: out/hello.raw
On a machine with systemd:
cp out/hello.raw /var/lib/extensions/
systemd-sysext merge
hello
confext
A configuration extension that adds /etc/hello.conf. It starts FROM scratch and copies in the file and the release file.
Result: out/hello.raw
On a machine with systemd:
cp out/hello.raw /var/lib/confexts/
systemd-confext merge
cat /etc/hello.conf
update
The files of an update for systemd-sysupdate, made from the partitions of
a disk. The root partition has SplitName=root, so it is shipped.
Result: the directory out/updates with root_1.0.raw, uki_1.0.efi and
SHA256SUMS.
The Imagefile has two transfer files that install the update into plain
files. They read the update from a file:// URL.