Bootable Linux Sparse Virtual Disk Images

The following recipe will get you a bootable sparse disk image that is 20GB in size, but only takes up a minimal amount of disk space (about 4.5MB to start). This process is suitable for creating disk images for Linux virtual machines.

First step is to create your sparse file:

truncate example.img --size 20G

This sets the size to about 20 gigabytes, but in reality it is not taking up any space:

# ls --size --block-size=1 example.img

0 example.img

# stat --format='%s' example.img

21474836480

Since you probably want to install a bootloader in order to make this a bootable image, we are going to need a partition table. I prefer the parted command for this over our old friend fdisk, since parted is a bit easier to script.

# parted example.img mklabel gpt

This creates a GPT partition table in the first 2048 sectors of the image file. The default partition type is MBR, which is fine if you plan on staying under 2TB, and do not mind dealing with extended and logical partitions. I see little to be lost by using GPT, since it part of UEFI, and is backward compatible with legacy BIOS type systems.

# parted example.img print

Model:  (file)
Disk /tmp/example.img: 21.5GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start  End  Size  File system  Name  Flags

Checking on the size, we see that our image file takes up about 40 kbytes, despite still appearing to be 20 gigabytes in size.

# ls --size --block-size=1 example.img

40960 example.img

# stat --format='%s' example.img

21474836480 

Now we can add a partition. In this case I am only going to create one partition that uses all of the available space and I am going to give it the name "vm-root" to avoid confusion later.

# parted example.img mkpart primary 0% 100%

# parted example.img name 1 vm-root

# parted example.img print

Model:  (file)
Disk /tmp/example.img: 21.5GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name     Flags
 1      1049kB  21.5GB  21.5GB               vm-root

Now we format the partition; I generally use ext4 these days. There is a fairly significant limitation to the ext4 mkfs tooling that forces us to use loopback devices at this point.

It was my hope that mkfs would figure out the partition size on its own, or perhaps let me specify it as an argument. But all attempts to do that caused mkfs to overrun the partition boundaries and break the backup GPT partition table.

When you specify fs-size to the mkfs.ext4 command, what you are specifying is the usable space you want, not the actual size of the available volume. The mkfs.ext4 command gets the volume size from the kernel's block layer, and then juggles a lot of complex logic to figure out how much space needs to be burned for meta information like superblocks and inode tables.

I probably could have figured out the math for the fs-size argument and formatted the image file partition directly, but there are too many variables to make me feel like that is a good use of my time. That being said, it would be of nice if the mkfs tooling got a virtual image mode that either allowed you to specify the device size, or detected the partition size from the partition table.

We need to bind our image file and partition to a loopback device:

DEV=$(losetup --show --find --partscan example.img)

 And now we can format our new partition:

# mkfs.ext4 -F ${DEV}p1

mke2fs 1.45.6 (20-Mar-2020)
/dev/loop0p1 contains a ext4 file system
        created on Sun Feb 21 19:32:16 2021
Discarding device blocks: done                            
Creating filesystem with 5242368 4k blocks and 1310720 inodes
Filesystem UUID: ede5cf4d-f7f6-4747-8c7a-28d794f92b92
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

 Another size check shows that we are now up to about 4.5 megabytes.

# ls --size --block-size=1 example.img

4505600 example.img

# stat --format='%s' example.img

21474836480

Pretty good so far. Now we can mount our new volume and take a look around.

# mkdir img-mp

# mount ${DEV}p1 img-mp

# df img-mp

Filesystem     1K-blocks  Used Available Use% Mounted on
/dev/loop0p1    20509264 45080  19399328   1% /tmp/img/img-mp

And now for the cleanup:

# umount ${DEV}p1

# losetup -d ${DEV}

To make this a bootable image, mount your volume again and copy the Linux OS file tree of your choice into the mounted volume. Then use the grub-install command to install the boot loader.

# grub-install --modules=part_gpt --root-directory /tmp/img/img-mp ${DEV}

Since this is a sparse image, it will grow larger as more data is written to it until you hit the size limit, but it will not get smaller when data is deleted.

The simplest way to compact this image is to use the zerofree command to zero out the empty space, and then use your VM hypervisor's tools to do the rest, such as virt-sparsify or VirtualBox modifymedium ${FILENAME} --compact.