An issue with DKMS and kernel initramfs images in Fedora 44

I have a somewhat convoluted kernel update process on my desktops, and in particular I run the development version of ZFS and every time I update my kernel, I first update ZFS. Lately, this overall update process has been achingly slow, slow enough to make me even more reluctant than usual to do kernel updates. Today, I grumbled about it on the Fediverse, which in the usual way of life led to me discovering the problem

The answer to where the time goes [in my kernel update process] is that 'dkms remove' rebuilds all initramfs images for all installed kernels, even if the module you're removing doesn't exist for those kernels and even if you are running 'dkms remove ... -k ...' to only remove the module for one kernel.

This is my unhappy face. I do not want random initramfs's to be rebuilt by surprise, and it also takes rather a while when you have six kernels, like I do.

(This turns out to be an incomplete diagnosis; 'dkms install' also does this.)

I handle the ZFS kernel modules through DKMS, and because of how I update ZFS and the kernel one after the other, I normally only have ZFS modules built for two kernels (the two most recent ones). When the ZFS DKMS RPM is updated, it removes all existing ZFS modules but only rebuilds the new version of the module for your current kernel, and then I install a new kernel and the overall kernel install process builds the ZFS module for the new kernel.

The actual 'dnf update' process of installing a new kernel takes a reasonable amount of time, including the DKMS build of kernel modules. Based on timestamps in /boot (and other evidence uncovered later), this process only touches (well, creates) the initramfs image for the new kernel, which is what I'd expect.

However, doing a 'dnf update' of the ZFS RPM packages is very slow, as is removing a module from a specific kernel version with 'dkms remove ... -k ...'. The easy one to observe is that the 'dkms remove' results in all of the /boot initramfs images having new timestamps (generally a couple of minutes apart). This turns out to be a general issue with both 'dkms remove' and 'dkms install', which is why updating the ZFS RPM packages is so slow (it does both, first removing the ZFS modules from the old package version and then installing the ZFS modules from the new one). On my work desktop, it takes DKMS about ten minutes or more to update all of the initramfs images; when DKMS does it twice during the upgrade of 'zfs-dkms', that's more than a twenty minute process (no wonder it felt so slow).

There's no need to do this initramfs image update for all kernels on removal, because if DKMS hasn't built a module for a kernel version, that module can't possibly be in that kernel version's initramfs image and so there's no need to touch the initramfs image on module removal. DKMS knows this for sure in the 'dkms remove ... -k ...' case, because it's being specifically told to only work on one kernel. Module installs are similar; unless DKMS is installing a module for every kernel, only the kernel the module is being installed for can possibly need an initramfs image change.

(On top of that, my ZFS module isn't in the kernel initramfs image, but DKMS probably can't easily see that.)

Digging into the details of Fedora 44's DKMS setup has led me to what is going on. A bunch of of DKMS behavior is controlled by settings in /etc/dkms/framework.conf, and on Fedora 44 one of these settings is:

post_transaction="dracut --regenerate-all --force"

As covered in the DKMS manual page, this is run after an assortment of DKMS commands such as 'dkms install' and 'dkms remove'. Based on the dracut(8) manual page, the arguments mean what they look like; all initramfs images for installed kernels will be regenerated (and because there's no '--parallel' provided, this regeneration will be done serially, one initramfs image at a time).

This post transaction action explains why 'dkms remove' winds up regenerating all initramfs images, even for 'dkms remove -k'. DKMS and dracut are doing exactly what they were told to do, and there's nothing in there to have DKMS and Dracut be more selective about which initramfs image to update. The situation with DNF installation of new kernels is more tangled, but having read the DKMS source code (ie, looked through the giant dkms shell script), I understand why it doesn't update all of the initramfs images in /boot.

On Fedora 44, DKMS hooks into kernel installs through /usr/lib/kernel/install.d/40-dkms.install. When a new kernel is installed, this runs the special, undocumented DKMS subcommand 'kernel_postinst':

dkms kernel_postinst --kernelver "$KERNEL_VERSION"

In DKMS, this basically does a 'dkms autoinstall' (which is documented), but the code in 'dkms' specifically doesn't run the post transaction command for this subcommand (although if you run the regular 'dkms autoinstall', dkms does). So when I install kernels specifically, the 'dkms kernel_postinst ...' runs and doesn't update any initramfs images (although the new kernel itself will use Dracut to create its normal initramfs image).

I suspect that Fedora would be uninterested in a bug report about this situation, since they explicitly put that 'dracut --regenerate-all' into their build of DKMS. It's in the Fedora dkms.spec and was added in the 3.2.0 commit.

Since I don't use DKMS to handle any kernel modules that I want in the initramfs, what I should probably do is create a file in /etc/dkms/framework.conf.d that overrides Fedora's 'post_transaction' setting and makes it either null or '/bin/true'. This should make DKMS effectively do nothing for the post transaction action and leave all of the initramfs images alone, making all versions of 'dkms remove' fast again.