↓Skip to main content

Linux Rescue: Fix broken initial RAM disk files & Bootloader recovery

822 words·
Linux Linux Rescue GRUB Initramfs Initial RAMdisk error Bootloader-Recovery Ubuntu
Table of Contents

Recently I stumbled into a broken initrd file after the reboot of an Ubuntu server. After practicing this situation on my VMware playground, here is my tutorial for both restoring a broken initrd file and a bootloader misconfiguration.

Update Initramfs
#

Prerequisites
#

In order to illustrate the renewal of the intital RAM disk files, I’m going to delete the files first which makes it impossible to mount the filesystem at the system startup.

The /boot directory should look like follows:

drwxr-xr-x  4 root root      4096 Mar 20 20:24 .
drwxr-xr-x 19 root root      4096 Nov 28 18:19 ..
-rw-r--r--  1 root root    261842 Jan  5 11:07 config-5.15.0-58-generic
-rw-r--r--  1 root root    261876 Jan 20 13:42 config-5.15.0-60-generic
drwxr-xr-x  5 root root      4096 Feb 14 07:14 grub
lrwxrwxrwx  1 root root        28 Feb 14 07:13 initrd.img -> initrd.img-5.15.0-60-generic
-rw-r--r--  1 root root 107349895 Feb 14 07:15 initrd.img-5.15.0-58-generic
-rw-r--r--  1 root root 109265198 Mar 20 20:24 initrd.img-5.15.0-58-generic
lrwxrwxrwx  1 root root        28 Feb 14 07:13 initrd.img.old -> initrd.img-5.15.0-58-generic
drwx------  2 root root     16384 Nov 28 18:15 lost+found
-rw-------  1 root root   6250921 Jan  5 11:07 System.map-5.15.0-58-generic
-rw-------  1 root root   6252560 Jan 20 13:42 System.map-5.15.0-60-generic
lrwxrwxrwx  1 root root        25 Feb 14 07:13 vmlinuz -> vmlinuz-5.15.0-60-generic
-rw-------  1 root root  11550848 Jan  5 11:15 vmlinuz-5.15.0-58-generic
-rw-------  1 root root  11554504 Jan 20 14:16 vmlinuz-5.15.0-60-generic
lrwxrwxrwx  1 root root        25 Feb 14 07:13 vmlinuz.old -> vmlinuz-5.15.0-58-generic

Remove the initramfs files, in Ubuntu they are stored in files called initrd:
rm initrd.img-5.15.0-58-generic initrd.img-5.15.0-58-generic

After the reboot the system get’s stuck with the following output: 

error: file `/boot/initrd.img-5.15.0-71-generic' not foud.
Press any key to continue...
Unable to mount root fs on unknown-block...

In order to create new initial RAM disk files we use an Ubuntu live system.

Boot from Disk
#

  • Boot into BIOS. In VMware this can be done with F2 right after the system starts.

  • Change the BIOS settings to boot from CD-Rom Drive / ISO Image.
    Choose an ISO live image from the same Linux distribution, in my case it’s “ubuntu-22.04.1-live-server-amd64.iso”.

  • Save the BIOS settings and reboot

Open Rescue Shell
#

  • Start the Linux setup and move forward beyond the keyboard layout selection, otherwise the standard / US layout is selected.
  • Press "Strg" + "Alt" + "F1" till the “Enter shell” submenu is available.

Mount partitions
#

This is the partiton mount layout of the original system from which I’m going to to mount the / and /boot partitons.

sda                         8:0    0    20G  0 disk
├─sda1                      8:1    0     1M  0 part
├─sda2                      8:2    0   1.8G  0 part /boot
└─sda3                      8:3    0  18.2G  0 part
  └─ubuntu--vg-ubuntu--lv 253:0    0    10G  0 lvm  /

Find partitions Use fdisk -l to list the available disks and their partitions, the output should look like this:

Device       Start      End  Sectors  Size Type
/dev/sda1     2048     4095     2048    1M BIOS boot
/dev/sda2     4096  3719167  3715072  1.8G Linux filesystem
/dev/sda3  3719168 41940991 38221824 18.2G Linux filesystem

Disk /dev/mapper/ubuntu--vg-ubuntu--lv: 10 GiB, 10737418240 bytes, 20971520 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Boot partiton In my case /dev/sda2 is the boot partition, but to be sure mount the partition with mount /dev/sda2 /mnt and check the output with ls /mnt. Unmount the boot partition with umount /mnt.

Root partition Use lvs -o +lv_path to find the LVM logical volume of the root partition and also output the path to mount it. The output should look like this:

  LV        VG         Path
  ubuntu-lv ubuntu-vg  /dev/ubuntu-vg/ubuntu-lv

Mount the partitions

mkdir /mnt/boot # Create directory for boot partition

mount /dev/ubuntu-vg/ubuntu-lv /mnt # Mount root partition
mount /dev/sda2 /mnt/boot/ # Mount boot partition

# Bind the following directories from the rescue system
mount --bind /dev /mnt/dev
mount --bind /proc /mnt/proc
mount --bind /sys /mnt/sys

Update or create Initramfs
# 

# Change the root system with
chroot /mnt
#or
chroot /mnt bash


# Create new initialize ram files:
update-initramfs -k all -c

# Or update them in case they were broken:
update-initramfs -k all -u


# Exit the chroot command
exit

# Reboot
reboot

Use man update-initramfs to get more details about the command.


Bootload Recovery
#

Prerequisites
#

In order to illustrate a bootloader recovery I’m going to intentionally create a bootloader failure.

Edit the boot loader configuration file with vi /boot/grub/grub.cfg and comment out all “menuentry” sections:

#menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os ...{
#...
#}

After the reboot the system get’s stuck with the following output: 

GNU GRUB version 2.06
grub>_

Previous steps
#

Proceed with the previous steps “Boot from Disk”, “Open Rescue Shell” and “Mount partitions” from the Update Initramfs tutorial.

Repair Bootloader
#

Change the root system with:
chroot /mnt or chroot /mnt bash

Reinstall the Bootloder with the following command:
grub-install /dev/sda

Create a new Bootloader configuration:
grub-mkconfig -o /boot/grub/grub.cfg

Use exit to exit the chroot command and reboot the system.