The screen says grub rescue and the server will not boot. What broke, how to fix it with chroot, and how to boot by hand. Captured on a lab VM.
You reboot the server, open the console, and there's no login. Just grub rescue> and a blinking cursor. SSH is dead, the site is down, and it feels like the whole system evaporated. It didn't. What broke is the bootloader, the little program that wakes up first and goes looking for your system. Fixing it is usually quick once you know the order.
In short. The prompt gives it away. Most often, with
grub rescue>the/boot/grubfolder, GRUB's modules included, has vanished. A plaingrub>means only the settings filegrub.cfgis missing. Case one: boot a rescue system, mount the disk,chrootinto it, rungrub-installon the whole disk, thenupdate-grub. Case two: start the system by hand from the prompt. I reproduced both on a virtual machine running Ubuntu 24.04 (BIOS boot, GRUB 2.12).
What GRUB is and what can break in it
Something has to wake up before Linux does, find the kernel and start it. That something is GRUB, a bootloader, and it works like a relay of three runners. If one drops the baton, the race is over.
Runner one sits in the boot sector (MBR), the first 512 bytes of the disk. Tiny, and it knows a single trick: where runner two lives. Runner two is core.img, tucked into the unused gap between the MBR and your first partition. It can read file systems, just enough to walk into /boot/grub on your partition. Runner three waits there: the modules (file system drivers, commands, the menu) and grub.cfg, the list of kernels you may start. That is the MBR layout of my lab VM. On a GPT disk the names differ (gpt1 instead of msdos1) and core.img sits in a BIOS boot partition; I didn't test that.
The cure follows from that. No /boot/grub folder? Reinstall the loader with grub-install. Only grub.cfg missing? update-grub rebuilds it. Swapping the two commands is the classic way to lose an evening, because grub-install puts the runners on the track while update-grub merely prints the list of kernels.
What's on the screen, and what it means
What you see | What is broken | What to do |
|---|---|---|
| GRUB cannot find its modules: the | Repair through |
The prompt |
| Boot by hand, then run |
The GRUB menu appears, then a kernel panic or | The loader is fine; the problem is the kernel, the initramfs image or the | A different topic. As a first step pick an older kernel under "Advanced options for Ubuntu" |
A black screen after "Booting from Hard Disk..." or "No bootable device" | An empty boot sector, or the BIOS is looking at the wrong disk | Check the boot order, then |
Break it on purpose and look
My lab was a virtual machine with a minimal Ubuntu 24.04, one disk, one partition, BIOS boot, GRUB 2.12. I deleted /boot/grub with rm -rf, which is how it usually happens in real life: somebody "tidies up old kernels in /boot" with a pattern that's a little too wide. Here is the next boot, copied from the VM console:
Read the error first. The loader did start from the boot sector, but there are no modules on disk to take it any further. ls lists the disk (hd0), its partition (hd0,msdos1) and a floppy drive (fd0) that doesn't exist. A second ls shows kernels and initrd images sitting right where they should, and no grub directory anywhere. insmod normal tries to load the menu module and trips over the same error.
Nothing is lost, and that's the part worth taking in. The kernels sit right there in /boot. Only the loader is gone, so the whole job is putting it back.
Restore GRUB with chroot, step by step
The plan: boot some other system, attach your disk, step inside it and install the loader as if you were logged into the server itself. That other system is a rescue mode, or a bootable ISO if your host offers one. In the my.weasel.cloud panel the service page has Console, Boot log and Rescue mode buttons. I haven't pressed Rescue mode for this article, so check how it behaves on your plan. What follows was captured on a VM where a second system boots next to the damaged disk.
Step 1. Find your disk
How your disk appears in a rescue system depends on the host; in my lab it was a second device. Ask for names, sizes and labels:
lsblk -o NAME,SIZE,FSTYPE,LABEL,MOUNTPOINTSRows like vda and vdb are disks, and vda1 and vdb1 are partitions on them. In my lab vda is the rescue system, the one with / mounted, and vdb is the patient. Yours could well be /dev/sdb. Trust the size and the label over the letter. The loader goes to whichever disk you name, and I'd rather you didn't learn that by overwriting the wrong one.
Step 2. Mount the partition and check it's the right one
Mount the partition with your system on /mnt, then look inside boot:
mount /dev/vdb1 /mntls /mnt/bootSeeing vmlinuz-... and initrd.img-... means you have the root of the right system. If your /boot is a separate little partition (lsblk will show it), mount that too, for instance mount /dev/vdb2 /mnt/boot with your own number. My lab gave this:
No grub folder, as predicted. The kernels are all there.
Step 3. Attach the system folders and step into the chroot
chroot makes a chosen folder the root, so programs inside see /mnt as /. That alone isn't enough, and the quickest way to see why is to try it:
chroot /mnt grub-install /dev/vdbBack came grub-install: error: cannot find a device for /boot/grub (is /dev mounted?), and the hint in brackets is right on the money. A disk that isn't running has empty /dev, /proc and /sys. A live kernel fills those, and the kernel you're running now isn't the one on that disk. So borrow the folders from the current system:
for d in dev proc sys; do mount --bind /$d /mnt/$d; donemount --bind simply shows an existing folder in a second place. Now /mnt sees the same devices you do. If root is on LVM, bind /run as well; my lab had no LVM and three folders were enough.
Step 4. Install the loader, rebuild the menu
Install onto the whole disk (vdb, not vdb1), then regenerate the settings:
chroot /mnt grub-install /dev/vdbchroot /mnt update-grubThe first command writes code into the boot sector, drops the core image into the gap behind it and recreates /boot/grub with all its modules. The second hunts for installed kernels and builds grub.cfg. The whole session, including the failed attempt:
That "UEFI Firmware Settings" line looks odd on a BIOS machine, but it appeared on mine too, so ignore it. What you're looking for is Installation finished. No error reported. and a /boot/grub that exists again with a grub.cfg inside. The last line, umount -R /mnt, releases everything we attached.
Some people like to work from the inside instead: chroot /mnt /bin/bash, the same commands without the chroot /mnt prefix, then exit. It ends up in the same place.
Step 5. Reboot
Leave rescue mode however the panel lets you and reboot. On my VM the GRUB menu popped up straight away with two entries, the kernel started on its own, and about a minute later the console said Ubuntu 24.04 LTS grublab tty1 and asked for a login. It was back.
A caveat, though. Some servers boot with UEFI (run ls /sys/firmware/efi in the rescue system; a folder there means UEFI), and for those the recipe changes: an EFI partition, a different grub-install call. I never tested that, so I'm not going to guess at details.
Boot by hand from the grub> prompt
Lose only grub.cfg and GRUB still loads all its modules, then leaves you at a full command line, grub>. You can start the system from there yourself. I deleted grub.cfg and looked around. I also typed the wrong path on my first try, which turned out to be the useful part:
My mistake was assuming /boot is its own partition. On a stock Ubuntu install it usually isn't: it's an ordinary folder on the root partition, so the kernel is at (hd0,msdos1)/boot/vmlinuz and not at (hd0,msdos1)/vmlinuz. The second error is only fallout. No kernel got loaded, so the initrd had nothing to hang on. Here are the commands that worked:
linux (hd0,msdos1)/boot/vmlinuz root=/dev/sda1initrd (hd0,msdos1)/boot/initrd.imgboot
linux loads the kernel and tells it through root= where the root file system is. initrd adds the starter image with the drivers needed to get going. boot fires it all off. When things go right the first two print nothing; after boot kernel messages scroll past, and about a minute later my VM showed a login prompt. /dev/sda1 is just what my virtual machine calls the disk. A VPS will more often say /dev/vda1, so check with ls and lsblk and don't copy mine blindly.
Once you're logged in, run sudo update-grub so you don't have to type all this at every power-on. And if more than grub.cfg got hurt, add sudo grub-install /dev/vda, using your own disk name.
What not to do
Don't hand grub-install a partition. grub-install /dev/vdb1 gets you grub-install: warning: File system `ext2' doesn't support embedding. and finally grub-install: error: will not proceed with blocklists. That's a refusal, and a sensible one: you named a partition and it wants a disk. Don't push past it with --force. That installs the loader anyway, on "block lists", which GRUB's own warning calls UNRELIABLE.
Don't expect update-grub to cure grub rescue>. It writes a menu, and the thing you lost was the loader.
Don't tidy /boot by hand. Old kernels go the clean way, with sudo apt autoremove --purge, and the menu gets refreshed on their way out. Deleting by hand, like my rm -rf /boot/grub, is how I ended up with the screen near the top of this page. Last one: stop and run lsblk before any grub-install, and read the disk name twice.
How to avoid a repeat
Three cheap habits help. Look at df -h / before a kernel update, since an update that runs out of space halfway can stop in the middle. Copy /etc/default/grub before you touch it and run sudo update-grub after. Leave /boot/grub/grub.cfg alone entirely; it's regenerated, so any edit of yours just disappears.
Once the server is up again, the previous boot's log often names the culprit. Try journalctl -b -1, if your journal survives reboots. How to read it is in the article on journalctl. If you're not sure which release you're on, here is how to check your Ubuntu or Debian version, and more of the usual breakages live in common Linux server problems.
FAQ
What is grub rescue and how do I get out of it?
It's GRUB's emergency prompt, the one you land in when the loader starts but can't find its modules. There's no magic command to leave it. insmod normal won't help (I tried, it's on the screen above), because the folder has to come back first. In practice that means a rescue system, a chroot and grub-install.
What is the difference between grub-install and update-grub?
Roughly, installing versus listing. grub-install puts the loader itself on the disk: boot sector code plus the modules in /boot/grub. update-grub only scans for kernels and writes grub.cfg. So a lost loader needs the first command, and a lost menu gets by on the second alone.
Will I lose data when I restore GRUB?
No. Both commands leave your files alone; they write to the boot sector and to /boot/grub, nowhere else. What can hurt is aiming grub-install at the wrong disk, which is why I keep going on about lsblk. And if something is irreplaceable, copy it off first anyway.
Do I need a console to fix this?
You do, since without a screen there's no GRUB prompt to read and no error to google. The service page in my.weasel.cloud has Console and Boot log buttons. The repair itself runs in a rescue system or from a boot disk.
What if the server boots with UEFI?
Then an EFI partition stands in for the boot sector, and grub-install takes different arguments. A folder at /sys/firmware/efi tells you which kind you have. I tested only the BIOS route described above.
Summary
grub rescue> most often means the modules in /boot/grub have gone missing, and a bare grub> means grub.cfg has. Your data is untouched either way; you're restoring a loader, not a system.
The order that worked for me: rescue system, lsblk, mount, bind dev proc sys, chroot /mnt grub-install on the disk, update-grub. Skip the bind mounts and you'll meet cannot find a device. Name a partition instead of a disk and GRUB refuses outright.
And if only the menu is gone, the grub> prompt lets you boot by hand: linux, initrd, boot, with /boot in the path.
🎁 40% off the first month of a new VPS. Promo code GUIDE40
Need a server for your tasks? The first month at a discount.
EnterGUIDE40at checkout. One use per customer.
Pick a VPS and claim the discount →
