Can not boot Manjaro after kernel update

[tinu]

Hi Manjaro-Community,

I am facing technical trouble with my manjaro installation after installing a new kernel (4.19 LTS). After doing so and rebooting the system I got the following message on screen:

mount: /sys/firmware/efi/efivars: unknown filesystem type 'efivarfs'.
Warning: /lib/modules/4.14.93-rt53-MANJARO/modules.devname not found - ignoring
ERROR: resume: hibernation device 'UUID=...' not found
ERROR: device 'UUID=...' not found. Skipping fsck.
mount: /new_root: can't find UUID=...
You are now being dropped into an emergency shell.
sh: can't access tty: job control turned off
[rootfs ]#

From where I can not type anything. After a quick google search I found the following:

Boot issues, "hibernation device ‘UUID=***' not found"

After updating yesterday I can no longer boot and I am hit with this: mount: /sys/firmware/efi/efivars: unknown filesystem type 'efivars' ERROR: resume" hibernation device 'UUID=super-long-number' not found I have tried using boot-repair-disk but it did not work, this is my first time using uefi instead of bios so I am not familiar with what this could be.

resp.

Using livecd v17.0.1 (and above) as grub to boot OS with broken bootloader

Read this first. [Simple Configfile Method] will do for most cases like when windows updates overrides our bootloader or when we install another linux OS (intel-ucode related). [Kernel Boot Method] is for borked grub (grub-customizer or grub install failures or no grub) [Special Cases] is for separate boot partitions, multiple Manjaro's or with other Arch derivative OS's. In all UEFI cases, the [UEFI - Additional Commands] listed further below must be performed. If further help is needed, …

However following the "Simple First Start" instructions I end up at the above stated point again. Unfortunately the creator of the first mentioned topic did not say what exactly he was doing to resolve his issue and I seem not to find the "starting point" in the second mentioned topic for my specific problem.

Is there anybody around who faced the same issues and could tell me how he resolved them or someone who generally knows what to do in this case?

Big Thanks in advance for any help!

Quick background to my system, I am using a Lenovo dual-boot machine with Win10 alongside Manjaro. Some time ago I had a similar issue after an Win10 update and opened a topic, however this time after a Linux update. But for the sake of completeness:

Grub falls in rescue mode repeatedly after Win10 update

Hi Manjaro-Community, a few months ago I decided to create a Dual-Boot System with Manjaro and Win10 on my Notebook. So far I could manage all troubles I encountered on that way by myself, thanks to this forum and other resources on the internet. However since yesterday I face a problem which I could not manage to solve by myself so far unfortunally... let me explain: Win10 decided to run an update. After the installation I tried to reboot and Grub falls into rescue mode. Ok so far nothing spe…

[SGS]

As always, post output from
inxi -Fxxxz

tl;dr

Boot with your old kernel.

[tinu]

@SGS

As always, post output from
inxi -Fxxxz

I am not able to boot into my system, I am currently forced to use Win10.

@SGS

Boot with your old kernel.

1. How (pardon my lack of knowledge)?
2. What if I want to upgrade to the new kernel later on? I would face the same issue again I suppose?

[SGS]

@tinu

I am not able to boot

You can do that after boot from USB-stick.

[HowTo] Dual-boot Manjaro - Windows 10 - Step by Step

Dual boot - Step by Step Target systems Firmware Checklist Windows preparation Manjaro installation Revisions Target systems UEFI Computers with preinstalled Windows (Windows 10) is computers using UEFI firmware. This guide is a generic guide targeted at UEFI installations. However some of the guide does apply even if you are using a BIOS/MBR setup. DO NOT mix UEFI with MBR partition scheme. To ensure a successful dual-boot installation using Windows and Manjaro there ar…

[tinu]

@SGS

As always, post output from
inxi -Fxxxz

Here is the output:

Resuming in non X mode: xrandr not found. For package install advice run: inxi --recommends
System:    Host: manjaro Kernel: 4.9.50-1-MANJARO x86_64 bits: 64 gcc: 7.2.0
           Desktop: N/A info: xfce4-panel dm: lightdm Distro: Manjaro Linux
Machine:   Device: laptop System: LENOVO product: 20HGS0B900 v: ThinkPad T470s serial: N/A
           Mobo: LENOVO model: 20HGS0B900 serial: N/A
           UEFI: LENOVO v: N1WET35W (1.14 ) date: 07/12/2017
           Chassis: type: 10 serial: N/A
Battery    BAT0: charge: 4.2 Wh 20.8% condition: 20.2/23.5 Wh (86%)
           volts: 10.9/11.4
           model: SMP 00HW023 Li-poly serial: <filter>status: Discharging cycles: 573
           BAT1: charge: 19.7 Wh 99.6% condition: 19.8/26.1 Wh (76%)
           volts: 12.7/11.5
           model: SMP 01AV406 Li-poly serial: <filter>status: N/A cycles: 527
CPU:       Dual core Intel Core i7-7600U (-HT-MCP-) 
           arch: Kaby Lake rev.9 cache: 4096 KB
           flags: (lm nx sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx) bmips: 11620
           clock speeds: min/max: 400/3900 MHz 1: 1165 MHz 2: 1206 MHz
           3: 1183 MHz 4: 1142 MHz
Graphics:  Card: Intel HD Graphics 620 bus-ID: 00:02.0 chip-ID: 8086:5916
           Display Server: N/A drivers: intel (unloaded: modesetting)
           tty size: 80x24
Audio:     Card Intel Device 9d71
           driver: snd_hda_intel bus-ID: 00:1f.3 chip-ID: 8086:9d71
           Sound: Advanced Linux Sound Architecture v: k4.9.50-1-MANJARO
Network:   Card-1: Intel Ethernet Connection (4) I219-LM
           driver: e1000e v: 3.2.6-k bus-ID: 00:1f.6 chip-ID: 8086:15d7
           IF: enp0s31f6 state: down mac: <filter>
           Card-2: Intel Wireless 8265 / 8275
           driver: iwlwifi bus-ID: 3a:00.0 chip-ID: 8086:24fd
           IF: wlp58s0 state: up mac: <filter>
Drives:    HDD Total Size: 542.9GB (1.5% used)
           ID-1: /dev/nvme0n1 model: LENSE20512GMSP34MEAT2TA size: 512.1GB
           serial: <filter> firmware: 1.9.8341
           ID-2: USB /dev/sda model: Ultra size: 30.8GB serial: <filter>
Partition: ID-1: / size: 15G used: 56M (1%) fs: overlay dev: N/A
           ID-2: swap-1 size: 8.59GB used: 0.00GB (0%)
           fs: swap dev: /dev/nvme0n1p8
Sensors:   System Temperatures: cpu: 40.0C mobo: N/A
           Fan Speeds (in rpm): cpu: 0
Info:      Processes: 187 Uptime: 3 min Memory: 1117.3/19832.0MB
           Init: systemd v: 234 Gcc sys: 7.2.0
           Client: Shell (bash 4.4.121 running in xfce4-terminal) inxi: 2.3.38 

[gohlip]

@tinu

However following the “Simple First Start” instructions I end up at the above stated point again.

Then use the "More Complicated Method" just below the "Simple First Start".
At Step 6 there, boot with the older kernel instead of 4.19.

You can get what older kernel you have at end of step 5

grub> ls ($root)/boot/

After booting up, do aslo

sudo pacman -Syyuu
sudo mhwd-kernel -r linux419
sudo mhwd-kerenl -i linux420

By the way, did you do the major upgrade just before you have this problem?
If you did, it may not be the kernel update that's giving you this problem.
If this is the case, you will need to chroot into the OS and do

sudo pacman -Syyuu

Good luck.

[SGS]

@tinu

Kernel: 4.9.50-1

Damn, he is so quick

From USB-stick, they use this old Kernel?

[tinu]

@gohlip

Then use the “More Complicated Method” just below the “Simple First Start”.
At Step 6 there, boot with the older kernel instead of 4.19.

You can get what older kernel you have at end of step 5

Following your instructions brings me here:

mount: /sys/firmware/efi/efivars: unknown filesystem type 'efivarfs'.
:: running early hook [udev]
Warning: /lib/modules/4.14.93-rt53-MANJARO/modules.devname not found - ignoring
starting version 239
:: running hook [udev]
:: Triggering uevents...
:: running hook [keymap]
:: Loading keymap...done.
:: running hook [resume]
ERROR: resume: no device specified for hibernation
ERROR: device 'b497...' not found. skipping fsck.
:: mounting 'b497...' on real root
mount: /new_root: no filesystem type specified
You are now being dropped into an emergency shell.
sh: can't access tty: job control turned off
[rootfs ]#

@SGS

From USB-stick, they use this old Kernel?

Yes, it's the kernel of the USB-stick. I used 4.14 previously on my machine.

[SGS]

OT

Zusammenfassung

@tinu

Yes, it’s the kernel of the USB-stick. I used 4.14 previously on my machine.

4.9.152-1 last LTS not 4.9.50-1maybe your USB has old download version, no matter, look that you can fix your problem, much luck

[tinu]

@gohlip

By the way, did you do the major upgrade just before you have this problem?
If you did, it may not be the kernel update that’s giving you this problem.
If this is the case, you will need to chroot into the OS and do

Following item 4 in this topic on how to chroot in my system:

Can't login - black screen & chroot guide

If you are stuck and you can't log into your system, use one of these steps below. They are sorted from easiest at the top. Start with the first step and if it doesn't work, try with the next one and so on.. Once you get access to the system, then proceed with updating the system or installing/removing packages, checking log files... Steps (you only need one) are: 1 - log to tty terminal 2 - boot without X 3 - chroot auto (with manjaro-chroot) 4 - chroot (with manjaro-chroot) 5 - chroot ol…

I'll receive the following message:

[manjaro@manjaro ~]$ lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
loop0         7:0    0  58,7M  1 loop /run/miso/sfs/livefs
loop1         7:1    0   239M  1 loop /run/miso/sfs/mhwdfs
loop2         7:2    0 874,1M  1 loop /run/miso/sfs/desktopfs
loop3         7:3    0 399,4M  1 loop /run/miso/sfs/rootfs
sda           8:0    1  28,7G  0 disk 
├─sda1        8:1    1   1,6G  0 part /run/miso/bootmnt
└─sda2        8:2    1     4M  0 part 
nvme0n1     259:0    0   477G  0 disk 
├─nvme0n1p1 259:1    0   450M  0 part 
├─nvme0n1p2 259:2    0    99M  0 part 
├─nvme0n1p3 259:3    0    16M  0 part 
├─nvme0n1p4 259:4    0 250,2G  0 part 
├─nvme0n1p5 259:5    0   504M  0 part 
├─nvme0n1p6 259:6    0 117,7G  0 part 
├─nvme0n1p7 259:7    0   100G  0 part 
└─nvme0n1p8 259:8    0     8G  0 part [SWAP] 
[manjaro@manjaro ~]$ sudo mount /dev/nvme0n1p6 /mnt
[manjaro@manjaro ~]$ sudo mount /dev/nvme0n1p7 /mnt/home
[manjaro@manjaro ~]$ sudo mount /mnt/boot/efi
mount: /mnt/boot/efi: konnte nicht in /etc/fstab gefunden werden.

Additional Info: nvme0n1p4 is my Win10 partition and gparted tells me nvme0n1p2 is the EFI system partition.

[gohlip]

@tinu

1. Your boots are no longer in uefi.
   Your failed boot and your livecd boot are not booted in uefi.
   Ensure your bios is set to boot in uefi.
2. Your chroot was done wrong.
   My procedure on chroot is the 'long' one.
   I have not used manjaro-chroot or other simple methods.
   Note I rarely chroot and I cannot recall the last time I did it.
   But you must chroot properly and exit when processes in chroot are done.
   Do not do other things while in chroot. We can really mess that up and a reinstall may be only way out.

Let's start all over again.
And according to you, your partitions are
/dev/nvme0n1p6 is /
/dev/nvme0n1p2 is /boot/efi
/dev/nvme0n1p7 is /home

Please recheck that above is correct and that they are in the fstab of your installed OS.

1. Boot up livecd in uefi mode.

test -d /sys/firmware/efi && echo UEFI || echo BIOS

must return the value of "UEFI"

Taking care the said partitions are not mounted at livecd.

2. At livecd terminal

sudo pacman-mirrors -f3
sudo pacman -Syyuu

3. Chroot into installed manjaro partitions

sudo mount /dev/nvme0n1p6 /mnt
sudo mount /dev/nvme0n1p7 /mnt/home
sudo mount /dev/nvme0n1p2 /mnt/boot/efi
mount -t proc proc /mnt/proc
mount -t sysfs sys /mnt/sys
mount -o bind /dev /mnt/dev
mount -t devpts pts /mnt/dev/pts/
chroot /mnt

4. Now you're chrooted and perform these commands.

sudo pacman-mirrors -f3
sudo pacman -Syyuu
sudo mhwd-kernel -r linux419
sudo mhwd-kernel -i linux420
sudo mkinitcpio -P
sudo grub-install
sudo update-grub
sudo cp /boot/grub/x86_64-efi/core.efi /boot/efi/EFI/boot/bootx64.efi
sudo efibootmgr -c -d /dev/nvme0n1 -p 2 -L "manjaro" -l "\EFI\Manjaro\grubx64.efi"

5. Do not do anything else and exit

exit

6. Reboot.

Good luck.
But if you still are having problems, I suggest you reinstall.

[gohlip]

@tinu, ignore this post. I want to talk to @AgentS openly.

@AgentS
Should I include these in?
Notice that the existing /etc/default/grub will not be overidden
Nor grubenv be cleared if these were not done.

But the OP's earlier errors (not uefi, wrong chroot) were obvious and most likely the cause.
But this hidden menu thingy can trip him up as well.

In /etc/default/grub, make sure you have one of the following specified.

GRUB_TIMEOUT_STYLE=menu
GRUB_TIMEOUT_STYLE=hidden
GRUB_TIMEOUT_STYLE=countdown

GRUB_TIMEOUT=5 (or more)

At terminal,

sudo rm /boot/grub/grubenv
sudo grub-editenv /boot/grub/grubenv create
sudo grub-editenv - set boot_success=0

Thanks for looking it over.

@tinu, if you reached grub and having black screen or problems booting, just 'press 'esc' . If still fail, reboot and repeat process - twice.

[tinu]

@gohlip

1. Boot up livecd in uefi mode.

test -d /sys/firmware/efi && echo UEFI || echo BIOS

must return the value of “UEFI”

It did.

@gohlip

2. At livecd terminal

sudo pacman-mirrors -f3
sudo pacman -Syyuu

Worked as well.

@gohlip

3. Chroot into installed manjaro partitions

sudo mount /dev/nvme0n1p6 /mnt

Returned:

[manjaro@manjaro ~]$ sudo mount /dev/nvme0n1p6 /mnt
mount: /mnt: unbekannter Dateisystemtyp „ext4“.

Well, I am not an expert but linux not recognizing ext4 systems looks kinda weird?! Please, correct me if I am wrong.

[AgentS]

@gohlip

Should I include these in?

Do you mean in the tutorial? I think they are included (I 've just checked).

@gohlip

most likely the cause

I am still not sure what is the actual cause for OP, maybe it's a not-updated-for-long-time issue, needing to merge .pacsave for /etc/default/grub?

OP may need to unhide grub and select an older kernel as @sgs suggested.

[gohlip]

@tinu

[manjaro@manjaro ~]$ sudo mount /dev/nvme0n1p6 /mnt mount: /mnt: unbekannter Dateisystemtyp „ext4“.

Yes, it is weird. /dev/nvme0n1p6 is your root partition right?
And what format is it?
Can you provide output of

LANG=C
sudo parted -l

[AgentS]

@tinu

linux not recognizing ext4 systems looks kinda weird?! Please, correct me if I am wrong.

I agree it is strange.
Please post

sudo inxi -Dpuoxxxz

[tinu]

@gohlip

Can you provide output of

LANG=C
sudo parted -l

[manjaro@manjaro ~]$ sudo parted -l
Model: SanDisk Ultra (scsi)
Disk /dev/sda: 30.8GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags: 

Number  Start   End     Size    Type     File system  Flags
 1      32.8kB  1706MB  1706MB  primary               boot
 2      1706MB  1710MB  4194kB  primary               esp


Model: Unknown (unknown)
Disk /dev/nvme0n1: 512GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 

Number  Start   End    Size    File system     Name                          Flags
 1      1049kB  473MB  472MB   ntfs            Basic data partition          hidden, diag
 2      473MB   577MB  104MB   fat32           EFI system partition          boot, esp
 3      577MB   593MB  16.8MB                  Microsoft reserved partition  msftres
 4      593MB   269GB  269GB   ntfs            Basic data partition          msftdata
 5      269GB   270GB  528MB   ntfs                                          hidden, diag
 6      270GB   396GB  126GB   ext4
 7      396GB   504GB  107GB   ext4
 8      504GB   512GB  8594MB  linux-swap(v1)

@AgentS

Please post

sudo inxi -Dpuoxxxz

[manjaro@manjaro ~]$ sudo inxi -Dpuoxxxz
Use of uninitialized value $speed in pattern match (m//) at /usr/bin/inxi line 8948.
Drives:    Local Storage: total: 505.58 GiB used: 6.85 GiB (1.4%) 
           ID-1: /dev/nvme0n1 model: LENSE20512GMSP34MEAT2TA size: 476.94 GiB serial: <filter> rev: 1.9.8341 scheme: GPT 
           ID-2: /dev/sda type: USB vendor: SanDisk model: Ultra size: 28.64 GiB serial: <filter> rev: 1.00 scheme: MBR 
Partition: ID-1: / size: 14.53 GiB used: 6.85 GiB (47.1%) fs: overlay source: ERR-102 uuid: N/A 
           ID-2: /run/miso/bootmnt raw size: 1.59 GiB size: N/A (hidden?) used: N/A (hidden?) fs: iso9660 dev: /dev/sda1 
           uuid: 2017-09-16-10-13-30-00 
           ID-3: /run/miso/sfs/desktopfs raw size: 874.1 MiB size: N/A (hidden?) used: N/A (hidden?) fs: squashfs 
           dev: /dev/loop2 uuid: N/A 
           ID-4: /run/miso/sfs/livefs raw size: 58.7 MiB size: N/A (hidden?) used: N/A (hidden?) fs: squashfs dev: /dev/loop0 
           uuid: N/A 
           ID-5: /run/miso/sfs/mhwdfs raw size: 239.0 MiB size: N/A (hidden?) used: N/A (hidden?) fs: squashfs dev: /dev/loop1 
           uuid: N/A 
           ID-6: /run/miso/sfs/rootfs raw size: 399.4 MiB size: N/A (hidden?) used: N/A (hidden?) fs: squashfs dev: /dev/loop3 
           uuid: N/A 
           ID-7: swap-1 size: 8.00 GiB used: 0 KiB (0.0%) fs: swap dev: /dev/nvme0n1p8 
           uuid: 86e0f0b3-375b-4e5d-8212-0cba26cf09de 
Unmounted: ID-1: /dev/nvme0n1p1 size: 450.0 MiB fs: ntfs label: Wiederherstellung uuid: 08D03BECD03BDE9E 
           ID-2: /dev/nvme0n1p2 size: 99.0 MiB fs: vfat label: N/A uuid: 7A3C-3A58 
           ID-3: /dev/nvme0n1p3 size: 16.0 MiB fs: N/A label: N/A uuid: N/A 
           ID-4: /dev/nvme0n1p4 size: 250.24 GiB fs: ntfs label: N/A uuid: 7A50448E50445355 
           ID-5: /dev/nvme0n1p5 size: 504.0 MiB fs: ntfs label: N/A uuid: 54F64430F6441520 
           ID-6: /dev/nvme0n1p6 size: 117.65 GiB fs: ext4 label: N/A uuid: b4973293-753a-4b3e-8221-06ddcdaec015 
           ID-7: /dev/nvme0n1p7 size: 100.00 GiB fs: ext4 label: N/A uuid: da2358a0-60be-4e86-9cf7-702be0f0b841 
           ID-8: /dev/sda2 size: 4.0 MiB fs: vfat label: MISO_EFI uuid: 7723-A0BA 

[gohlip]

Now,

@gohlip

Taking care the said partitions are not mounted at livecd.

make sure the partitions /dev/nvme0n1p6 /dev/nvme0n1p2 and /dev/nvme0n1p7 are not mounted first before doing (this is mounting them, so they cannot be mounted before that).

sudo mount /dev/nvme0n1p6 /mnt
sudo mount /dev/nvme0n1p7 /mnt/home
sudo mount /dev/nvme0n1p2 /mnt/boot/efi

maybe that's only that. try again.
Oh, please preface with "LANG=C" before typing commands. Thanks.

[AgentS]

@tinu

2. At livecd terminal

sudo pacman-mirrors -f3
sudo pacman -Syyuu

Worked as well.

Did you run the rest of the following commands (like (un)install kernel)?

Are you able to see the grub menu, when booting to your installed Manjaro (not-LiveISO)?
If yes, can you boot to the other kernels, like 4.19 you had just installed?
From the begining it seems you boot to your 4.14.rt.kernel, not to 4.19...

[tinu]

@AgentS

Did you run the rest of the following commands (like (un)install kernel)?

No, I stopped after mounting of /dev/nvme0n1p6 failed.

@AgentS

Are you able to see the grub menu, when booting to your installed Manjaro (not-LiveISO)?
If yes, can you boot to the other kernels, like 4.19 you had just installed?

Yes, I am able to see the grub menu, all I can select is "Manjaro Linux", some kind of secure/advanced mode and Windows. I don't know which kernel it is, it just says "Manjaro Linux". Just before the problem arose I installed the newer kernel under manjaro settings and restarted the system. (Pardon my lack of knowledge, but this intuitively felt like the way to do it.)

Interestingly I am able to mount the partitions without prior updating the live stick (i.e. omitting "sudo pacman -Syyuu"). Note: I did not continued to chroot into the system, should I try?