Failed to execute init (error -2) kernel panic after latest update

Newbie Corner
, , ,
#1

I updated my laptop with yaourt -Syua this morning, and now after a reboot I'm getting a rather nasty kernel panic. I've tried all sorts of things to isolate the problem, but I've come up blank on all counts - so I'm forced to admit that I'm not able to fix it myself, and I need the help of this lovely community :slightly_smiling_face:

Firstly, a few details about my laptop and setup:

  • Make / model: Entroware Apollo with a Pentium 4415U, 8GB RAM, and a 120 GB SSD
  • Install method: manjaro-architect
  • Init system: OpenRC
  • Bootloader: rEFInd
  • Partition layout: /dev/sda (1: Default Ubuntu EFI partition, 2: Default Ubuntu OS partition, 3: Manjaro boot EFI partition, 4: Manjaro OS partition) - I dual-booted with the default ubuntu installation that came with the device, just in case. Haven't used it yet though.

More details available on request, obviously :stuck_out_tongue:

I've only been using Manjaro OpenRC for a few weeks, actually (since ~mid august?), so I'm rather new to this :slightly_smiling_face: I'm coming from ubuntu as a user who wants to see what the OpenRC side of the fence is like for educational purposes.

Now, to the error. Here's a photo of my screen:

20170918_202527

....and here's the bit (I think) is important typed out:

[Firmware Bug]: TSC_DEADLINE disabled due to Errata; please update microcode to version: 0x52 (or later)
Failed to execute /init (error -2)
Kernel panic - not syncing: No working init found. Try passing init= option to kernel. See Linux Documentation/admin-guide/init.rst
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.13.2-1-MANJARO #1
Hardware name: Entroware Apollo/Apollo, BIOS 1.05.05 04/27/2017

From what I'm reading, it can't find init. I've done the following:

  • Booted from the exact manjaro-architect flash drive I used to install manjaro in the first place and mounted / chrooted to the broken system. Once done I:
    • Checked that /usr/bin/init exists (it does) - I also found /usr/bin/init-openrc
    • Checked that the partition UID specified in /boot/refind_linux.conf matches the partition it should boot from (it did)
    • Ran mkinitcpio -p linux413 (succeeded, but upon a reboot the error did not change)
  • Tried editing the kernel parameters (F2 then F2 again in the rEFInd menu) and appended init=/usr/bin/init (didn't work, same error as before except it said /usr/bin/init in the place of /init
  • Tried editing the kernel parameters again and appended init=/bin/sh instead (didn't work, same as before with /bin/sh in replace of /init)

From the reading I've been doing, sysvinit doesn't appear to require the init= kernel parameter, so I'm not sure what up with that.

Lastly, the things I do remember from the upgrade process:

  • It replaced a bunch of openrc packages with equivalents from the AUR.
  • It installed openrc, libopenrc(something) (can't remember the whole name), and openrc-sysvinit from the AUR
  • There was a message about the init= kernel parameter that needed changing during the yaourt build process, but I can't remember what it said
  • The update completed successfully (after failing because I started yaourt as root).
  • There were no more upgrades available after the upgrade completed successfully.

...sorry for the wall of text! I'm hoping that by taking the time to write a detailed help request, someone will be able to help me better :slight_smile:

Can anyone tell me what's going on, and how I can go about isolating and fixing the issue please?
Thanks! :smiley_cat:

1 Like
Kernal Panic after upgrade
#2

There's your problem. OpenRC development and support has moved to Artix Linux:

Hmm... I thought installer support had been dropped by then...

There are threads (I seem to remember?) where this has been resolved already (I'll come back and edit when I find them, unless someone else gets there first). Depending on how attached you are to your current install, though, a reinstall might be the easiest way forwards.

#3

Ah, I see. Yes, I did see that post. I did, however, wish to have a few weeks to think it over before making a decision - especially since from what I can tell the migration path is less than smooth.

There are existing topics? Oops! I tried to find some with a few searches, but came up empty. Apparently I wasn't searching with the correct terms. or looking in the right places :stuck_out_tongue:

....maybe it was earlier than mid august, I can't remember.

#4

Looking forward to that.

I don't know how you got your OpenRC, but this date makes sense--the last .iso dropped on 4 August.

Installing using it continues to work just fine, but as you discovered, updating it doesn't, anymore.

And this appears to be true for more than one single reason.

#5

From your link, it looks like since the OpenRC packages were removed from the repos, the package manager tries to update some of those from AUR (and I don't think those are maintained any more). That probably is what causes the breakage.

1 Like
#6

That makes sense. The first breakage came via the AUR while I was running Manjaro unstable repos. The second happened even under stable and even just doing a pacman -Syyu.

Both of them have been beyond my ability to fix, and I didn't even think to ask for help with it, since the standard answer has been Get Thee to Artix.

#7

Since nobody wanted to support openrc on our end and from our team, we simply had to drop that init-system. To check if it is still kernel based, you may download one of our current v17.0.5 install media, which include all updates from release date.

#8

Another thing... depending your $esp is /boot or /boot/efi, check that refind conf has the correct kernel if there is any kernel update. I don't use refind (tested briefly - forgot mostly) but I find we need to change refind conf after each kernel update (some claim it does that automatically). Check also that boot initrd is not just intel-ucode.img

#9

Right. Might be also a problem ...

#10

Depends on how you use it. Refind has two modes: automatic and manual. In manual mode you specify your boot entry, and with it you need to update refind.conf manually.

In automatic mode you use the automatically generated refind_linux.conf that does not reside on esp, but on /boot. It gives you boot entries for all kernels in /boot, but has much more limited configuration options (for example, only one initrd entry, so microcode cannot be used).

#11

Ahso desu ne, hai. (oh...that's how it is). :smiley:
Arigato.

#12

DĹŤitashimashite

#13

Thanks for the great discussion, everyone!

Yeah, that's the feeling I'm getting, @cimarronline. So I need to start from the correct install media (the one released on 4th August I think?) -> chroot to broken system -> re-install openrc & dependencies from the package cache on the iso (not sure on how to do that) -> mark it as held though /etc/pacman.conf?

These links don't work - it says "Sorry, you don't have access to that topic!" (screenshot). I think I'm missing something?

Yeah, that's been my finding so far too, @ kernelKurtz. I don't feel that I can realistically migrate to Artix if my machine doesn't boot in the first place - I'd like a known-good state and a proper backup to restore to (haven't gotten around to that for my system files just yet - it'll be the first thing I do once I've sorted this out :stuck_out_tongue:) before I migrate.

Ok! What do I do with the install media once I've downloaded it @philm? I can chroot to the broken system with it IIRC, but I'm not sure what the next steps from there would be.

Yeah, I'm pretty sure I'm in automatic mode - I've got a /boot/refind_linux.conf file, and it automatically adds extra options to the menu when I plug in, say, a flash drive. I was looking for a command like update-grub that I could run, but I couldn't find anything when researching, so I've come to the conclusion that it isn't needed.

I believe I only have just the one kernel installed at the moment - the latest in the 4.13 series.

I don't understand this, @ Chrysostomus, sorry!

#14

I'd think you would still have the previous version of your openrc packages in your own pacman cache (/var/cache/pacman/pkg).

You'd have to look in /var/log/pacman.log, at the end of the file, to see what packages were replaced. Then chroot and install the earlier version from your cache. Probably start with the openrc and see what it does. To install from your cache, just

cd /var/cache/pacman/pkg

sudo pacman -U [packagename].pkg.tar.xz (for each package you want to replace.)

I think you can use tab to autocomplete the package name once you type the first few letters.

Yeah, a backup of system files would have really saved you here. I use Timeshift myself (it's in AUR).

1 Like
#15

I was responding to gohlip. He thanked me in japanese, so I responded with what I think is an appropriate response (according to Google translate :slight_smile:)

Refind generates boot entries at boot time, so yeah, no need for update-grub equilevant. Although I have considered writing one, so we could have automatically updating manual boot stanzas. This would allow for things like encrypted /boot (i think).

#16

Just guessing here, @sbrl, but I think you joined recently, and that topic is in the off-topic ghetto, where recent joins can't read.

Or some similar wanky social-engineering community-minded nonsense ...

#17

Ah....found it.. Japanese language can be tricky, Protocol and politeness level is difficult to navigate. But we foreigners generally get a pass. Trouble is, I don't look foreign. :laughing:

Can you help the OP find out in refind where the boot initrd is and the kernel?
Hopefully we can determine/eliminate that as a cause. Thanks.

1 Like
#18

@sbrl

While waiting for Chrysostomus, let's try to boot up using grub instead of refind.
Yes, I know you do not use grub in the OS, but it will work.

Boot up install media (17.0.1 and above).
Do not boot up to live OS, but press 'c' at the menu and we'll get to the grub prompt (grub>).
Using this grub prompt, we can then boot up our installed OS.

grub> search -f /boot/intel-ucode.img

Any output?
If none try

grub> search -f /intel-ucode.img

Copy down the output and remember if intel-ucode is in first command or the second. It means your $esp is either /boot/efi (first command output) or $esp is /boot (second command output).

Then do
(first command output)

grub> search -f /boot/intel-ucode.img --set=root

or (second command output)

grub> search -f /intel-ucode.img --set=root

Follow up in respective case either

grub> ls ($root)/boot/
grub> ls ($root)/

Copy down the kernel and intramfs listed.
Something like.... vmlinuz-4.9-x86_64 and initramfs-4.9-x86_64.img

If first command output,

grub> probe -u $root --set=abc
grub> linux /boot/vmlinuz-4.9-x86_64 root=UUID=$abc rw
grub> initrd /boot/initramfs-4.9-x86_64.img
grub> boot

If second command output,
find out what the Manjaro root partition is, (let's say it is /dev/sda3) and verify with

grub> cat (hd0,3)/etc/lsb-release

It should print out details of Manjaro distribution. If verified, continue with...

grub> linux /vmlinuz-4.9-x86_64 root=/dev/sda3 rw
grub> initrd /initramfs-4.9-x86_64.img
grub> boot

If/when booted, correct your refind conf; Chrysostomus can help (I cannot help in refind); or if you want to use grub, let us know and I can help you change your bootloader to grub.

ps: but can't you find out from livecd OS whether your refind conf is correct? Isn't that simpler?
And correct it from there? Again, I cannot help here with refind. Too complicated for me. :laughing:

#19

Thanks for the link! I got different translation when translating from finnish. Seems that I should have gone with that instead. The finnish expression "eipä kestä!" seems similar to Japanese custom. Literally translated it means "(matter) doesn't bear (thanking)" ~ there is no reason to thank for this.

@sbrl, I think it is probably better to just recover data and reinstall from scratch, since openrc is not supported anymore.

That init= parameter is usually not needed, so guessing the correct value is difficult. If you want to fix this system instead of reinstalling, here is what I would do:

  1. check pacman log to see what packages were updated
  2. find those packages in AUR. One of them has <package_name>.install file. This is the file that printed you the correct value of init= parameter. Reading those install files you should be able to find it.

Problem is, data recovery+ reinstall is much easier, especially if you have separate /home partition.

@gohlip, since @sbrl use the automatic mode refind appends that initrd value at boot time. Fixing this from refind is not realistic, unless it is about the init parameter.

@sbrl, you could still try to use init=/bin/bash, like you tried init=/bin/shelltile.

#20

Yep, that must be it. Guess I'll just have to wait then :stuck_out_tongue:

Thanks for the detailed instructions, @gohlip! That first command did the trick, but I still get the same kernel panic as before :confused:

Yeah, I guess. I'm just rather reluctant as I've spent a while configuring my Manjaro OpenRC setup already.

Ok, thanks for the tip, @Chrysostomus! I'm not rightly sure which ones I need to find in the AUR, though.
I have extracted a bunch of useful information via my flash drive though - you can find here. It contains: pacman.log - The pacman log from the broken system, pacman_cache.txt - A list of files in /var/cache/pacman/pkg/, and pacman_installed.txt - a list of packages currently installed on the broken system.

I did find though that the two packages I'm sure have installed from the AUR are openrc and openrc-sysvinit. If I download a snapshot of each of them from the AUR now, then I find that the openrc package doesn't currently contain a something.install file, but the openrc-sysvinit package does! It's called sysvinit.install, and contains an echo statement that says To boot using OpenRC append init=/usr/bin/init-openrc to your kernel bootline..

Sadly though, appending init=/usr/bin/init-openrc to my kernel boot parameters (via rEFInd or grub) fails with the same error, except it says

Failed to execute /init (error -2)
Kernel panic - not syncing: Requested init /usr/bin/init-openrc failed (error -2).

Instead :confused:

The two lines from the pacmann.log I've linked to above that are really important I think are

installed openrc-sysvinit (2.88-10)
upgraded openrc (0.26.3-1 -> 0.27.2-1)

...so in theory I should be able to remove openrc-sysvinit and downgrade openrc to 0.26.3-1 if someone still has a copy of that package file?

Sorry, tried that - it doesn't work, sadly! It comes up with the same error :frowning:

Right! I'm not sure that the versions I need are still there though - please see the file I've linked to above.

Indeed it would! I hadn't gotten as far as setting it up (I was going to do it that evening :slightly_frowning_face:) Next time.... :stuck_out_tongue:

Forum kindly sponsored by