Surface Pro 4 Win10 + Manjaro on uSD

Technical Issues and Assistance
, , , ,
#50

Alright, let's get to it:

Test C

grub> insmod cryptodisk
grub> insmod luks
grub> insmod gcry_rijndael
grub> insmod gcry_rijndael
grub> insmod gcry_sha256

grub> cryptomount hd0,gpt2
Attempting to decrypt master key...
Enter passphrase for hd0,gpt2 (d4c...754) #that's the correct UUID

Aha! "One hour later..."
Got impatient, pressed some buttons (enter, esc, ctrl+c) looked away, back in an unresponsive Live Boot grub menu. Hard reboot and try again...

...

...
grub> cryptomount hd0,gpt2
Attempting to decrypt master key...
Enter passphrase for hd0,gpt2 (d4c...754) #that's the correct UUID

Still slow, going to start work now and I'll keep an eye on it, just gonna let it sit for a while.

I'll update this post later

EUREKA!

Slot 0 opened
grub> set root=(crypto0)
grub> configfile (crypto0)/boot/grub/grub.cfg

Boots into Manjaro grub menu
Selected Manjaro
Boots right to Manjaro login screen; i.e. no further decryption keys required.

3 Likes
#51

Test D

grub> insmod cryptodisk
grub> insmod luks
grub> insmod gcry_rijndael
grub> insmod gcry_rijndael
grub> insmod gcry_sha256


grub> cryptomount hd0,gpt2

Timed the decryption this time; it took around 7 minutes.

grub> set root=(crypto0)
grub> probe -u (hd0,gpt2) --set=abc
error: unknown filesystem.

Hmm... So much for Test D

Some extra output I thought my be useful:

grub> echo $root
crypto0
grub> ls
(cypto0) (proc) (hd0) (hd0,gpt2) (hd0,gpt1) (hd1) (hd1,gpt4) (hd1,gpt3) (hd1,gpt2) (hd1,gpt1) (cd0) (cd0,msdos2)
grub> ls ($root)/
lost+found/ boot/ proc/ sys/ ... #you get the point, it lists the root of my Manjaro install
grub> echo $grub_platform #I was triggered by that (cd0,msdos2) output earlier
efi

My thinking, if I read some of the things in this post on LUKS encryption on Slackware.. perhaps some extra insmod somemod? Since my Manjaro install's grub menu doesn't have any problems, should we check which mods are available in there, compared to the one of the Live Boot USB?

I'll just wait for a reply for a bit before continuing.

2 Likes
#52

Okay! Thanks!
I think I've got why we failed test d.

grub> probe -u (hd0,gpt2) --set=abc
error: unknown filesystem.

It cannot get the uuid of sda2 but it can get the uuid of crypto0. That's the uuid of decrypted sda2.
Not sure if we can boot without uuid of sda2 but I'll try to see. busy at the moment. Will reply in 3~4 hours.

Any idea why decrypting sda2 took 'forever' the first time, very long the second time and 7 minutes the 3rd try? They are all at the same stage. I would think your 'usual' boot won't take a few seconds.
It is troubling, still and if you or anyone have an idea, however unproven, it will still be useful.

Cheers, catch up later.

#53

Ah, still busy.
But I think that's because your grub.cfg has uuid of sda2 and since it is decrypted to its decrypted uuid, and uncrypted sda2 uuid is not available.

I will look at it when back at desktop.
Meantime appreciate any input.

#54

Right, a quick summary.
o Test C which uses (configfile) the OS grub.cfg worked.
o Test D (direct boot) failed because it is unable to access sda2 uuid, though it is written by the OS into its grub.cfg and /etc/default/grub. Not being able to access it is also part of the decision to try to boot the OS without it. :smile:
o Test D can work (I think) if we are to manually input the uuid of the partition (sda2) which requires prior knowledge of that uuid and makes the method tedious and impractical.
o The long times to decrypt the partition is still troubling and unresolved.

ps: to clarify, the uuid of 'raw' sda2 is 'd4c943e7-6eb0-46ae-b85c-7327618dc754'
the uuid of the decrypted sda2 is 'bccdf1c7-d0f0-45ff-a0ab-df028a2c5d87'
they are different.

So now we make one attempt to see if we can boot up without the need of raw uuid of sda2.
Reminder that test c uses configfile which contains sda2 uuid and test d failed to access sda2 uuid and therefore failed to boot. So it is a modification of test d and won't rely on accessing sda2 uuid.

###############################################
TEST E

grub> insmod cryptodisk
grub> insmod luks
grub> insmod gcry_rijndael
grub> insmod gcry_rijndael
grub> insmod gcry_sha256


grub> cryptomount hd0,gpt2
grub> set root=(crypto0)
grub> probe -u $root --set=pqr

grub> linux   /boot/vmlinuz-4.19-x86_64 root=UUID=$pqr rw 
grub> initrd  /boot/initramfs-4.19-x86_64.img
grub> boot

Thanks for testing @infallible_haibt and pinging @AgentS

ps: no further tests are planned.
ps: @infallible_haibt, can you print out from your terminal (after booting up OS)

grub-editenv list

I am trying to look for any variable there that makes the decrypting faster.
maybe a "cryptodisk=y" or "cryptodisk=1" and without it in our commands make it slower.
Adding it (if it is there) will make it faster (I think).

1 Like
#55

I've been out of the air for a bit, sorry for that. Got distracted by day-to-day life. Let's get on with it though, I don't believe we were finished!

Test E

grub> insmod cryptodisk
grub> insmod luks
grub> insmod gcry_rijndael
grub> insmod gcry_rijndael
grub> insmod gcry_sha256

grub> cryptomount hd0,gpt2
Enter passphrase for hd0,gpt2 (d4c94...dc754):
Slot 0 opened # This finished while I started this reply, didn't look, maybe 2 min?
grub> set root=(crypto0)
grub> probe -u $root --set=pqr

grub> linux   /boot/vmlinuz-4.19-x86_64 root=UUID=$pqr rw 
grub> initrd  /boot/initramfs-4.19-x86_64.img
grub> boot

New screen:

:: Running early hook [udev]
Starting version 241.607-1-manjaro
:: running hook [udev]
:: Triggering uevents...
:: Loading keymap...done.
:: running hook [encrypt]
Waiting 10 seconds for device /dev/disk/by-uuid/bccdf1c7 ... # decrypted sda2 uuid
[   3.455089] sd 0:0:0:0: [sda] Asking for cache data failed
[   3.455089] sd 0:0:0:0: [sda] Assuming drive cache: write through
:: running hook [openswap]
mount: /openswap_keymount: no filesystem type specified.
Failed to open key file.
umount: can't unmount openswap_keymount: Invalid argument
:: running hook [resume]
ERROR: resume: no device specified for hibernation
Waiting 10 seconds for device /dev/disk/by-uuid/bccdf1c7... # decrypted sda2 uuid
:: mounting 'UUID=bccdf1c7...' on real root # decrypted sda2 uuid
mount: /new_root: can't find UUID=bccdf1c7... # decrypted sda2 uuid
You ar enow being dropped into an emergency shell.
sh: can't access tty; job control turned off
[rootfs ]# _

Close but no cigar?

#56

Noted you did not mention any long times to decrypt sda2. If it has improved, it might be time for a small celebration.

So it looks like Test E proved that it is still necessary to input the uuid of sda2.

So this line in Test D

grub> linux   /boot/vmlinuz-4.19-x86_64 root=UUID=$pqr rw cryptdevice=UUID=$abc:luks-$abc root=/dev/mapper/luks-$abc

is required. While $pqr can be determined, $abc needs to be manually input.
And the line to determine $abc must be removed.

grub> probe -u (hd0,gpt2) --set=abc

We can determine $abc with a cat command in grub after setting decrypted sda2 as root.

grub> cat ($root)/etc/crypttab

@infallible_haibt, while I do not have any encryption, you can check if the uuid of sda2 can be found in /etc/crypttab. You should have 2 entries, one for sda2 and the other for swap.

I had hoped that uuid for sda2 ($abc) can be 'mined' like we have done for decrypted sda2 ($pqr) but unfortunately, while 'cat', 'ls' is available in grub, 'grep', 'more' or 'less' is not available. Perhaps my 'regexp' (regular expression) foo is not sufficient.

A practical procedure to ensure we have a bootable encrypted system if the bootloader goes kaput is to therefore to have a backup grub rescue cd with the working grub.cfg burned into it.

Thanks @infallible_haibt for helping to test this out and everyone here for the contribution.

Cheers, take care.

1 Like
#57

It was a bit over a minute, which is slower than the 'regular' boot. But much improved over the earlier ~7 mins, somehow.

Yep, this works. As you said, there are two entries.

If I run Test D and swap out:

grub> probe -u (hd0,gpt2) --set=abc

for:

grub> set abc=(d4c94...) # The UUID from ($root)/etc/crypttab

It will continue into the same result as from Test E.

Meaning it doesn't actually boot properly.

Is there anything you still need me to test? I understand correctly that there is nothing to do to resolve my error when doing a 'regular boot' w/o live usb and without going through UEFI, falling back into grub rescue?

EDIT: Just checked the decryption-time from the uSD, it's about 15sec.

#58

Correct. And I understood it. What I was saying in my last post is that after removing the probe to mine $abc, we will need to manually input $abc (the full entry as in d4c943e7-6eb0-46ae-b85c-7327618dc754) which can be obtained from "cat ($root)/etc/crypttab" after $root is set to decrypted sda2.

I did not write out the complete Test (Test F?) because I personally thought it is not practical enough. I propose an alternative which is the link to make a bootable grub rescue disk.
I can write out the Test F if you want to test it out. But don't forget you will need to manually type in the full uuid of sda2 ("d4c943e7-6eb0-46ae-b85c-7327618dc754") 3 times at the linux line. . You don't have to for the uuid of decrypted sda2 ("bccdf1c7-d0f0-45ff-a0ab-df028a2c5d87") though.

There is really no need for us to continue, but if you want to test TEST F where you will need to manually input 3 times uuid of sda2 at the linux line, give a shout. :smile:. I suggest you try out the grub rescue cd and see if that boots. I think you may want to keep it as insurance. And if you change the kernel and haven't made a new rescue cd, you can manually (press 'e') just change the kernel and that should also boot. I can work with you if you have any questions on it. As said, I think that is more practical.

ps: I am still going through what 'regexp' (insmod regexp) can do if it can mine out sda2 uuid. i know about regexp in grub but so far all these while, I see no need to use it (probe -u is easier, for example). It was fun using it (detecting drives and partitions) but there are other practical and easier alternatives.

Cheers!

[EDIT] - 15 secs? That's good! I was wondering about the long decrypt times. Shouldn't happen unless we (meaning I) mess up some grub commands. Great to hear.

#59

I actually understood what test F should look like, so I already did it anyway. It was what I was reporting about. I don't quite understand why you say I should manually type it out three times, though. I just manually set abc=(UUID) instead of using the probe -u --set=abc-construct.
My point was that although it kind of starts to boot, it doesn't actually properly boot into Manjaro. Just like Test E.

Anyway, thanks for the help! Glad to have been of use testing.

And as for the decryption times, the 15 sec is fine for me. And that's even from a microSD card (so over USB). I imagine it should do even better if I were to install Manjaro on the NVMe SSD, which is obviously a faster drive.

#60

Hey.. just a typo ...maybe
No parenthesis. Just

set abc=d4c943e7-6eb0-46ae-b85c-7327618dc754

no apostrophes, and no parenthesis.
But you have to make sure you typed correctly.

#61

Oh...microSD card? There might be some problem using microSD card to boot. Some technical things about something. Some will work if inserted before starting with it inserted. Some have reported it won't either. But usb ports are fine. Some say using usb ports for microSD is still not fine. I don't know for sure now. Just a caution.

#62

Just tried this, no luck. Same error.

Hehe, thought this was already clear. I guess there's just too many uncertainties in my current setup. :slight_smile:

#63

Okay, noted. But I just used the same as in your original booting (I hope :rofl:) grub.cfg.
Note I cannot recheck (I don't have encryption) but if you find any error or typo later on, or a rechecking done but still an error, let us know.

You can recheck with, say...
grub> echo $abc
grub> echo $pqr
grub> echo $root #maybe $root was wrong? It should refer to 'crypto0' not (hd0,gpt2)

But good luck. You're literally on your own here. :smile:
I'm lucky I have @AgentS who checks many of the errors I make.

#64

On this one I cannot follow on. I am glad you can also troubleshoot things you don't really use.. :stuck_out_tongue_winking_eye:

My main question on this issue is "why don't you use provided tools (Calamares, M-A) and contribute to their bugs" ??

#65

The only thing I missed out from your grub.cfg linux line is resume=xxxx parameter.
Maybe that' is important in encrypted systems?
Normally for unencrypted systems, I do away with this resume line (for testing, just bothersome and lazy) {and intel-ucode} and that will work. But if you want to test that out, you will also need to decrypt it (swap) put in the correct uuid for it (similarly obtained from cat /etc/crypttab) and entered manually.

As said, I thought that shouldn't matter but maybe it needs it for encrypted systems. That's the only thing I left out. Thought I should leave out another variable that can cause havoc if done wrong.

#66

Learning from you, you have taught well. :rofl:

1 Like
#67

Another problem solved: I no longer boot into grub rescue.
Turns out I had to rearrange boot order where internal storage should be the first entry.
The boot order now looks like this:

BootCurrent: 0003
Timeout: 0 seconds
BootOrder: 0000,0005,0004,0001,0002
Boot0000* Internal Storage	FvVol(a881d567-6cb0-4eee-8435-2e72d33e45b5)/FvFile(50670071-478f-4be7-ad13-8754f379c62f)SDD.
Boot0001* USB Storage	FvVol(a881d567-6cb0-4eee-8435-2e72d33e45b5)/FvFile(50670071-478f-4be7-ad13-8754f379c62f)USB.
Boot0002* PXE Network	FvVol(a881d567-6cb0-4eee-8435-2e72d33e45b5)/FvFile(50670071-478f-4be7-ad13-8754f379c62f)PXE.
Boot0003* MsTemp	PciRoot(0x0)/Pci(0x1c,0x0)/Pci(0x0,0x0)/NVMe(0x1,00-25-38-41-5B-10-84-48)/HD(1,GPT,7626a74d-7e69-4adf-815b-b06a09f2d6a3,0x800,0x82000)
Boot0004* Manjaro	HD(1,GPT,7626a74d-7e69-4adf-815b-b06a09f2d6a3,0x800,0x82000)/File(\EFI\Manjaro\grubx64.efi)
Boot0005* Windows Boot Manager	HD(1,GPT,7626a74d-7e69-4adf-815b-b06a09f2d6a3,0x800,0x82000)/File(\EFI\manjaro\grubx64.efi)WINDOWS.........x...B.C.D.O.B.J.E.C.T.=.{.9.d.e.a.8.6.2.c.-.5.c.d.d.-.4.e.7.0.-.a.c.c.1.-.f.3.2.b.3.4.4.d.4.7.9.5.}....................

Seems like everything is working as it should. Just have a nasty-looking red bar on booting up:
Microsoft Surface w/o Safe boot enabled

#68

Actually… you just swapped 0005 and 0004, comparing with your previous report. 0000 is not booting anything (I suppose), so the next is the WinBoot, which is hijacked by Manjaro.

#69

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.

Forum kindly sponsored by