Raspberry Pi OS • Re: Bookworm LVM rootfs

With Pimoroni advertising an upcoming dual nvme pcie hat, I may have to reconsider mdadm being pointless.

Slightly off topic but...

I can see that particular product doing more harm than good. Some folks don't understand the impact of having only a single PCIe lane on NVMe performance, how are they going to cope with splitting that between two drives? And how wil they cope with not being able to boot from it?

I wouldn't mind but the observable trend is that rather than ask the seller/manufacturer, folks will ask on here.

Not offtopic (imo)

Imagine, for a moment, a sane implementation of mdadm....where /dev/md0 is /boot/ (aka /boot/firmware/) and /dev/md1 is the OS and both are raid1. We'll ignore the fact it is likely best to have (at least) a /dev/md2 also (for data) as it doesn't factor into the boot problem.

I'd be amazed if rpi firmware supported booting off an mdadm device even in the distant future so we can't use /dev/md0 for /boot/ which then means some rsync solution where (say) a systemd service clones /dev/sda1 <-> /dev/sdb1. The key issue will be how the firmware handles the PCI NVMe devices: will it be consistent or will there be a race as to which NVMe device appears first? Will the firmware give up immediately or iterate all NVMe devices?

I've only ever used mdadm on an rpi once, as an afternoon experiment. I can't even remember what I did. Was it a pair of thumb drives? Was a single thumb drive with multiple partitions on it acting as a /dev/mdX of did I do similar with some spare space on the end of an ssd? No idea. I certainly never simulated a device not working (flakey device would be the real test). I likely forced one offline then had it rebuild: that would have been the sum total of the test.

I imagine a scenario with a failing NVMe where the system is booting but using part of one disk and part of the other. The combinations are manyfold and would require the user to notice. My PC has a pair of NVMe drives (along with SSD's). I don't use mdadm on it any more because I'd have to take the thing to bits if one those NVMe's failed. Rather defeats the point of raid. Also, like you say, the main purpose of NVMe is speed so why slow it down by using raid? My hope is smartd will give me some warning so I can start using the other NVMe while I order a pair of replacements, in the hope I only have to dismantle the PC one time.

I reckon I will get one of those pimoroni hats whenever I get a second rpi5 and perhaps I'll experiment with mdadm before I put it to use although in my use case I can only see use for raid0 (one big fast disk) and it would have to be significant in terms of speed wrt carving it up with LVM - where I'd likely just use one NVMe and the other for LVM snapshots or rsync target.

I suspect most users will follow a raid1 google tutorial and we'll hear nothing until it falls in a heap!

Statistics: Posted by swampdog — Sat Mar 23, 2024 8:37 pm

---