Bug 369077 - Issues with UEFI dual booting with other Ubuntu OS's
Summary: Issues with UEFI dual booting with other Ubuntu OS's
Status: CONFIRMED
Alias: None
Product: neon
Classification: KDE Neon
Component: Packages User Edition (show other bugs)
Version: unspecified
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: Neon Bugs
URL: https://forum.kde.org/viewtopic.php?f...
Keywords:
Depends on:
Blocks:
 
Reported: 2016-09-19 20:54 UTC by Morgan Cox
Modified: 2023-11-06 15:54 UTC (History)
12 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Morgan Cox 2016-09-19 20:54:53 UTC
When I install Neon, then install another Ubuntu based OS - for example Mint, The neon UEFI boot entry loads grub from the wrong partition (i.e the Mint one)..

i.e after installing the second os (mint), I have a 'neon' UEFI entry as well as an 'ubuntu' entry - the problem is now either the neon or the ubuntu uefi entry both boot from the mint partition, this means I can load neon from the mint grub, however this is not ideal as say I install a kernel update in neon, the mint grub entry wouldn't register the new kernel image in grub.

Also say I delete mint, this means I have an unbootable neon os....

I tried reinstalling and ensuring that I have a separate efi partition for mint, I even ensured whilst installing mint that I unselected the neon efi boot loader partition (i.e do not use this partition) and it still sticks the UEFI boot files in the neon EFI partition.

My question I guess is : Once I have installed another ubuntu based OS how to I have the neon UEFI entry the 'active' one again ?

If it helps here is the output of 'efibootmgr -v'

------------------------------------------
root@MS-7758:~# efibootmgr  -v

BootCurrent: 0001
Timeout: 1 seconds
BootOrder: 0006,0003,0002,0001,0000,000C,0013,0016,0019,0018,0005,0004
Boot0000* ubuntu        HD(1,GPT,e09282d1-0ceb-450f-9bbb-ce15704bf86f,0x800,0x5f000)/File(\EFI\ubuntu\shimx64.efi)
Boot0001* neon  HD(1,GPT,e09282d1-0ceb-450f-9bbb-ce15704bf86f,0x800,0x5f000)/File(\EFI\neon\shimx64.efi)                                                                                                                                     
Boot0002* archlinux     HD(7,GPT,b2931b61-aaf5-4f63-bf5d-0c27b32a5b84,0x4570f800,0x64000)/File(\EFI\archlinux\grubx64.efi)                                                                                                                   
Boot0003* opensuse      HD(9,GPT,62b2fb01-63da-4f1a-ba58-2f3893c679dd,0x6af73800,0x4f000)/File(\EFI\opensuse\grubx64.efi)                                                                                                                    
Boot0004  opensuse-secureboot   HD(9,GPT,62b2fb01-63da-4f1a-ba58-2f3893c679dd,0x6af73800,0x4f000)/File(\EFI\opensuse\shim.efi)                                                                                                               
Boot0005* UEFI: Built-in EFI Shell      VenMedia(5023b95c-db26-429b-a648-bd47664c8012)AMBO                                                                                                                                                   
Boot0006* Linux Boot Manager    HD(7,GPT,b2931b61-aaf5-4f63-bf5d-0c27b32a5b84,0x4570f800,0x64000)/File(\EFI\goofiboot\goofibootx64.efi)                                                                                                      
Boot000C* ubuntu        HD(1,GPT,e09282d1-0ceb-450f-9bbb-ce15704bf86f,0x800,0x5f000)/File(\EFI\Ubuntu\grubx64.efi)                                                                                                                           
Boot0013* UEFI OS       HD(9,GPT,62b2fb01-63da-4f1a-ba58-2f3893c679dd,0x6af73800,0x4f000)/File(\EFI\BOOT\BOOTX64.EFI)                                                                                                                        
Boot0016* UEFI OS       HD(7,GPT,b2931b61-aaf5-4f63-bf5d-0c27b32a5b84,0x4570f800,0x64000)/File(\EFI\BOOT\BOOTX64.EFI)                                                                                                                        
Boot0018* UEFI: KingstonDataTraveler 2.01.00    PciRoot(0x0)/Pci(0x1d,0x0)/USB(1,0)/USB(5,0)/HD(1,MBR,0x4294967286,0x400338,0x1240)AMBO                                                                                                      
Boot0019* UEFI: Samsung D3 Station 0200 PciRoot(0x0)/Pci(0x14,0x0)/USB(8,0)/HD(1,MBR,0x4294967171,0x40,0xe8e074c1)AMBO      
------------------------------------------


If I load mint (or ubuntu, etc) then neon and do a 


    sudo grub-install /dev/sda

Then mint (or whatever distro) takes priority again (as it should)

If you do it with neon, it doesn't - it creates a neon entry but that loads the mint grub - the other way round its fine.

i.e if I install neon then any other ubuntu distro I can no longer boot from the neon efi loader unless I reinstall neon. which means removal of the other os means non-bootable system, with mint at least I could go to a live cd, chroot and fix the bootloader, I cannot with neon.

Reproducible: Always

Steps to Reproduce:
1. install neon (uefi mode)
2. install another ubuntu in UEFI mode


Actual Results:  
I expected that 

after using 

sudo grub-install /dev/sda

In neon I expected it to be make neon's grub the default in the neon uefi entry

Expected Results:  
I expected that 

sudo grub-install /dev/sda would make neon's grub the default from the neon uefi entry
Comment 1 Roman Gilg 2016-10-07 22:41:53 UTC
I have the same problem on my laptop with a Neon User edition install and a Neon Dev unstable edition.

I assume the problem is the following:
Go as root into the uefi partition:
cd /boot/efi/EFI

There are two directories: neon and ubuntu and in both you find a config-file grub.cfg

On my system after executing grub-install from my firstly installed Neon User Edition it changes the denoted bootpartition only in the grub.cfg of the neon directory to its User Edition partition, but leaves the ubuntu/grub.cfg with the other value of the Dev Edition as it is. I assume the Uefi implementation of Neon then still favours this file instead of neon/grub.cfg
Comment 2 Jonathan Riddell 2016-10-19 11:36:41 UTC
Yes unfortunately we are using the ubuntu EFI builds and they have some hardcoded paths in them giving some limitations.  Editing this needs certification by Microsoft which is future work.
Comment 3 Morgan Cox 2016-10-24 15:31:47 UTC
Just to confirm the same happens with neon + ubuntu 16.10.

My only way of getting the 'neon' UEFI entry back to neon's grub is to re-install...

Is there another way ?

i.e what occurs during install (so I can try to recreate it).

At present I installed neon, then ubuntu 16.10 (to test it out) - I want to remove ubuntu 16.10 but if I do I've lost access to neon (until I reinstall it)
Comment 4 Roman Gilg 2016-11-15 01:14:53 UTC
Some more info: After today's update it changed my GRUB menu from one Neon install to the other. This wasn't possible before with by 'sudo grub-install', i.e. it didn't successfully install GRUB from the other install (didn't change the UEFI partition to point to the other install).

That it now changed after the update, means that there is some way apparently to change the UEFI partition.
Comment 5 Max McKinney 2018-06-14 07:17:17 UTC
I just reproduced this issue while helping someone troubleshoot, and I can confirm that it still exists in the latest Neon User Edition (5.12.5). In a nutshell, there is some overlap between Neon and Ubuntu as far as the files they use on the ESP partition.

When I first installed Neon the EFI/ESP partition looked like this

root@neon-efi:/boot/efi/EFI# ls -lR
.:
total 12
drwx------ 2 root root 4096 Jun 13 00:01 BOOT
drwx------ 2 root root 4096 Jun 12 23:16 neon
drwx------ 3 root root 4096 Jun 13 02:07 ubuntu

./BOOT:
total 1252
-rwx------ 1 root root 1196736 Jun 13 00:01 BOOTX64.EFI
-rwx------ 1 root root   79856 Jun 13 00:01 fbx64.efi

./neon:
total 3412
-rwx------ 1 root root     126 Jun 13 00:58 grub.cfg
-rwx------ 1 root root 1133944 Jun 13 00:58 grubx64.efi
-rwx------ 1 root root 1153336 Jun 13 00:58 mmx64.efi
-rwx------ 1 root root 1196736 Jun 13 00:58 shimx64.efi

./ubuntu:
total 76
drwx------ 2 root root  4096 Jun 12 23:53 fw
-rwx------ 1 root root 67536 Jun 12 23:53 fwupx64.efi
-rwx------ 1 root root   126 Jun 12 23:53 grub.cfg

./ubuntu/fw:
total 0

At this point Ubuntu wasn't installed yet, so I found it a bit odd that there was already an ubuntu directory with a grub.cfg but no EFI binaries. The grub.cfg in the ubuntu directory was exactly the same as the one in the neon directory. Both looked like this:

search.fs_uuid 29462c84-402a-4111-8f66-bf30fc0caca5 root hd0,gpt2 
set prefix=($root)'/boot/grub'
configfile $prefix/grub.cfg

Basically just a pointer to my grub.cfg in the Neon root file system at /boot/grub/grub.cfg.

efibootmgr looked like this

root@neon-efi:/boot/efi/EFI# efibootmgr -v
BootCurrent: 0003
BootOrder: 0003,0000,0001,0002
Boot0000* EFI DVD/CDROM PciRoot(0x0)/Pci(0x1,0x1)/Ata(1,0,0)
Boot0001* EFI Hard Drive    PciRoot(0x0)/Pci(0xd,0x0)/Sata(0,0,0)
Boot0002* EFI Internal Shell    MemoryMapped(11,0x2100000,0x28fffff)/FvFile(7c04a583-9e3e-4f1c-ad65-e05268d0b4d1)
Boot0003* neon  HD(1,GPT,ca5c06d9-47c0-4d84-a568-a61cbfcc6d5d,0x800,0x100000)/File(\EFI\neon\shimx64.efi)

From this you can see that Neon is using the /boot/efi/EFI/neon directory, so it would be logical to assume that it's also using the grub.cfg in that same directory. This is NOT actually the case though. I took a look at /boot/efi/EFI/neon/grubx64.efi with hexdump and noticed that it uses the Ubuntu directory on the ESP partition:

root@neon-efi:/boot/efi/EFI/neon# hexdump -C ./grubx64.efi
...
001137d0  03 00 00 00 18 00 00 00  2f 45 46 49 2f 75 62 75  |......../EFI/ubu|
001137e0  6e 74 75 00 00 00 00 00  00 00 00 00 00 00 00 00  |ntu.............|
...

The string "/EFI/neon" doesn't appear anywhere in the file. Jonathan mentioned above that this is because they are using Ubuntu's grub build which contains hard coded paths and is digitally signed. Unfortunately this causes some serious problems if another Ubuntu based distribution is installed.

I installed Kubuntu and the Kubuntu grub menu took over and wouldn't go away (same problem other have reported above). No combination of grub-install, grub-update, or reinstalling grub packages would bring back the Neon Grub menu. I could see that Ubuntu had replaced the grub.cfg file in /boot/efi/EFI/ubuntu/ with its own version that looked like this.

search.fs_uuid 26b73c4e-f909-4279-b5c2-d5f4c6697f60 root hd0,gpt4 
set prefix=($root)'/boot/grub'
configfile $prefix/grub.cfg

Now it contains the partition number and file system UUID of my Kubuntu root partition, which is why I always get the Kubuntu grub menu even when Neon's grub binary is selected. The grub.cfg in the /boot/efi/EFI/neon directory was still correct, but it doesn't appear to even be used by Neon's grubx64.efi binary.

I went ahead and overwrote the grub.cfg in the ubuntu directory with the one from the neon directory, and that solved the problem. The normal Neon grub menu is back.

Unfortunately copying that config back into place isn't a perfect solution since Kubuntu is likely to overwrite it again at some point. Also, if I want to get rid of Kubuntu and go back to just Neon I'd need to be aware that the Ubuntu grub.cfg file needs to be manually replaced with the one from the Neon directory.

There is no way to "fix" Neon's Grub menu that I could find short of manually changing EFI/ubuntu/grub.cfg to point to the Neon partition/fs. Most people would expect grub-install to be able to handle this, but it since it doesn't modify the grub.cfg in EFI/ubuntu it can't fix this issue on its own.

I understand that getting new signed Grub binaries would be a pain, but the current situation basically breaks grub-install and could easily lead to an unbootable system if someone were to delete their secondary Ubuntu based OS.
Comment 6 David Izquierdo 2018-06-24 11:16:50 UTC
I just stumbled with this issue after clearing an ESP because of a windows reinstall. Since there was no /EFI/ubuntu directory, grub only booted into its shell. Running `set` in the GRUB shell showed that $prefix (the variable from which grub knows where to load its actual config file from) is indeed hardcoded to /EFI/ubuntu, but EFI/ubuntu is never created once it goes away. 

My suggestion to work-around this issue is to either override the grub2-common package with a wrapper for grub-install that will `cp -rf /boot/efi/EFI/{neon,ubuntu}`, or to use /boot/efi/EFI/ubuntu in the first place. Otherwise, the expectation that grub-install fixes the bootloader is broken.

Unless we can actually get secure-boot working for our grub, that is.
Comment 7 Rik Shaw 2018-09-27 11:58:58 UTC
I work with another Ubuntu-based respin (Wasta-Linux: www.wastalinux.org) and was also experiencing this problem *only with Ubuntu 18.04* based respins.  In the past under Ubuntu 16.04(-) when installing in EFI mode the grub.cfg was placed in /boot/efi/EFI/ubuntu/grub.cfg

But, starting with 18.04, it was getting installed to /boot/efi/EFI/wasta-linux/grub.cfg.  However, when booting the system, it would drop to a "grub>" prompt because it was looking for /boot/efi/EFI/ubuntu/grub.cfg.  So maybe Ubuntu updated part that was hard-coded to go to /boot/efi/EFI/ubuntu but not all parts??

Through manual testing, I have confirmed that the place the EFI/<folder name> comes from is the FIRST word in the GRUB_DISTRIBUTOR variable from /etc/default/grub.  So in summary, if this is set to GRUB_DISTRIBUTOR="Ubuntu Wasta-Linux 18.04" everything will work correctly when installing in EFI mode (it will install grub to /boot/efi/EFI/ubuntu/grub.cfg).  So I would guess that for a quick hack Neon could set GRUB_DISTRIBUTOR="Ubuntu KDE Neon 18.04" and everything would work.

Sorry I haven't been smart enough to figure out which grub customization needs corrected in the Ubuntu grub packages or else I would attempt a proper patch.

By the way for others that want to re-spin: we maintain an updated Remastersys from old and now call it wasta-remastersys: https://github.com/wasta-linux/wasta-remastersys  We know others like Pinguy have made some changes but we have done some cleanup and refactoring to perform better and be compatible with newer Ubuntu versions.
Comment 8 zinczorphin 2023-11-06 15:54:13 UTC
If any of you is still facing this issue, there's a way you could fix. Since the ubuntu string has been hardcoded. The other efi entries' prefix would be ubuntu regardless. You could grub-install to a folder named "ubuntu" to actually boot up ( grub entries wont be affected if you're wondering ) since it's where it searches.

What I did:
sudo grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=ubuntu