Extend `generate-update-metadata()` to read from `/usr` #938

HuijingHei · 2025-05-29T05:53:46Z

In generate-update-metadata(), checks
usr/lib/efi/(shim|grub)/%{version}-%{release}/EFI/ first for
EFI path, for example: usr/lib/efi/shim/1.2-3/EFI/BOOT/.
If the path usr/lib/efi doesn’t exist, falls back to the legacy
OSTree location usr/lib/ostree-boot/efi/EFI/.

Then move existing EFI path to the usr/lib/bootupd/updates/.

See #926

ci: add testing generate-update-metadata in bootc container
Thanks Colin's PR #936

In `generate-update-metadata()`, checks `/usr/lib/efi/(shim|grub) /%{version}-%{release}/EFI/` first for EFI path, for example: `usr/lib/efi/shim/1.2-3/EFI/`. If the path `/usr/lib/efi` doesn’t exist, falls back to the legacy OSTree location `/usr/lib/ostree-boot/efi/EFI/`. Then move existing EFI path to the `/usr/lib/bootupd/updates/`. See coreos#926

HuijingHei · 2025-06-05T07:03:56Z

Ready for reviewing now.

I did testing under fedora-bootc container, and upgrade grub2-efi-x64 shim-x64 to new, remove the old /usr/lib/bootupd/updates/* and run generate-update-metadata to create metadata.

Build new version according to #926 (comment):
grub2:
https://koji.fedoraproject.org/koji/taskinfo?taskID=133336624
shim:
https://koji.fedoraproject.org/koji/taskinfo?taskID=133341841

Download and prepare the repo

# cat myrepo.repo
[myrepo]
name=My Custom Repository
baseurl=http://localhost:8080/bootloader/
enabled=1
gpgcheck=0

Build patch

# make && make install-all DESTDIR=/srv/bootupd-test/out

Run testing under fedora-bootc container

podman run --privileged -it -v ./:/opt -w /opt -v ./tmp/myrepo.repo:/etc/yum.repos.d/myrepo.repo --net host quay.io/fedora/fedora-bootc:42 bash


bash-5.2# dnf upgrade -y grub2-efi-x64 shim-x64 --repo myrepo

bash-5.2# rpm -ql grub2-efi-x64 shim-x64 | grep usr/lib/efi
/usr/lib/efi/grub2/2.12-34.fc43/EFI/fedora/grubx64.efi
/usr/lib/efi/shim/15.8-4/EFI/BOOT/BOOTX64.EFI
/usr/lib/efi/shim/15.8-4/EFI/BOOT/fbx64.efi
/usr/lib/efi/shim/15.8-4/EFI/fedora/BOOTX64.CSV
/usr/lib/efi/shim/15.8-4/EFI/fedora/mmx64.efi
/usr/lib/efi/shim/15.8-4/EFI/fedora/shim.efi
/usr/lib/efi/shim/15.8-4/EFI/fedora/shimx64.efi

(dnf reinstall -y grub2-efi-x64 shim-x64 --repo myrepo)
bash-5.2# cp -rav out/usr /
bash-5.2# rm -rf /usr/lib/bootupd/updates/*

bash-5.2# /usr/bin/bootupctl backend generate-update-metadata / -vvv
[TRACE bootupd] executing cli
[DEBUG bootupd::ostreeutil] Using dbpath "--dbpath=/usr/lib/sysimage/rpm"
Generated update layout for BIOS: grub2-tools-1:2.12-34.fc43.x86_64
[DEBUG bootupd::ostreeutil] Using dbpath "--dbpath=/usr/lib/sysimage/rpm"
Generated update layout for EFI: grub2-efi-x64-1:2.12-34.fc43.x86_64,shim-x64-15.8-4.x86_64
[DEBUG bootupd::efi] Unmounting

### check the result:
bash-5.2# cat /usr/lib/bootupd/updates/BIOS.json 
{"timestamp":"2025-05-30T03:39:14Z","version":"grub2-tools-1:2.12-34.fc43.x86_64"} 
bash-5.2# cat /usr/lib/bootupd/updates/EFI.json 
{"timestamp":"2025-05-30T06:29:19Z","version":"grub2-efi-x64-1:2.12-34.fc43.x86_64,shim-x64-15.8-4.x86_64"}

Thanks Colin's PR coreos#936

cgwalters

Overall looks sane to me, a variety of comments. I don't have anything blocking.

I think we should be testing this scenario in CI, I doubt we are? I am not up to date on things, does it require us pulling in a copr?

cgwalters · 2025-06-09T17:45:08Z

src/efi.rs

        let dest_efidir = component_updatedir(sysroot_path, self);

-        if ostreebootdir.exists() {
+        // New EFI dir /usr/lib/efi


Do you know if this this specific to the new Fedora grub/shim packages? I suspect it is...we may want to somehow make this a build time or even runtime conditional so in theory it's more pluggable/controllable by others.

I'd at least factor it out into a const somewhere that explains where it came from.

Hmm actually, this topic also strongly relates to #766 right?

Thank you so much for the reviewing!

Do you know if this this specific to the new Fedora grub/shim packages? I suspect it is...we may want to somehow make this a build time or even runtime conditional so in theory it's more pluggable/controllable by others.

Yes.

I'd at least factor it out into a const somewhere that explains where it came from.

SGTM.

Hmm actually, this topic also strongly relates to #766 right?

Actually this is related to issue #926 (comment), but we can extend it support #766 too.

cgwalters · 2025-06-09T17:51:53Z

src/efi.rs

+                    Ok(acc)
+                });
+            packagesystem::query_files(sysroot_path, all_files?.into_iter())?
+        } else if ostreebootdir.exists() {


Would it be cleaner perhaps if we checked first for ostreebootdir, and migrated it to usr/lib/efi if that doesn't exist? It should be an error if both exist.

Then we get closer to thinking of the ostreebootdir one as legacy.

cgwalters · 2025-06-09T17:53:00Z

src/efi.rs

+                        })
+                        .collect::<Vec<String>>();
+
+                    Command::new("mv")


I think we have an opportunity here to make usr/lib/efi the standard path actually if it exists, maybe we could make things as simple as the equivalent of ln -sr /usr/lib/efi /usr/lib/bootupd/updates/EFI?

EDIT: Ah I see it's not that simple based on find_all_efi_dirs.

Hmmm...but actually I like the idea of that layout, what if we tried to adopt that as the standard and migrate our current EFI layout to it? (It'd break updates for older bootupd though...without having dual layouts for a while, ug)

cgwalters · 2025-06-09T17:57:10Z

src/efi.rs

@@ -615,6 +656,29 @@ fn find_file_recursive<P: AsRef<Path>>(dir: P, target_file: &str) -> Result<Vec<
    Ok(result)
 }

+// Find EFI dirs under usr/lib/efi
+// for exmaple: usr/lib/efi/shim/15.8-4/EFI, usr/lib/efi/grub2/2.12-34.fc42/EFI
+fn find_all_efi_dirs(sysroot_lib: &Path) -> Result<Vec<PathBuf>> {


BTW looking at this layout, I think we can now avoid invoking rpm to query the file information for these which would be a huge side benefit.

So again I think this relates to #766 in that perhaps we make this layout our new "API" for adding content in the ESP?

Actually thinking about things here...you know, it probably wouldn't be terribly hard to change what rpm-ostree does to automatically do this instead (via an opt-in). That'd require some coordination but the powerful benefit is we'd effectively automatically "backport" support for /usr/lib/efi even for older OSes which seems like it'd help us a lot here.

HuijingHei requested review from travier and cgwalters May 29, 2025 06:23

HuijingHei force-pushed the generate-metadata-from-usr branch 2 times, most recently from 01b09e0 to be19115 Compare May 29, 2025 07:42

HuijingHei added the do-not-merge/work-in-progress label May 29, 2025

HuijingHei force-pushed the generate-metadata-from-usr branch 5 times, most recently from 54d905f to fa2e65c Compare June 2, 2025 11:57

openshift-merge-robot added the needs-rebase label Jun 2, 2025

HuijingHei force-pushed the generate-metadata-from-usr branch from fa2e65c to 2a8961f Compare June 2, 2025 12:19

openshift-merge-robot removed the needs-rebase label Jun 2, 2025

HuijingHei force-pushed the generate-metadata-from-usr branch 2 times, most recently from d31c745 to ae28fbc Compare June 5, 2025 01:44

HuijingHei force-pushed the generate-metadata-from-usr branch from ae28fbc to efa3b42 Compare June 5, 2025 06:44

HuijingHei removed the do-not-merge/work-in-progress label Jun 5, 2025

ci: add testing generate-update-metadata in bootc container

4cac136

Thanks Colin's PR coreos#936

cgwalters approved these changes Jun 9, 2025

View reviewed changes

cgwalters mentioned this pull request Jun 9, 2025

support dynamically extending/adding update payloads #766

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Extend `generate-update-metadata()` to read from `/usr` #938

Extend `generate-update-metadata()` to read from `/usr` #938

Uh oh!

HuijingHei commented May 29, 2025 •

edited

Loading

Uh oh!

HuijingHei commented Jun 5, 2025 •

edited

Loading

Uh oh!

cgwalters left a comment

Uh oh!

cgwalters Jun 9, 2025

Uh oh!

HuijingHei Jun 10, 2025

Uh oh!

cgwalters Jun 9, 2025

Uh oh!

cgwalters Jun 9, 2025

Uh oh!

cgwalters Jun 9, 2025

Uh oh!

Uh oh!

Extend generate-update-metadata() to read from /usr #938

Are you sure you want to change the base?

Extend generate-update-metadata() to read from /usr #938

Uh oh!

Conversation

HuijingHei commented May 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HuijingHei commented Jun 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cgwalters left a comment

Choose a reason for hiding this comment

Uh oh!

cgwalters Jun 9, 2025

Choose a reason for hiding this comment

Uh oh!

HuijingHei Jun 10, 2025

Choose a reason for hiding this comment

Uh oh!

cgwalters Jun 9, 2025

Choose a reason for hiding this comment

Uh oh!

cgwalters Jun 9, 2025

Choose a reason for hiding this comment

Uh oh!

cgwalters Jun 9, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Extend `generate-update-metadata()` to read from `/usr` #938

Extend `generate-update-metadata()` to read from `/usr` #938

HuijingHei commented May 29, 2025 •

edited

Loading

HuijingHei commented Jun 5, 2025 •

edited

Loading