Skip to content

rpm: consult dnf database for repository information#869

Merged
github-actions[bot] merged 31 commits intoquay:mainfrom
hdonnay:hack/dnf
Jul 1, 2025
Merged

rpm: consult dnf database for repository information#869
github-actions[bot] merged 31 commits intoquay:mainfrom
hdonnay:hack/dnf

Conversation

@hdonnay
Copy link
Copy Markdown
Member

@hdonnay hdonnay commented Mar 13, 2023 •
edited
Loading

These changes make the rhel machinery consult both the rpm and dnf databases when possible.

See-also: #809

@hdonnay hdonnay force-pushed the hack/dnf branch 2 times, most recently from 5acce04 to 5a9d76e Compare April 13, 2023 15:03
@hdonnay hdonnay mentioned this pull request Jan 12, 2024
Merged
@codecov
Copy link
Copy Markdown

codecov bot commented Jan 12, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 29.52381% with 74 lines in your changes missing coverage. Please review.

Project coverage is 55.23%. Comparing base (c2fc225) to head (57bb054).

Files Patch % Lines
rpm/dnf.go 25.71% 52 Missing ⚠️
rpm/packagescanner.go 14.28% 18 Missing ⚠️
rhel/coalescer.go 57.14% 1 Missing and 2 partials ⚠️
rpm/native_db.go 85.71% 1 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #869      +/-   ##
==========================================
- Coverage   55.39%   55.23%   -0.17%     
==========================================
  Files         282      283       +1     
  Lines       17836    17923      +87     
==========================================
+ Hits         9881     9899      +18     
- Misses       6923     6990      +67     
- Partials     1032     1034       +2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@hdonnay hdonnay changed the title wip: rpm: consult dnf database for repository information rpm: consult dnf database for repository information Dec 19, 2024
hdonnay
hdonnay commented Dec 19, 2024
View reviewed changes
rpm/dnf.go Outdated
Comment on lines +49 to +52
if fi, err := fs.Stat(sys, `root/buildinfo/content_manifests`); errors.Is(err, nil) && fi.IsDir() {
// This is a RHEL layer, skip.
return nil, nil
}
Copy link
Copy Markdown
Member Author

@hdonnay hdonnay Dec 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Update this to check the files for a marker value and continue if found.

Should look for from_dnf_hint key in the yaml files.
See also: containerbuildsystem/atomic-reactor#2140

Copy link
Copy Markdown
Member Author

@hdonnay hdonnay Jan 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Punting on this unless we determine it's needed.

@cgwalters
Copy link
Copy Markdown

cgwalters commented Jan 6, 2025

I created rpm-software-management/rpm#3505 related to this

@hdonnay hdonnay force-pushed the hack/dnf branch 3 times, most recently from 10b07ff to e59f165 Compare January 13, 2025 15:57
@hdonnay hdonnay force-pushed the hack/dnf branch 3 times, most recently from 8c86d88 to 7a459ba Compare January 22, 2025 20:08
@hdonnay hdonnay marked this pull request as ready for review January 22, 2025 20:22
@hdonnay hdonnay requested a review from a team as a code owner January 22, 2025 20:22
@hdonnay hdonnay requested review from RTann and removed request for a team January 22, 2025 20:22
@RTann RTann requested a review from crozzy January 22, 2025 22:11
RTann
RTann reviewed Jan 23, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@RTann RTann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

made it through most of the files, but I'll need to come back to this another time. Would you mind making more commits and then squashing things at the end so it's easier to see the diffs after each round? Thanks

RTann
RTann reviewed Jan 24, 2025
View reviewed changes
crozzy
crozzy reviewed Feb 13, 2025
View reviewed changes
crozzy
crozzy reviewed Feb 14, 2025
View reviewed changes
hdonnay added 24 commits June 27, 2025 18:24
@hdonnay
rpm: reorganize Info, fs walking code
f4359c2 
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
@hdonnay
rpm: move database adapters to new path
7c1af10 
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
@hdonnay
internal/rpm: internal rpm handling package
b33be14 
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
@hdonnay
internal/dnf: add dnf-handling package
db469a8 
This adds two helpers: one to discover repoids, and one to annotate a
stream of Packages.

Signed-off-by: Hank Donnay <hdonnay@redhat.com>
@hdonnay
rhel: add package scanner
1e19b36 
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
@hdonnay
rhel: fix testcase names
0216629 
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
@hdonnay
rhel: have repository scanner consult dnf
bf9fb25 
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
@hdonnay
rhel: use repoids in coalescer
70c6574 
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
@hdonnay
rhel: rework tests in light of repoid information
91d5928 
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
@hdonnay
rpmtest: handle repoids in comparisons
3aa9071 
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
@hdonnay
rpmtest: add archive-based test helper
68203bd 
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
@hdonnay
rpm: delegate path checking to internal package
8bfef9f 
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
@hdonnay
rpm: move to internal packages
3190789 
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
@hdonnay
periodic: update to new interfaces
633d7c7 
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
@hdonnay
ndb: remove unused code
b959049 
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
@hdonnay
bdb: remove unused code
35bdb7c 
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
@hdonnay
sqlite: remove unused code
45175d9 
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
@hdonnay
rhel: use archive test helper
e77c9cd 
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
@hdonnay
rhel: test cleanups
a57f546 
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
@hdonnay
rhel: add coalescer test
26f693a 
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
@hdonnay
rhel: documentation pass
e9322d6 
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
@hdonnay
rhel: update bug report URL
689bd1b 
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
@hdonnay
bdb: start next iteration when continuing from error return
45d0dd1 
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
@hdonnay
bdb: explain short read beavior for "rope" type
7f697ad 
I saw this and had to re-check why the short read wasn't handled, so
this adds a note explaining it.

Signed-off-by: Hank Donnay <hdonnay@redhat.com>
@hdonnay
Copy link
Copy Markdown
Member Author

hdonnay commented Jul 1, 2025

LGTM. Last question would be: have you tested this out on any Konflux-built images to see if everything works as expected (and the dnf database even exists)?

Not specifically, no.

@hdonnay
Copy link
Copy Markdown
Member Author

hdonnay commented Jul 1, 2025

/fast-forward

@github-actions github-actions bot merged commit 7f697ad into quay:main Jul 1, 2025
6 checks passed
@hdonnay hdonnay deleted the hack/dnf branch July 1, 2025 21:07
@RTann RTann mentioned this pull request Sep 16, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@crozzy crozzy crozzy approved these changes

@RTann RTann RTann approved these changes

Assignees

No one assigned

Labels

None yet

Milestone

v1.5.40

Development

Successfully merging this pull request may close these issues.

4 participants

@hdonnay @cgwalters @crozzy @RTann