Add Root IO vulnerability data provider#963
Conversation
| } | ||
| } | ||
| ], | ||
| "database_specific": {"source": "Root"} |
There was a problem hiding this comment.
This should also assert something like:
if "database_specific" not in vuln_entry:
vuln_entry["database_specific"] = {}
if "anchore" not in vuln_entry["database_specific"]:
vuln_entry["database_specific"]["anchore"] = {}
vuln_entry["database_specific"]["anchore"]["record_type"] = "advisory"has been done. Otherwise, grype-db doesn't know to emit unaffectedPackageHandles and this data just makes affected package handles and the NAKS don't do anything.
chait-slim
force-pushed
the
feat/rootio-provider-2
branch
from
09e61c6 to
85f1e1d
Compare
| # Fix date patching is optional and requires authentication | ||
|
|
||
| # Fetch and process each OSV record | ||
| for osv_id in osv_ids: |
There was a problem hiding this comment.
Please make this concurrent in some way. Right now this provider does ~9K sequential, blocking http gets, which makes it very slow for a relatively small amount of data. Many of the other providers have some concurrent.futures.ThreadPoolExecutor use and a config that controls the concurrency (and sets a default higher than 1). Please imitate that pattern here.
There was a problem hiding this comment.
It's probably fine to enter a concurrent section that pulls down all the osv docs and then process them sequentially, which is probably easier than trying to get the entire record normalized and processed concurrently.
| - name: github | ||
| use_cache: true | ||
| images: | ||
| - cr.root.io/cassandra@sha256:b3cc918a6a364af0a6b0a45becef0d0979db7e604751fad627ec2a94945b4e03 |
There was a problem hiding this comment.
I think you changed this image to be on a different repo?
chait-slim
force-pushed
the
feat/rootio-provider-2
branch
from
c9e7a9e to
d92900e
Compare
|
Hi @chait-slim, I was just running this locally to check on it, and the quality gate currently fails like this: I think these are the missing labels that I was asking for at anchore/vulnerability-match-labels#167 (comment) - if you agree that these are things that rootio patched, they should be labeled as false positives in that PR. I'm also sort of surprised to see differences in these packages, which don't look like RootIO packages to me. Am I missing something? |
|
Hi @chait-slim I just tried to do an end to end test but I think maybe something changed about the upstream data? Is |
|
@willmurphyscode we had a tiny hiccup in the api. Its fixed now. Sorry about that |
|
@willmurphyscode did you check the fixed api? |
chait-slim
force-pushed
the
feat/rootio-provider-2
branch
3 times, most recently
from
2de13be to
66b3fbd
Compare
|
@willmurphyscode Following our talk today:
|
chait-slim
force-pushed
the
feat/rootio-provider-2
branch
2 times, most recently
from
bb6e667 to
a2605f7
Compare
| id_objects = response.json() | ||
|
|
||
| # Extract and validate ID strings from each object | ||
| all_ids = [obj["id"] for obj in id_objects] |
There was a problem hiding this comment.
In local testing, I had to add .strip() here because some ids in all.json have a space on the end. Can you do that here (and maybe fix all.json as well)?
This adds a new provider for Root IO vulnerability data, which provides
security information for Root IO patched packages across multiple
ecosystems (Debian, Ubuntu, Alpine, NPM, PyPI).
Implementation details:
- Fetches OSV 1.6.1 format data from Root IO API (api.root.io/external/osv)
- Implements NAK pattern: rootio- prefixed packages only match Root IO vulnerabilities
- Supports ecosystem-specific version suffixes (.root.io.N for Debian/Ubuntu,
-root.io.N for NPM, +root.io.N for PyPI)
- Provider class in src/vunnel/providers/rootio/__init__.py
- OSV record parser in src/vunnel/providers/rootio/parser.py
- Registered in src/vunnel/providers/__init__.py and src/vunnel/cli/config.py
Signed-off-by: Chai Tadmor <chai.tadmor@root.io>
….py _normalize() 2. Added comprehensive tests to verify the metadata is set correctly 3. Updated all 5 snapshot fixtures with the new metadata Signed-off-by: Chai Tadmor <chai.tadmor@root.io>
Signed-off-by: Chai Tadmor <chai.tadmor@root.io>
Swap cassandra for rootpublic/ubuntu:22.04 which demonstrates Root IO correctly suppressing Ubuntu won't-fix FPs via unaffectedPackageHandles. Update expected_namespaces and additional_providers accordingly. Signed-off-by: Chai Tadmor <chai.tadmor@root.io>
Signed-off-by: Chai Tadmor <chai.tadmor@root.io>
- Remove relaxed quality gate thresholds; use defaults
- Point grype/grype-db at feat/rootio-support and feat/rootio-feed
branches pending merge of those PRs
Signed-off-by: Chai Tadmor <chai.tadmor@root.io>
chait-slim
force-pushed
the
feat/rootio-provider-2
branch
from
a2605f7 to
81fdf36
Compare
Signed-off-by: Chai Tadmor <chai.tadmor@root.io>
| # - ALWAYS leave the "import-db" annotation as-is | ||
| # - this version should ALWAYS match that of the other "grype" tool below | ||
| version: main+import-db=build/vulnerability.db | ||
| version: github.com/chait-slim/grype@feat/rootio-support+import-db=build/vulnerability.db |
There was a problem hiding this comment.
I was trying to suggest that this be changed for local testing, not necessarily that the changes needed to be pushed. For example when you run the quality gate test you can do file:///path/to/grype+... here and on line 39 below, and to a local checkout of grype-db that has a go work use . ../grype in the grype db version.
Running the tests like this doesn't include database build time changes (because grype-db@main doesn't have your change from the grype PR yet), so when you run the quality gate with the config file as is, you only see differences in canonical's upstream data between when the reference db was published and when the cache was built, and don't see the effect of an actually changed rootio matching. (We're working to improve this part; I know it's confusing.)
Fork branch references were for local testing only and should not be committed. Signed-off-by: Chai Tadmor <chai.tadmor@root.io>
Rebuild e2e-rootio-alpine-test on Alpine 3.18 (fixes missing rootio APK
packages) and extend it with @rootio/semver from the rootio NPM registry,
providing end-to-end NAK suppression coverage for JavaScript packages.
- Fix Dockerfile.rootio-test: use Alpine 3.18 (rootio-openssh/openssl only
exist in 3.18, not 3.19); fix npm scoped-registry auth via .npmrc
- Add @rootio/semver@5.7.1-root.io.1 installation (fixes CVE-2022-25883)
- Update config.yaml image digest to sha256:b823...
- Bump vulnerability-match-labels submodule (new npm labels)
Signed-off-by: Chai Tadmor <chai.tadmor@root.io>
2 participants
This adds a new provider for Root IO vulnerability data, which provides
security information for Root IO patched packages across multiple
ecosystems (Debian, Ubuntu, Alpine, NPM, PyPI).
Implementation details: