Update PathMaskChecker for Directory List #2505
Conversation
| .filter( Objects::nonNull ) | ||
| .anyMatch(groupIdPath -> path.startsWith( groupIdPath ) || groupIdPath.startsWith( path ) ); | ||
|
|
||
| if ( matches ) |
There was a problem hiding this comment.
does this mean it is faster to check the groupId against meta pattern before checking all patterns?
There was a problem hiding this comment.
Checking all patterns does not work for some cases, like the following patterns of the repo maven:remote:koji-com.sun.mail-all-1.6.1.redhat_1-1:
"path_mask_patterns": [ "com/sun/mail/javax.mail/maven-metadata.xml", "r|com\\/sun\\/mail\\/.+\\/1.6.1.redhat-1\\/.+|", "r|javax\\/mail\\/.+\\/1.6.1.redhat-1\\/.+|" ],
it does not work if the directory listing path is "/com/sun/mail/android". And the android-1.6.1.redhat-1.pom is also one of the artifacts produced by that brew build.
There was a problem hiding this comment.
Fixing this https://issues.redhat.com/browse/MMENG-4453 will resolve the above issue, but we need to make it compatible on the exiting remote koji repos.
There was a problem hiding this comment.
would this make a lot many repos return true? if group matches (/com/sun/mail), such repo will be listed even the artifact not match.
There was a problem hiding this comment.
I think this function is aiming to reduce the repo candidates for further query against cassandra or brew. When I debug this, it now returns all koji-* repos which is ~2500 on prod.
There was a problem hiding this comment.
As I understand this pr is to fix the file missing issue when listing, so you use the groupId to filter first. Am I right? If so, let's go with it.
There was a problem hiding this comment.
but why it now returns all koji-* repos? if the patterns do not match, it should filter most of them...
There was a problem hiding this comment.
is it because old line 76? if ( isRegexPattern( pattern ) ) --> this could return true when hit the regex pattern. maybe it always hit it and finally returns all koji repos.
There was a problem hiding this comment.
yeah, that's it line 76 and the reason in the comment line 78~80.
And this PR is still for the listing performance issue, https://issues.redhat.com/browse/MMENG-4447 .
There are attached logs I found during tests that would be helpful to know how it costs.
2 participants
Issue: https://issues.redhat.com/browse/MMENG-4447
The PR tries to extract the groupId path from the metadata file pattern, and using the groupId path to match with the specific path to filter the remote repos, especially the koji repos.
For those repos without the metadata file patterns, will still follow the previous process.