-
Notifications
You must be signed in to change notification settings - Fork 111
Composite aggregates #3266
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Composite aggregates #3266
Conversation
a25ae95
to
4c9ec49
Compare
72f7a46
to
2f5e9b8
Compare
8f97639
to
55cd005
Compare
adec3d2
to
91ed1cf
Compare
6fa2c04
to
0aeab9c
Compare
...ore/src/main/java/com/apple/foundationdb/record/query/combinatorics/EnumeratingIterable.java
Show resolved
Hide resolved
...java/com/apple/foundationdb/record/query/plan/cascades/expressions/RelationalExpression.java
Outdated
Show resolved
Hide resolved
...a/com/apple/foundationdb/record/query/plan/cascades/expressions/MatchableSortExpression.java
Outdated
Show resolved
Hide resolved
yaml-tests/src/main/java/com/apple/foundationdb/relational/yamltests/YamlTestExtension.java
Outdated
Show resolved
Hide resolved
return IntersectionResult.noViableIntersection(); | ||
} | ||
|
||
final var compensation = |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe we can immediately return IntersectionResult.noViableIntersection()
if the resulting intersection compensation
is impossible, correct?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No. It may be that we are currently impossible but that there is a common ordering (in fact there must be one for the follow up to work). If there is no common ordering the result will be discarded. If there is a common ordering but it is currently impossible to (think 2 out of 3 aggregate functions are satisfied), we want that intersection access to survive because otherwise we would never get to the intersection that creates the one where we get 3 out of 3.
...in/java/com/apple/foundationdb/record/query/plan/cascades/rules/AggregateDataAccessRule.java
Show resolved
Hide resolved
} | ||
isCompensationImpossible |= resultCompensationFunction.isImpossible(); | ||
|
||
groupByMappings = |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be nice to add some comments explaining the handling of matched group by aggregations and groups here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I extracted the common code between SelectExpression
, LogicalTypeFileExpression
, and GroupByExpression
. The new code also has some documentation.
...com/apple/foundationdb/record/query/plan/plans/RecordQueryMultiIntersectionOnValuesPlan.java
Outdated
Show resolved
Hide resolved
} | ||
} | ||
|
||
return IntersectionResult.of(hasCommonOrdering ? intersectionOrdering : null, compensation, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think it is possible to retrieve an IntersectionResult
which non-empty list of RelationalExpression
s and null
-common ordering, if we examine the code flow above.
Having said that, we could probably cleanup the code, such that we can remove the hasCommonFlag
maybe?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, You can have a common ordering but no expressions due to the compensation being impossible. You still want to signal the caller to keep this intersection (even though you didn't produce a single expression) in the sieve so you can potentially find a bigger intersection that is not impossible.
6e1ca10
to
5032dfb
Compare
ffb8384
to
fe4895f
Compare
f8e3a47
to
ad4b04f
Compare
7c91798
to
ae571c6
Compare
ae571c6
to
4b2c8c1
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great work, let's bring it in.
This PR adds the ability to plan arbitrary aggregation queries using aggregation indexes as well as intersections of aggregation indexes. It also corrects/adds the infrastructure to deal with complicated order-by requirements.