CSHARP-3222: Add LINQ support for median and percentile accumulators/window functions #1743

adelinowona · 2025-07-29T11:03:56Z

This PR introduces the capability to calculate the median and percentile of numeric values in the MongoDB aggregation pipeline for $group and $setWindowFields stages.

…ow expressions

rstam

Initial quick review. Will review more thoroughly after next commit.

rstam · 2025-07-29T17:30:08Z

...tionExpressionTranslators/MethodTranslators/MedianMethodToAggregationExpressionTranslator.cs

+            var method = expression.Method;
+            var arguments = expression.Arguments;
+
+            if (IsMedianMethod(method))


Normally we do if (method.IsOneOf(__medianMethods)).

Do we want to do it differently here?

I noticed that pattern but I just felt that would be too much boilerplate kind of code and there's an easier way to do it. Plus I noticed the StandardDeviationMethodsToAggregationExpressionTranslator follows a similar pattern already.

There is some boilerplate in setting up the __medianMethods static field.

There isn't any boilerplate in the if statement. It's roughly the same.

One advantage of using __medianMethods is that it is VERY precise. There is no danger of false hits like there is with the IsMedianMethod approach.

Plus I noticed the StandardDeviationMethodsToAggregationExpressionTranslator follows a similar pattern already.

Yes. That's older code that predates the newer practice of being more precise.

...tionExpressionTranslators/MethodTranslators/MedianMethodToAggregationExpressionTranslator.cs

rstam

Changes requested

src/MongoDB.Driver/Linq/Linq3Implementation/Ast/Expressions/AstComplexAccumulatorExpression.cs

src/MongoDB.Driver/Linq/Linq3Implementation/Ast/Visitors/AstNodeVisitor.cs

...xpressionTranslators/MethodTranslators/MedianMethodToAggregationExpressionTranslatorTests.cs

...xpressionTranslators/MethodTranslators/WindowMethodToAggregationExpressionTranslatorTests.cs

...er.Tests/Linq/Linq3ImplementationWithLinq2Tests/Translators/AggregateGroupTranslatorTests.cs

rstam · 2025-07-29T18:53:35Z

...tionExpressionTranslators/MethodTranslators/MedianMethodToAggregationExpressionTranslator.cs

+            var method = expression.Method;
+            var arguments = expression.Arguments;
+
+            if (IsMedianMethod(method))


There is some boilerplate in setting up the __medianMethods static field.

There isn't any boilerplate in the if statement. It's roughly the same.

One advantage of using __medianMethods is that it is VERY precise. There is no danger of false hits like there is with the IsMedianMethod approach.

Plus I noticed the StandardDeviationMethodsToAggregationExpressionTranslator follows a similar pattern already.

Yes. That's older code that predates the newer practice of being more precise.

...xpressionTranslators/MethodTranslators/MedianMethodToAggregationExpressionTranslatorTests.cs

src/MongoDB.Driver/Linq/Linq3Implementation/Ast/Visitors/AstNodeVisitor.cs

rstam · 2025-07-30T01:41:10Z

src/MongoDB.Driver/Linq/Linq3Implementation/Ast/Expressions/AstComplexAccumulatorExpression.cs

+
+namespace MongoDB.Driver.Linq.Linq3Implementation.Ast.Expressions
+{
+    internal sealed class AstComplexAccumulatorExpression : AstAccumulatorExpression


This is clever but I don't like the idea of a Dictionary<string, AstExpression>. A class that can represent anything in this way is not easy to work with.

I would prefer you just create two new classes AstMedianAccumulatorExpression and AstPercentileAccumulatorExpression.

That would mirror AstMedianExpression and AstPercentileExpression.

It would also avoid the messiness in the VisitComplexAccumulatorExpression method. Instead we could just have two simple methods VisitMedianAccumulatorExpression and VisitPercentileAccumulatorExpression.

The whole point of the Ast classes is to have type-safe representations of MQL, and the use of a Dictionary<string, AstExpression> throws type-safety out the window.

That's fair. I'll revert this then

src/MongoDB.Driver/Linq/Linq3Implementation/Ast/Optimizers/AstGroupingPipelineOptimizer.cs

...tionExpressionTranslators/MethodTranslators/MedianMethodToAggregationExpressionTranslator.cs

...ExpressionTranslators/MethodTranslators/PercentileMethodToAggregationExpressionTranslator.cs

...tionExpressionTranslators/MethodTranslators/WindowMethodToAggregationExpressionTranslator.cs

rstam · 2025-08-01T02:51:27Z

src/MongoDB.Driver/Linq/MongoEnumerable.cs

+        /// </summary>
+        /// <param name="source">The sequence of values.</param>
+        /// <returns>The median value.</returns>
+        public static double Median(this IEnumerable<decimal> source)


I'm still concerned that the return type is double instead of decimal.

Same below.

rstam · 2025-08-01T02:54:47Z

src/MongoDB.Driver/Linq/MongoEnumerable.cs

+        /// </summary>
+        /// <param name="source">The sequence of values.</param>
+        /// <returns>The median value.</returns>
+        public static double Median(this IEnumerable<float> source)


Enumerable.Average(IEnumerable<float>) returns float.

Should we also?

rstam · 2025-08-01T02:55:42Z

src/MongoDB.Driver/Linq/MongoEnumerable.cs

+        /// <param name="source">A sequence of values to calculate the percentiles of.</param>
+        /// <param name="percentiles">The percentiles to compute (each between 0.0 and 1.0).</param>
+        /// <returns>The percentiles of the sequence of values.</returns>
+        public static double[] Percentile(this IEnumerable<decimal> source, IEnumerable<double> percentiles)


Should we return decimal[]

Should the percentiles be decimal also to match?

rstam · 2025-08-01T02:56:48Z

src/MongoDB.Driver/Linq/MongoEnumerable.cs

+        /// <param name="source">A sequence of values to calculate the percentiles of.</param>
+        /// <param name="percentiles">The percentiles to compute (each between 0.0 and 1.0).</param>
+        /// <returns>The percentiles of the sequence of values.</returns>
+        public static double[] Percentile(this IEnumerable<float> source, IEnumerable<double> percentiles)


Should we return float[]?

SHould the percentiles web IEnumerable<flost> to match?

rstam · 2025-08-01T02:57:59Z

src/MongoDB.Driver/Linq/ISetWindowFieldsPartitionExtensions.cs

+        /// <param name="selector">The selector that selects a value from the input document.</param>
+        /// <param name="window">The window boundaries.</param>
+        /// <returns>The median of the selected values.</returns>
+        public static double Median<TInput>(this ISetWindowFieldsPartition<TInput> partition, Func<TInput, decimal> selector, SetWindowFieldsWindow window = null)


Should we return decimal?

adelinowona added 6 commits July 29, 2025 06:48

Add enumerable extensions for median and percentile

1837037

add setWindowFields extensions for median and percentile

b5e24ab

Add AstExpression support for median and percentile accumulators/wind…

0235684

…ow expressions

Implement translators for median and percentile methods

ef6c98b

Update grouping pipeline optimizer

3697ac4

add tests

84d9e39

adelinowona requested review from rstam and sanych-sun July 29, 2025 11:03

adelinowona added the improvement label Jul 29, 2025

adelinowona requested a review from a team as a code owner July 29, 2025 11:03

fix some tests failures

91efb8b

rstam approved these changes Jul 29, 2025

View reviewed changes

rstam requested changes Jul 29, 2025

View reviewed changes

sanych-sun requested changes Jul 29, 2025

View reviewed changes

rstam requested changes Jul 29, 2025

View reviewed changes

...xpressionTranslators/MethodTranslators/MedianMethodToAggregationExpressionTranslatorTests.cs Outdated Show resolved Hide resolved

sanych-sun added feature and removed improvement labels Jul 29, 2025

adelinowona added 4 commits July 29, 2025 16:48

rearrange the methods in MongoEnumerable.cs

dc6d059

add median feature and undo accidental change

cafdd06

use reflection pattern to recognize methods in translators

2604f8a

fix complex accumulator visitor bug

61824fd

adelinowona requested review from rstam and sanych-sun July 29, 2025 23:38

sanych-sun requested changes Jul 30, 2025

View reviewed changes

src/MongoDB.Driver/Linq/Linq3Implementation/Ast/Visitors/AstNodeVisitor.cs Outdated Show resolved Hide resolved

rstam requested changes Jul 30, 2025

View reviewed changes

adelinowona added 2 commits July 31, 2025 20:13

use separate classes for median and percentile accumulator ASTs

691b8aa

use StandardSerializers

0cc714d

rstam requested changes Aug 1, 2025

View reviewed changes

CSHARP-3222: Add LINQ support for median and percentile accumulators/window functions #1743

Are you sure you want to change the base?

CSHARP-3222: Add LINQ support for median and percentile accumulators/window functions #1743

Uh oh!

Conversation

adelinowona commented Jul 29, 2025

Uh oh!

rstam left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

rstam left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!