FEATURE: Summarize the steps when max steps limit reached #4184

PauiC · 2025-09-22T07:10:09Z

Description

This feature provides intelligent summarization when ML Commons Chat Agents reach their maximum iteration limit without completing tasks.

Key Components:

Trigger Mechanism: Activates when agent execution reaches the max_iteration parameter limit

Control Parameter: summarize_when_max_iteration (boolean, default: false) enables/disables the feature

Related Issues

#4150

Check List

New functionality includes testing.
New functionality has been documented.
API changes companion pull request created.
Commits are signed per the DCO using --signoff.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Jiaru Jiang <jiaruj@amazon.com>

… into max-step-summary

PauiC · 2025-09-22T08:41:06Z

Here is an example

xinyual · 2025-09-22T09:23:18Z

Do we plan to summarize the memory when it's too long?

PauiC · 2025-09-22T09:31:27Z

Do we plan to summarize the memory when it's too long?

As far as I know, there’s no such plan.

codecov · 2025-09-22T23:09:31Z

Codecov Report

❌ Patch coverage is 64.28571% with 20 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.01%. Comparing base (7bf4b54) to head (51dbde8).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
.../ml/engine/algorithms/agent/MLChatAgentRunner.java	64.28%	10 Missing and 10 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #4184      +/-   ##
============================================
- Coverage     82.02%   82.01%   -0.01%     
- Complexity     9195     9206      +11     
============================================
  Files           789      789              
  Lines         39541    39597      +56     
  Branches       4387     4396       +9     
============================================
+ Hits          32432    32475      +43     
- Misses         5219     5224       +5     
- Partials       1890     1898       +8

Flag	Coverage Δ
ml-commons	`82.01% <64.28%> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: Jiaru Jiang <jiaruj@amazon.com>

… into max-step-summary

pyek-bot · 2025-09-30T05:45:49Z

ml-algorithms/src/main/java/org/opensearch/ml/engine/algorithms/agent/MLChatAgentRunner.java

+                sendTraditionalMaxIterationsResponse(
+                    sessionId,
+                    listener,
+                    question,
+                    parentInteractionId,
+                    verbose,
+                    traceDisabled,
+                    traceTensors,
+                    conversationIndexMemory,
+                    traceNumber,
+                    additionalInfo,
+                    summaryThought,
+                    0,
+                    tools
+                );
+            }, e -> { log.warn("Failed to generate LLM summary", e); }));
+        } else {
+            sendTraditionalMaxIterationsResponse(
+                sessionId,
+                listener,
+                question,
+                parentInteractionId,
+                verbose,
+                traceDisabled,
+                traceTensors,
+                conversationIndexMemory,
+                traceNumber,
+                additionalInfo,
+                lastThought,
+                maxIterations,
+                tools
+            );


in both places, we are using the same method? so can we just define the parameters that change and call the method once

pyek-bot · 2025-09-30T05:46:26Z

ml-algorithms/src/main/java/org/opensearch/ml/engine/algorithms/agent/MLChatAgentRunner.java

+        String incompleteResponse;
+        if (maxIterations == 0) {
+            incompleteResponse = lastThought.get();
+        } else {
+            incompleteResponse = (lastThought.get() != null && !lastThought.get().isEmpty() && !"null".equals(lastThought.get()))
+                ? String.format("%s. Last thought: %s", String.format(MAX_ITERATIONS_MESSAGE, maxIterations), lastThought.get())
+                : String.format(MAX_ITERATIONS_MESSAGE, maxIterations);
+        }


is maxIterations here being used as a flag? I don't understand the need for this

Yes, maxIterations is used as a flag so we can reuse the sendTraditionalMaxIterationsResponse method.

ml-algorithms/src/main/java/org/opensearch/ml/engine/algorithms/agent/MLChatAgentRunner.java

Signed-off-by: Jiaru Jiang <jiaruj@amazon.com>

PauiC added 2 commits September 22, 2025 14:51

init max step summary

af5ea63

Signed-off-by: Jiaru Jiang <jiaruj@amazon.com>

fix:recover build.gradle

51dbde8

Signed-off-by: Jiaru Jiang <jiaruj@amazon.com>

PauiC temporarily deployed to ml-commons-cicd-env-require-approval September 22, 2025 07:11 — with GitHub Actions Inactive

Merge branch 'main' of https://github.com/opensearch-project/ml-commons…

45fb807

… into max-step-summary

PauiC marked this pull request as ready for review September 22, 2025 08:41

PauiC requested review from HenryL27, Zhangxunmt, austintlee, b4sjoo, dhrubo-os, jngz-es, mingshl, model-collapse, pyek-bot, rbhavna, sam-herman, xinyual, ylwu-amzn and zane-neo as code owners September 22, 2025 08:41

PauiC requested a deployment to ml-commons-cicd-env-require-approval September 22, 2025 23:10 — with GitHub Actions Waiting

PauiC added 2 commits September 23, 2025 10:12

add:increase test coverage

86e65a1

Signed-off-by: Jiaru Jiang <jiaruj@amazon.com>

fix:spotlessApply

24c90ec

Signed-off-by: Jiaru Jiang <jiaruj@amazon.com>

Merge branch 'main' of https://github.com/opensearch-project/ml-commons…

bdbb695

… into max-step-summary

PauiC requested a deployment to ml-commons-cicd-env-require-approval September 30, 2025 03:37 — with GitHub Actions Waiting

pyek-bot reviewed Sep 30, 2025

View reviewed changes

ml-algorithms/src/main/java/org/opensearch/ml/engine/algorithms/agent/MLChatAgentRunner.java Show resolved Hide resolved

pyek-bot reviewed Sep 30, 2025

View reviewed changes

ml-algorithms/src/main/java/org/opensearch/ml/engine/algorithms/agent/MLChatAgentRunner.java Show resolved Hide resolved

fix: delete stop

bff9bb5

Signed-off-by: Jiaru Jiang <jiaruj@amazon.com>

PauiC dismissed xinyual’s stale review via bff9bb5 October 10, 2025 05:42

PauiC requested a deployment to ml-commons-cicd-env-require-approval October 10, 2025 05:44 — with GitHub Actions Waiting

fix: refactor

750904e

Signed-off-by: Jiaru Jiang <jiaruj@amazon.com>

PauiC requested a deployment to ml-commons-cicd-env-require-approval October 10, 2025 07:08 — with GitHub Actions Waiting

fix: json serialization

2ddad4c

Signed-off-by: Jiaru Jiang <jiaruj@amazon.com>

PauiC requested a deployment to ml-commons-cicd-env-require-approval October 10, 2025 07:56 — with GitHub Actions Waiting

fix: parameter

8de0f73

Signed-off-by: Jiaru Jiang <jiaruj@amazon.com>

PauiC requested a deployment to ml-commons-cicd-env-require-approval October 13, 2025 06:01 — with GitHub Actions Waiting

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

FEATURE: Summarize the steps when max steps limit reached #4184

FEATURE: Summarize the steps when max steps limit reached #4184

PauiC commented Sep 22, 2025 •

edited

Loading

Uh oh!

PauiC commented Sep 22, 2025

Uh oh!

xinyual commented Sep 22, 2025

Uh oh!

PauiC commented Sep 22, 2025

Uh oh!

codecov bot commented Sep 22, 2025 •

edited

Loading

Uh oh!

pyek-bot Sep 30, 2025

Uh oh!

pyek-bot Sep 30, 2025

Uh oh!

PauiC Oct 10, 2025

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

FEATURE: Summarize the steps when max steps limit reached #4184

Are you sure you want to change the base?

FEATURE: Summarize the steps when max steps limit reached #4184

Conversation

PauiC commented Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Check List

Uh oh!

PauiC commented Sep 22, 2025

Uh oh!

xinyual commented Sep 22, 2025

Uh oh!

PauiC commented Sep 22, 2025

Uh oh!

codecov bot commented Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

pyek-bot Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

pyek-bot Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

PauiC Oct 10, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

PauiC commented Sep 22, 2025 •

edited

Loading

codecov bot commented Sep 22, 2025 •

edited

Loading