combine json chunks from requests

[jiapingzeng]

Description

[Describe what this change achieves]

Related Issues

4314

Check List

• New functionality includes testing.
• New functionality has been documented.
• API changes companion pull request created.
• Commits are signed per the DCO using --signoff.
• Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[nathaliellenaa]

Thanks for the fix @jiapingzeng! Wondering if you see any additional latency due to combining the chunks?

[jiapingzeng]

I didn't notice additional latency from my side. There are some concerns of collectList() running out of memory when waiting for an infinite upstream: https://stackoverflow.com/a/72042404, but I don't think that applies here as agent execute requests are a few kBs at most.

[jiapingzeng]

Tests with failures:
 - org.opensearch.ml.rest.RestMLRAGSearchProcessorIT.testBM25WithBedrockWithConversation
131 tests completed, 1 failed, 11 skipped

=== Standard output of node `node{:opensearch-ml-plugin:integTest-0}` ===

»    ↓ errors and warnings from /__w/ml-commons/ml-commons/plugin/build/testclusters/integTest-0/logs/opensearch.stdout.log ↓
» WARN ][stderr                   ] [integTest-0] Oct 14, 2025 7:57:42 PM org.opensearch.javaagent.bootstrap.AgentPolicy setPolicy
» WARN ][stderr                   ] [integTest-0] INFO: Policy attached successfully: org.opensearch.bootstrap.OpenSearchPolicy@20de05e5
» WARN ][o.o.p.PluginsService     ] [integTest-0] Missing plugin [opensearch-security], dependency of [opensearch-ml]
» WARN ][o.o.p.PluginsService     ] [integTest-0] Some features of this plugin may not function without the dependencies being installed.
»  
» WARN ][stderr                   ] [integTest-0] Oct 14, 2025 7:57:42 PM org.apache.lucene.internal.vectorization.PanamaVectorizationProvider <init>
» WARN ][stderr                   ] [integTest-0] INFO: Java vector incubator API enabled; uses preferredBitSize=256; FMA enabled
» WARN ][o.o.g.DanglingIndicesState] [integTest-0] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
» WARN ][o.o.m.a.p.TransportPredictionStreamTaskAction] [integTest-0] StreamTransportService is not available.
» WARN ][o.o.m.a.e.TransportExecuteStreamTaskAction] [integTest-0] StreamTransportService is not available.
» WARN ][o.o.d.FileBasedSeedHostsProvider] [integTest-0] expected, but did not find, a dynamic hosts list at [/__w/ml-commons/ml-commons/plugin/build/testclusters/integTest-0/config/unicast_hosts.txt]
» WARN ][a.d.h.t.HuggingFaceTokenizer] [integTest-0] maxLength is not explicitly specified, use modelMaxLength: 512
» ERROR][o.o.m.a.MLModelAutoReDeployer] [integTest-0] Failed to query need undeploy models, no action will be performed
»  org.opensearch.index.IndexNotFoundException: no such index [.plugins-ml-model]
»  	at org.opensearch.cluster.metadata.IndexNameExpressionResolver$WildcardExpressionResolver.indexNotFoundException(IndexNameExpressionResolver.java:1101) ~[opensearch-3.3.0-SNAPSHOT.jar:3.3.0-SNAPSHOT]
»  	at org.opensearch.cluster.metadata.IndexNameExpressionResolver$WildcardExpressionResolver.innerResolve(IndexNameExpressionResolver.java:1038) ~[opensearch-3.3.0-SNAPSHOT.jar:3.3.0-SNAPSHOT]
»  	at org.opensearch.cluster.metadata.IndexNameExpressionResolver$WildcardExpressionResolver.resolve(IndexNameExpressionResolver.java:994) ~[opensearch-3.3.0-SNAPSHOT.jar:3.3.0-SNAPSHOT]

Don't think test failure is related to my change. Tried ./gradlew build locally and it passes.

[codecov]

Codecov Report

❌ Patch coverage is 0% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.12%. Comparing base (3950a87) to head (17f9415).

Files with missing lines	Patch %	Lines
.../opensearch/ml/rest/RestMLExecuteStreamAction.java	0.00%	14 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #4317      +/-   ##
============================================
- Coverage     80.14%   80.12%   -0.03%     
+ Complexity    10172    10171       -1     
============================================
  Files           854      854              
  Lines         44219    44227       +8     
  Branches       5113     5114       +1     
============================================
- Hits          35440    35437       -3     
- Misses         6630     6641      +11     
  Partials       2149     2149              

Flag	Coverage Δ
ml-commons	80.12% <0.00%> (-0.03%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[jiapingzeng]

Looks like CI is passing on Windows but still failing on Linux due to unrelated errors. I will add UTs and run sanity tests locally and then we can merge this.

[jiapingzeng]

Added UTs for combineChunks. Not possible to create UT for prepareRequest as chunk collection happens async.

However, ran the below sanity tests:

1. Verified that sample workflow from here still works: [FEATURE] Agent Execute Stream #4212
2. Tested chunked request:

(
    echo -n '{"parameters":'
    sleep 0.1
    echo -n '{"question":'
    sleep 0.1
    echo -n '"What is'
    sleep 0.1
    echo -n ' the status'
    sleep 0.1
    echo -n ' of my'
    sleep 0.1
    echo -n ' cluster?"}}'
) > "$FIFO" &

curl -N -X POST "${OPENSEARCH_URL}/_plugins/_ml/agents/${AGENT_ID}/_execute/stream" \
  -H 'Content-Type: application/json' \
  -H 'Transfer-Encoding: chunked' \
  --data-binary @"$FIFO"