Skip to content

Conversation

@brianf-aws
Copy link
Contributor

@brianf-aws brianf-aws commented Oct 22, 2025

Description

Upon making a CVE fix #4298 . which involved bumping netty, there was a netty exception.

? ERROR][o.o.m.e.a.a.MLAgentExecutor] [integTest-0] Failed to run conversational agent
?  org.opensearch.OpenSearchStatusException: Error communicating with remote model: java.lang.IllegalStateException: unexpected message type: LastHttpContent$1, state: 0
?  	at org.opensearch.ml.engine.algorithms.remote.MLSdkAsyncHttpResponseHandler.onError(MLSdkAsyncHttpResponseHandler.java:108) [opensearch-ml-algorithms-2.19.4.0-SNAPSHOT.jar:?]
?  	at software.amazon.awssd

There exists a cherry pick which bumped netty on mainline but required code changes
#4175 . The issue here is that the version catalog in that mainline commit was not synced from core 2.19.4 . Making the change non-trivial

image

Reviewer objectives

  • To ensure at least 1 CI passes that no errors are related to netty
  • To review commits from use mainline versions.aws via hardcode and above
  • To note that the lack of version catalog is because core's 2.19.4 version catalog is 11 months old!

Related Issues

Resolves the snapshot PR #4143

Next steps

  • backport the Core version catalog to 2.19.4 [OS core]
  • Update 2.19.4 to have proper version catalog [ML-Commons]

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@brianf-aws brianf-aws changed the title 2.19.4 CVE fix Address Netty failure at Agent Execute runtime Oct 22, 2025
@brianf-aws brianf-aws requested a deployment to ml-commons-cicd-env-require-approval October 22, 2025 23:51 — with GitHub Actions Waiting
@brianf-aws brianf-aws requested a deployment to ml-commons-cicd-env-require-approval October 22, 2025 23:51 — with GitHub Actions Waiting
@brianf-aws brianf-aws requested a deployment to ml-commons-cicd-env-require-approval October 22, 2025 23:51 — with GitHub Actions Waiting
@brianf-aws brianf-aws requested a deployment to ml-commons-cicd-env-require-approval October 22, 2025 23:51 — with GitHub Actions Waiting
@brianf-aws brianf-aws requested a deployment to ml-commons-cicd-env-require-approval October 22, 2025 23:51 — with GitHub Actions Waiting
@brianf-aws brianf-aws requested a deployment to ml-commons-cicd-env-require-approval October 22, 2025 23:51 — with GitHub Actions Waiting
@brianf-aws brianf-aws requested a deployment to ml-commons-cicd-env-require-approval October 23, 2025 00:04 — with GitHub Actions Waiting
@brianf-aws brianf-aws requested a deployment to ml-commons-cicd-env-require-approval October 23, 2025 00:04 — with GitHub Actions Waiting
@brianf-aws brianf-aws requested a deployment to ml-commons-cicd-env-require-approval October 23, 2025 00:04 — with GitHub Actions Waiting
@brianf-aws brianf-aws requested a deployment to ml-commons-cicd-env-require-approval October 23, 2025 00:04 — with GitHub Actions Waiting
@brianf-aws brianf-aws requested a deployment to ml-commons-cicd-env-require-approval October 23, 2025 00:04 — with GitHub Actions Waiting
@brianf-aws brianf-aws requested a deployment to ml-commons-cicd-env-require-approval October 23, 2025 00:04 — with GitHub Actions Waiting
@brianf-aws brianf-aws had a problem deploying to ml-commons-cicd-env-require-approval October 23, 2025 00:06 — with GitHub Actions Failure
@brianf-aws brianf-aws had a problem deploying to ml-commons-cicd-env-require-approval October 23, 2025 00:06 — with GitHub Actions Error
@brianf-aws brianf-aws had a problem deploying to ml-commons-cicd-env-require-approval October 23, 2025 00:06 — with GitHub Actions Failure
@brianf-aws brianf-aws had a problem deploying to ml-commons-cicd-env-require-approval October 23, 2025 00:06 — with GitHub Actions Error
@brianf-aws brianf-aws had a problem deploying to ml-commons-cicd-env-require-approval October 23, 2025 00:06 — with GitHub Actions Error
brianf-aws and others added 6 commits October 26, 2025 17:12
Signed-off-by: Brian Flores <iflorbri@amazon.com>
Signed-off-by: Brian Flores <iflorbri@amazon.com>
Signed-off-by: Brian Flores <iflorbri@amazon.com>
* fix model it by replace claude v1/v2

Signed-off-by: xinyual <xinyual@amazon.com>

* remove useless change

Signed-off-by: xinyual <xinyual@amazon.com>

---------

Signed-off-by: xinyual <xinyual@amazon.com>
Signed-off-by: Brian Flores <iflorbri@amazon.com>
…opensearch-project#3761)

Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Brian Flores <iflorbri@amazon.com>
* fix Cohere IT

Signed-off-by: xinyual <xinyual@amazon.com>

* apply spotless

Signed-off-by: xinyual <xinyual@amazon.com>

* delete useless it

Signed-off-by: xinyual <xinyual@amazon.com>

---------

Signed-off-by: xinyual <xinyual@amazon.com>
Signed-off-by: Brian Flores <iflorbri@amazon.com>
@brianf-aws brianf-aws had a problem deploying to ml-commons-cicd-env-require-approval October 27, 2025 00:13 — with GitHub Actions Error
@brianf-aws brianf-aws had a problem deploying to ml-commons-cicd-env-require-approval October 27, 2025 00:13 — with GitHub Actions Error
@brianf-aws brianf-aws had a problem deploying to ml-commons-cicd-env-require-approval October 27, 2025 00:13 — with GitHub Actions Failure
@brianf-aws brianf-aws had a problem deploying to ml-commons-cicd-env-require-approval October 27, 2025 00:13 — with GitHub Actions Error
@brianf-aws brianf-aws had a problem deploying to ml-commons-cicd-env-require-approval October 27, 2025 00:13 — with GitHub Actions Error
@brianf-aws brianf-aws had a problem deploying to ml-commons-cicd-env-require-approval October 27, 2025 00:13 — with GitHub Actions Failure
@brianf-aws
Copy link
Contributor Author

Hi Maintainers can we please kickoff CI again?

There was 2 PRs missing from 2.19

@brianf-aws brianf-aws temporarily deployed to ml-commons-cicd-env-require-approval October 27, 2025 17:21 — with GitHub Actions Inactive
@brianf-aws brianf-aws had a problem deploying to ml-commons-cicd-env-require-approval October 27, 2025 17:21 — with GitHub Actions Failure
@brianf-aws brianf-aws temporarily deployed to ml-commons-cicd-env-require-approval October 27, 2025 17:21 — with GitHub Actions Inactive
@brianf-aws brianf-aws temporarily deployed to ml-commons-cicd-env-require-approval October 27, 2025 17:21 — with GitHub Actions Inactive
@brianf-aws brianf-aws temporarily deployed to ml-commons-cicd-env-require-approval October 27, 2025 17:21 — with GitHub Actions Inactive
@brianf-aws brianf-aws temporarily deployed to ml-commons-cicd-env-require-approval October 27, 2025 17:21 — with GitHub Actions Inactive
@gaiksaya
Copy link
Member

The lack of backport PRs makes this experience very painful (Having to reverse search for fixes hoping that the specified PRs include the error in the description field) to understand I'm wondering if we need to tune some workflow to make sure all changes get introduced which can be driven by Infra team. (cc: @gaiksaya )

The PR authors or maintainers need to decide which PRs should be backported. Once they are up, infra may take a look into merging during releases. However, the decision to backport or not solely depends on maintainers

@codecov
Copy link

codecov bot commented Oct 27, 2025

Codecov Report

❌ Patch coverage is 90.90909% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.36%. Comparing base (b6b6c84) to head (b8e69bd).
⚠️ Report is 1 commits behind head on 2.19.

Files with missing lines Patch % Lines
...arch/ml/common/httpclient/MLHttpClientFactory.java 93.33% 0 Missing and 1 partial ⚠️
...main/java/org/opensearch/ml/utils/MLNodeUtils.java 83.33% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               2.19    #4339      +/-   ##
============================================
+ Coverage     80.32%   80.36%   +0.03%     
- Complexity     6967     6977      +10     
============================================
  Files           610      610              
  Lines         30438    30424      -14     
  Branches       3411     3407       -4     
============================================
- Hits          24450    24449       -1     
+ Misses         4521     4516       -5     
+ Partials       1467     1459       -8     
Flag Coverage Δ
ml-commons 80.36% <90.90%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@brianf-aws brianf-aws requested a deployment to ml-commons-cicd-env-require-approval October 27, 2025 18:18 — with GitHub Actions Waiting
@brianf-aws brianf-aws requested a deployment to ml-commons-cicd-env-require-approval October 27, 2025 18:18 — with GitHub Actions Waiting
@brianf-aws brianf-aws requested a deployment to ml-commons-cicd-env-require-approval October 27, 2025 18:18 — with GitHub Actions Waiting
@brianf-aws
Copy link
Contributor Author

Okay great the CI Finally passed on Linux 21 & 17 we can merge now

cc: @gaiksaya

@gaiksaya
Copy link
Member

Okay great the CI Finally passed on Linux 21 & 17 we can merge now

cc: @gaiksaya

Can we get one of the maintainers to approve please?

@mingshl mingshl merged commit 9e5b341 into opensearch-project:2.19 Oct 27, 2025
15 of 27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants