sync with open source how#118
Draft
lesterhaynes wants to merge 8228 commits intolinkedin:li_trunkfrom
Draft
Conversation
* Fix race condition in stager.py when starting multiple pipelines concurrently (fixes #36847) * remove race condition test
…ks (#37058) Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.32.3 to 1.32.4. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json) - [Commits](aws/aws-sdk-go-v2@v1.32.3...v1.32.4) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/config dependency-version: 1.32.4 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Use internal DEFAULT_PICKLE_LIB * Enable only on Dataflow
* Bump Beartype bounds to >= 0.22.0 * image requirement updates * widen bounds * add pyjwt to py313 base image
…37028) Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.5.0 to 2.6.0. - [Release notes](https://github.com/urllib3/urllib3/releases) - [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst) - [Commits](urllib3/urllib3@2.5.0...2.6.0) --- updated-dependencies: - dependency-name: urllib3 dependency-version: 2.6.0 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.93.0 to 1.93.1. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json) - [Commits](aws/aws-sdk-go-v2@service/s3/v1.93.0...service/s3/v1.93.1) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/service/s3 dependency-version: 1.93.1 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2) from 0.33.0 to 0.34.0. - [Commits](golang/oauth2@v0.33.0...v0.34.0) --- updated-dependencies: - dependency-name: golang.org/x/oauth2 dependency-version: 0.34.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* add SHOW command * add SHOW CURRENT * add SHOW CURRENT * add LIKE pattern * spotless * use SqlCall; remove @nullable * spotless * minor fixes * use PrintWriter and flush all at once * correct PrintWriter usage * cleanup * fix name * address comments * spotless
* Unpin the pip version in setupVirtualenv * trigger a python precommit to test
Bumps [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) from 1.20.13 to 1.20.15. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json) - [Commits](aws/aws-sdk-go-v2@service/emr/v1.20.13...service/emr/v1.20.15) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager dependency-version: 1.20.15 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
) Bumps [nodemailer](https://github.com/nodemailer/nodemailer) from 7.0.7 to 7.0.11. - [Release notes](https://github.com/nodemailer/nodemailer/releases) - [Changelog](https://github.com/nodemailer/nodemailer/blob/master/CHANGELOG.md) - [Commits](nodemailer/nodemailer@v7.0.7...v7.0.11) --- updated-dependencies: - dependency-name: nodemailer dependency-version: 7.0.11 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [node-forge](https://github.com/digitalbazaar/forge) from 1.3.1 to 1.3.2. - [Changelog](https://github.com/digitalbazaar/forge/blob/main/CHANGELOG.md) - [Commits](digitalbazaar/forge@v1.3.1...v1.3.2) --- updated-dependencies: - dependency-name: node-forge dependency-version: 1.3.2 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* stash * done. * Add test. * Comments. * comments. * Comments. --------- Co-authored-by: Claude <cvandermerwe@google.com>
Bumps [js-yaml](https://github.com/nodeca/js-yaml) from 4.1.0 to 4.1.1. - [Changelog](https://github.com/nodeca/js-yaml/blob/master/CHANGELOG.md) - [Commits](nodeca/js-yaml@4.1.0...4.1.1) --- updated-dependencies: - dependency-name: js-yaml dependency-version: 4.1.1 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* remove groovy pubsublite dependencies * remove checkstyle suppressions * remove role config for pubsublite service and update role files
#37428) * Refactor: Unify batching args in ModelHandler constructors - Added 'max_batch_weight' and 'element_size_fn' to __init__ of all ModelHandlers (PyTorch, Sklearn, TF, ONNX, XGBoost, TensorRT, Hugging Face, vLLM, VertexAI). - Updated subclasses to delegate these args to 'super().__init__' or internal batching kwargs. - Removed 'with_element_size_fn' builder method from base class to enforce API consistency. - Updated tests to reflect the new API signature. * Address review comments: refactor tests and fix linting
…iguration (#37356) * [#37198] Make withBackOffSupplier public to enable bounded retry configuration Users need to configure bounded backoff to prevent infinite retry loops. Making withBackOffSupplier public allows users to set FluentBackoff.DEFAULT.withMaxRetries(n) and control retry behavior. Changes: - Changed withBackOffSupplier() visibility from package-private to public - Added comprehensive integration test with zero-delay BoundedBackOff - Test verifies: responses empty, 1 failure emitted, call count = maxRetries+1 The test uses a serializable BoundedBackOff class with assertions on both PAssert (pipeline outputs) and Metrics (retry counts) to ensure bounded retry behavior works correctly. Fixes #37198 Related to #37176 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * Add explicit public modifier to withSleeperSupplier method Address review feedback from @damccorm to explicitly add the public keyword to withSleeperSupplier() for consistency. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Fix Spotless formatting for withSleeperSupplier method Break long line to comply with code style requirements. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Bumps [github.com/lib/pq](https://github.com/lib/pq) from 1.10.9 to 1.11.1. - [Release notes](https://github.com/lib/pq/releases) - [Changelog](https://github.com/lib/pq/blob/master/CHANGELOG.md) - [Commits](lib/pq@v1.10.9...v1.11.1) --- updated-dependencies: - dependency-name: github.com/lib/pq dependency-version: 1.11.1 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
njeevan233-spec
approved these changes
Feb 1, 2026
* Upgrade GCP Bom to LTS version * license
#37112) * Allow multiprocessshared to spawn process and delete directly with obj * Remove oom protection * Resolve comments * Rename unsafe_hard_delete for the proxy object to prevent collision * Remove support for proxy on proxy to avoid complexity * Fix import order * Update reap test to be compatiable for windows * Update print to logging * Try to tearDown test in a cleaner way * Try patching atexit call to prevent hanging on window * Try weakref so windows can GC the process * Try GC manually to make sure p is cleaned up * Use a different way to check if parent is alive * Close the pipe atexit as well
* Disable build isolation for workflow tarball by default. Add an env var to enable it if needed. * Apply build isolation setting to all pip install commands. Use experiment to control. * Only search experiment in experiment options.
* [TypeScript] Add LRU cache eviction to CombinePerKeyPrecombineOperator - Added touchKey() method to move accessed keys to end (most recently used) - Added flushLRU() method to evict entries from front (least recently used) - Updated process() to track key usage for proper LRU ordering JavaScript Maps preserve insertion order, so we implement LRU by: - On access: delete and re-insert the key (moves to end) - On eviction: iterate from front (oldest entries first) Fixes #37465 * Address review comments: remove flush, inline touchKey
* remove pubsublite logs * update branch * add note about auth * correct numbering * remove whitespace
* Add Take transform and take() in transforms/util.py * Add PCollection.take(n) in pvalue.py * Add TakeTest in util_test.py * Update CHANGES.md Fixes #37429
…7472) * fix: nil pointer dereference * fix infinite loop
Bumps [pip](https://github.com/pypa/pip) from 25.3 to 26.0. - [Changelog](https://github.com/pypa/pip/blob/main/NEWS.rst) - [Commits](pypa/pip@25.3...26.0) --- updated-dependencies: - dependency-name: pip dependency-version: '26.0' dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…tes-container Fix postcommit python validates container dataflow
* gradle and formating * openai module fix * add testing doc * revert deps
…37113) * Add model manager that automatically manage model across processes * Add pydoc and move gpu detection to start * Add comments and helper function to make it easier to understand the code and cleanup some code logics * Add TODO for threading * Remove tracked model proxy and have model manager store tags instead of model instance * Fix import order * Clean up and logs * Added timeout for waiting too long on model acquire * Throw error if timeout * Add test for timeout and adjust * Update sdks/python/apache_beam/ml/inference/model_manager.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update sdks/python/apache_beam/ml/inference/model_manager.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Gemini clean up * Update sdks/python/apache_beam/ml/inference/model_manager_test.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update GPU monitor test * Format * Cleanup upating is_unkown logic * Try to fix flake * Fix import order * Fix random seed to avoid flake * Fix identation * Try fixing doc again --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Fix linting/compilation violation * fmt
* Increment Pyarrow Bounds * fix dynamic work rebalancing test * fix pandas pin
* Add tagged typehint support. * Just warn when bare tagged output * Remove contains tagged output check. * Mapped bare TaggedOutput to Any * Extract tagged outputs after strip_iterable.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please add a meaningful description for your change here
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, commentfixes #<ISSUE NUMBER>instead.CHANGES.mdwith noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI.