BF: Address datalad DeprecationWarnings for compatibility with datalad PR #7793, make compatible with boto3 by use of datalad interfaces#149
Merged
yarikoptic merged 5 commits intomasterfrom Mar 12, 2026
Conversation
…d PR #7793 - Replace assure_* with ensure_* (assure_ variants emit DeprecationWarning) - Remove six dependency: iteritems→.items(), string_types→str, text_type→str, PY3 guard removed, add_metaclass→metaclass=, `from six import next` removed - Replace `from mock import` with `from unittest.mock import` (stdlib) - Replace subdatasets(fulfilled=True) with subdatasets(state='present') - Replace deprecated AnnexRepo.get_file_key() with get_file_annexinfo() - Fix finalize() return-without-yield bug: when aggregate_metadata is unavailable, bare `return` dropped pipeline data preventing tag/cleanup/yield - Replace distutils.version.LooseVersion with looseversion package - Remove mock from test dependencies, add looseversion to install_requires - Update tox.ini filterwarnings for datalad-deprecated internal warnings Co-Authored-By: Claude Code 2.1.63 / Claude Opus 4.6 <noreply@anthropic.com>
- Drop EOL Python 3.7 and 3.8 from test matrix - Add Python 3.12 and 3.13 to test matrix - Update release workflow to use Python 3.13 - Fix typo: aggrement -> agreement in cfgs/common.cfg Co-Authored-By: Claude Code 2.1.63 / Claude Opus 4.6 <noreply@anthropic.com>
The standalone mock package is no longer a dependency (unittest.mock is standard since Python 3.3), so external_versions['mock'] returns None causing a TypeError on comparison. Co-Authored-By: Claude Code 2.1.63 / Claude Opus 4.6 <noreply@anthropic.com>
boto (last release 2018) is broken on Python 3.12+ due to its vendored six. Make the import conditional so the rest of the test suite and non-S3 pipelines work on modern Python. S3 crawling will raise a clear ImportError at runtime if boto is not available. Also mark boto dependency as python_version<"3.12" in setup.py. Co-Authored-By: Claude Code 2.1.63 / Claude Opus 4.6 <noreply@anthropic.com>
boto (last release 2018) is broken on Python 3.12+ due to its vendored six. Replace all boto usage with boto3 via datalad's new S3Downloader client API: - Remove boto.s3.key.Key/Prefix/DeleteMarker imports - Add _list_bucket() helper that paginates list_object_versions / list_objects_v2 and yields tagged dicts with '_type' key - Update get_key_url() to accept entry dict + bucket_name - Update get_version_for_key() to work with dicts and datetime objects - Update crawl_s3.__call__() to use downloader.client / ._bucket_name, dict-based type checks, and inline FileStatus construction - Handle ParamValidationError when authenticating with bucket-only URL - Add boto3 as explicit core dependency in setup.py - Add type annotations to all new/modified functions Co-Authored-By: Claude Code 2.1.63 / Claude Opus 4.6 <noreply@anthropic.com>
yarikoptic
changed the title
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #149 +/- ##
==========================================
- Coverage 77.51% 76.58% -0.93%
==========================================
Files 60 58 -2
Lines 4731 4733 +2
Branches 689 596 -93
==========================================
- Hits 3667 3625 -42
- Misses 902 947 +45
+ Partials 162 161 -1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
assure_*withensure_*(assure_ variants emit DeprecationWarning)from six import nextremovedfrom mock importwithfrom unittest.mock import(stdlib)returndropped pipeline data preventing tag/cleanup/yield