upgrade python version

[keighrim]

New Feature Summary

ATM, clams SDK's are based on py 3.8, but the version is reaching the end of support very soon. We'd like to migrate to a stable python version before it's too late.

I'm divided between 3.10 and 3.12, but more leaning toward 3.10 to support older (but still very well supported) linux distros that do not ship the upstream versions. For example,

• The latest debian stable still uses 3.11 as the default python
  • https://packages.debian.org/search?suite=default&section=all&arch=any&searchon=names&keywords=python3
• Ubuntu focal or jammy probably will never receive python 3.12 before they are discontinued on 2025 and 2027 respectively.
  • https://packages.ubuntu.com/search?searchon=names&keywords=python3

Related

No response

Alternatives

No response

Additional context

python support status https://devguide.python.org/versions/

[mrharpo]

Is it possible (or worth the effort) to support multiple versions of python at once?

I agree with @keighrim about preferring 3.10 for compatibility, but there's also a substantial speedup when running 3.12, some say as much as 2x for certain tasks. We probably won't get those results in general, especially if the computation if limited by the models.

But for a quick comparable example, I just ran a test with (vanilla) Whisper on GPU, where 3.12 ran about 5% faster than 3.10, with no other changes on the system.

To me, that's a "worth the effort" sort of improvement considering the amount of processing we intend to do.

[keighrim]

In theory, any code written for python 3.x will run on python 3.y as long as y >= x, because python3 versions are supposed to be backward compatible. The only (I can think of now) major "breaking" scenario of running an app written for 3.x on python 3.y is the app having a "strictly pinned" dependency that doesn't pip-install on 3.y (but still installs on 3.x). Thus, in practice, as long as app devs avoid strict pins (or upper-caps) - which I believe a better practice in general anyway, under the current python eco-system - we won't have to add extra work to support 3.12.

That being said, the SDK itself is not an exception from the issue of upper-bound dependency, and in fact this clams-python has one (https://github.com/clamsproject/clams-python/blob/453b4a7a6a1517f193cb3ce956526884d504d2e8/requirements.txt). So maybe adding more test envs with all future python versions in the CI pipeline might be a good idea, to make sure the SDK run on all python>=3.10.

[keighrim]

Upon learning of the imminent release of Debian (trixie), I revisited this issue. However, my conclusion remains the same: Python 3.10 should be the minimum, as the current Ubuntu LTS continues to ship with 3.10.