Hardcode python3 version for compat == 5?

[serhii-karelov]

I've noticed that we dynamically determine python3 version when virtualenv is being created (reference).

def get_python_version(compat):
    """Return the version of Python that an app will use.
    Based on the compat level.
    """
    if compat < 5:
        return '2.7'
    if sys.version_info.major == 3:
        return sysconfig.get_python_version()
    return subprocess.check_output((
        'python3', '-c',
        'import sysconfig; print(sysconfig.get_python_version())'
    )).strip()

In newer versions of Ubuntu python3.6 is the default and environment with incompatible interpreter can be created.

What do you think about hardcoding python3 version and display error message when there is no interpreter with required version on developer's machine?

@yola/service-engineering @yola/operations-engineering

[kosmos342]

I think we have to define hardcoded versions for another compat (e.g 3.6 for compat == 6).

[serhii-karelov]

I think we have to define hardcoded versions for another compat (e.g 3.6 for compat == 6).

I think it is safe to run python3.4 code with python3.6, but not vice versa.

The problem is that we do not run python3.6 in production, but we can create virtualenvironment in development if python3 points to python3.6.

[stefanor]

In newer versions of Ubuntu python3.6 is the default and environment with incompatible interpreter can be created.

No, please don't do this. Python 3 is python 3. You don't need to differentiate between minor versions.

[serhii-karelov]

No, please don't do this. Python 3 is python 3. You don't need to differentiate between minor versions.

Why not? Does it mean that we never use new features from newer python3.* minor releases?

In my case, yodeploy builds an environment with python3.6 and I want to take advantage of it and use new features like f-strings. I will be sure that the code will run in other environments until I push it. Only then, I may receive a feedback from Jenkins (but there is no guarantee that 100% of incompatibilities will be caught).
I think that it is more convenient to have a correct version in development rather keep in mind that code that I write with my version of the interpreter must be compatible with another version that is run in production.

I think that it is not a big problem. But it is a thing that might be easily improved and may prevent some mistakes in the future. Or maybe I'm missing something and there is a much bigger cost to incorporate such change.

There is already the case when different minor versions have small incompatibilities.

[stefanor]

Why not? Does it mean that we never use new features from newer python3.* minor releases?

No. We are using the Ubuntu system python. So on 16.04, that's 3.5, and on 18.04, that will be 3.6.

Personally, I wouldn't use 3.6's new features yet, for maximum flexibility. But you can if you need to.

I think that it is more convenient to have a correct version in development rather keep in mind that code that I write with my version of the interpreter must be compatible with another version that is run in production. ... Or maybe I'm missing something and there is a much bigger cost to incorporate such change.

This is getting to the philosophy of how we deploy applications at Yola. We give applications virtualenvs, where they get to control the python libraries they are deployed with, but we aren't containerising everything, and the application does not get to select specific versions of other parts of the system (like the Python interpreter). So, there is no "correct version" to use in development. Rather we aim to have our services run across the entire range of currently supported Ubuntu LTS versions. This is usually trivial for older applications, but requires us to hold back a little from using the latest language features.

In the context of the work you're doing, it isn't going to need to work on older Ubuntu versions. You don't have to have Python 3.2 support, for example, that'd be crazy.

I agree that it isn't ideal that applications don't get to declare the python versions they're supporting somewhere. But the solution you're proposing is to tie us to a specific version, making it far harder to migrate to future versions.

There were unrelated reasons why we cared about minor versions of python under python 2. They made it incredibly hard for us to migrate off Ubuntu 12.04 (we were stuck on it for a lot longer than I'd have liked) and I don't want to repeat that.

[Toshakins]

I think it is safe to run python3.4 code with python3.6, but not vice versa.

No, it is not. Minor versions have deprecations. You officially need to port your code to be compatible with the Python 3.6.

So, there is no "correct version" to use in development.

Literally on my first day in Yola I've stumbled into a bug that was caused by using latest Python instead of 3.5(some methods started to expect a path instead of a string). We had similar issues once or twice since then. Moreover, recently we trapped into a bug that was fixed in patch version of 3.5, but still not present in Ubuntu's repositories.

We don't need to agree which version to use(it is 3.5.2 in the previous LTS, I think?). But enforcing that restriction for build-virtualenv would make my life a little bit better.

[RayeN]

New Django requires 3.6. Can we prioritize switching everything to >= 18.04?