From 08db644ea3ef4750db1e71b579b5daf998e70ff7 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Mon, 16 Mar 2026 22:18:05 +0100 Subject: [PATCH 1/5] PEP 803: Add `abi3t` filename tag --- peps/pep-0803.rst | 56 +++++++++++++++++++++++++++++++++++++---------- 1 file changed, 45 insertions(+), 11 deletions(-) diff --git a/peps/pep-0803.rst b/peps/pep-0803.rst index c44cf0b78f6..ef056591300 100644 --- a/peps/pep-0803.rst +++ b/peps/pep-0803.rst @@ -288,6 +288,20 @@ The tag ``abi3t`` is chosen to reflect the fact that this ABI is similar to uses the letter ``t`` in existing, version-specific ABI tags like ``cp314t``). +Filename tag +------------ + +On systems that use the ``abi3`` tag in filenames, a new filename tag +(``abi3t``) is added so that older stable ABI extensions +(:samp:`{name}.abi3.so`) can be installed in the same directory as ones that +support Stable ABI for free-threaded Python (:samp:`{name}.abi3t.so`). + +There can only be one ABI tag in a filename (there is no concept of "compressed +tag sets like in wheel tags), so extensions that are compatible with both ABIs +at once need to use *one* of the tags -- the new one (``abi3t``), which +is currently unused. + + Specification ============= @@ -461,14 +475,34 @@ and may be removed in future CPython versions, if the internal object layout needs to change. -The ``abi3t`` wheel tag ------------------------ +The ``abi3t`` wheel and filename tags +------------------------------------- Wheels that use a stable ABI compatible with free-threading CPython builds should use a new `ABI tag`_: ``abi3t``. .. _ABI tag: https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#abi-tag +On systems where filenames of Stable ABI extensions end with ``.abi3.so``, +extensions that support free-threading should instead use ``abi3t.so``. +This includes extensions compatible with *both* ``abi3`` and ``abi3t``. + +On these systems, all builds of CPython -- GIL-enabled and free-threaded -- +will load extensions with the ``abi3t`` tag. +Free-threaded builds will -- unlike in 3.14 -- *not* load extensions +with the ``abi3`` tag. +If files are present with both tags, GIL-enabled builds will prefer +"their" ``*.abi3.so`` over ``*.abi3t.so``. + +Put another way, ``importlib.machinery.EXTENSION_SUFFIXES`` will be +(for ``x86_64-linux-gnu`` builds of CPython): + +* ``python3.15``: + ``['.cpython-315-x86_64-linux-gnu.so', '.abi3.so', '.abi3t.so', '.so']`` +* ``python3.15t``: + ``['.cpython-315-x86_64-linux-gnu.so', '.abi3t.so', '.so']`` + + Recommendations for installers ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -504,6 +538,13 @@ In the above, :samp:`{v}` stands for a the lowest Python version with which the extension should be compatible, in :c:func:`Py_PACK_VERSION` format. In the cases above, this version must be set to 3.15 or higher. +In both cases, if a resulting extension's *filename* would use the ``abi3`` +tag (that is, :samp:`{modulename}.abi3.so` on Linux, macOS, and similar), +the ``abi3t`` tag should be used instead (that is, +:samp:`{modulename}.abi3t.so` on those systems). +If this tag is not used, there should be no change. (This is always the case +on Windows, where the filename should remain :samp:`{name}.pyd`.) + .. _compressed tag set: https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#compressed-tag-sets The version of the Stable ABI, both ``abi3`` and ``abi3t``, is indicated by @@ -783,15 +824,8 @@ Naming this ``abi4`` Instead of ``abi3t``, we could “bump the version” and use ``abi4`` instead. The difference is largely cosmetic. -However, one thing this PEP does not propose is changing the *filename* -tag: extensions will be named with the extensions like ``.abi3.so``. -Changing this while keeping compatibility with GIL-enabled builds would be -an unnecessary technical change. - -Using ``abi3.abi4`` in wheel tags but only ``.abi3`` in filenames would -look more inconsistent than ``abi3.abi3t`` and ``.abi3``. - -If we added an ``abi4`` tag, the ``Py_LIMITED_API`` value would either need to: +If we added an ``abi4`` tag, the value of the opt-in macro (``Py_TARGET_ABI4`` +or ``Py_LIMITED_API`` or some such) would either need to: * change to start with ``4`` to match ``abi4``, but no longer correspond to ``PY_VERSION_HEX`` (making it harder to generate and check), or From 8ddba5f3890da5c4881ba9210933a60a03ca1889 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Thu, 19 Mar 2026 17:21:58 +0100 Subject: [PATCH 2/5] Bare ``.so`` for ``abi3.abi3t`` plus light editing --- peps/pep-0803.rst | 115 +++++++++++++++++++++++++++++----------------- 1 file changed, 72 insertions(+), 43 deletions(-) diff --git a/peps/pep-0803.rst b/peps/pep-0803.rst index ef056591300..9e823ccf4cc 100644 --- a/peps/pep-0803.rst +++ b/peps/pep-0803.rst @@ -16,7 +16,7 @@ Abstract ======== Add a new variant of the Stable ABI, called “Stable ABI for Free-Threaded -Python” (or ``abi3t`` for short), and a corresponding API limitations. +Python” (or ``abi3t`` for short). ``abi3t`` will be based on the existing Stable ABI (``abi3``), but make the :c:type:`PyObject` structure opaque. @@ -288,20 +288,6 @@ The tag ``abi3t`` is chosen to reflect the fact that this ABI is similar to uses the letter ``t`` in existing, version-specific ABI tags like ``cp314t``). -Filename tag ------------- - -On systems that use the ``abi3`` tag in filenames, a new filename tag -(``abi3t``) is added so that older stable ABI extensions -(:samp:`{name}.abi3.so`) can be installed in the same directory as ones that -support Stable ABI for free-threaded Python (:samp:`{name}.abi3t.so`). - -There can only be one ABI tag in a filename (there is no concept of "compressed -tag sets like in wheel tags), so extensions that are compatible with both ABIs -at once need to use *one* of the tags -- the new one (``abi3t``), which -is currently unused. - - Specification ============= @@ -484,29 +470,27 @@ should use a new `ABI tag`_: ``abi3t``. .. _ABI tag: https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#abi-tag On systems where filenames of Stable ABI extensions end with ``.abi3.so``, -extensions that support free-threading should instead use ``abi3t.so``. -This includes extensions compatible with *both* ``abi3`` and ``abi3t``. - -On these systems, all builds of CPython -- GIL-enabled and free-threaded -- -will load extensions with the ``abi3t`` tag. -Free-threaded builds will -- unlike in 3.14 -- *not* load extensions -with the ``abi3`` tag. -If files are present with both tags, GIL-enabled builds will prefer -"their" ``*.abi3.so`` over ``*.abi3t.so``. - -Put another way, ``importlib.machinery.EXTENSION_SUFFIXES`` will be -(for ``x86_64-linux-gnu`` builds of CPython): - -* ``python3.15``: - ``['.cpython-315-x86_64-linux-gnu.so', '.abi3.so', '.abi3t.so', '.so']`` -* ``python3.15t``: +extensions compatible with ``abi3t`` should instead use ``.abi3t.so``. +Extensions compatible with *both* ``abi3`` and ``abi3t`` should not use +a tag; they sould end in ``.so`` only. + +On these systems, free-threaded builds of CPython will load extensions +with the ``abi3t`` tag. + +For example, for ``x86_64-linux-gnu`` builds of CPython, +``importlib.machinery.EXTENSION_SUFFIXES`` will be: + +* for ``python3.15``: + ``['.cpython-315-x86_64-linux-gnu.so', '.abi3.so', '.so']`` +* for ``python3.15t``: ``['.cpython-315-x86_64-linux-gnu.so', '.abi3t.so', '.so']`` Recommendations for installers ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Package installers should treat this tag as completely separate from ``abi3``. +Package installers should treat the ``abi3t`` tag as completely separate +from ``abi3``. They should allow ``abi3t``-tagged wheels for free-threaded builds wherever they currently allow ``abi3``-tagged ones for (otherwise equal) non-free-threaded builds. @@ -526,25 +510,29 @@ Stable ABI (``abi3``): - defining ``Py_GIL_DISABLED`` (on Windows) - building with free-threaded CPython headers (elsewhere) - Such extensions should be tagged with the `compressed tag set`_ - ``abi3.abi3t``. + The filename of the resulting extension should not have a :pep:`3149` + tag: it should be named :samp:`{modulename}.so`, :samp:`{modulename}.pyd` + or similar. + + Wheels with such extensions should be tagged with the + `compressed tag set`_ ``abi3.abi3t``. - Compile extensions compatible with *only* ``abi3t``, by defining only :samp:`Py_TARGET_ABI3T={v}` and tagging the result with ``abi3t``. This will initially offer no advantages over the ``abi3.abi3t`` option above, but there is a possibility that it will become useful in the future. + On systems that use the :pep:`3149` tag ``abi3`` tag (e.g. on Linux, macOS, + and similar, which use :samp:`{modulename}.abi3.so`), + such extensions should use the tag ``abi3t`` (that is, the extension should + be named :samp:`{modulename}.abi3t.so` on those systems). + + Wheels with such extensions should be tagged with the ABI tag ``abi3t``. + In the above, :samp:`{v}` stands for a the lowest Python version with which the extension should be compatible, in :c:func:`Py_PACK_VERSION` format. In the cases above, this version must be set to 3.15 or higher. -In both cases, if a resulting extension's *filename* would use the ``abi3`` -tag (that is, :samp:`{modulename}.abi3.so` on Linux, macOS, and similar), -the ``abi3t`` tag should be used instead (that is, -:samp:`{modulename}.abi3t.so` on those systems). -If this tag is not used, there should be no change. (This is always the case -on Windows, where the filename should remain :samp:`{name}.pyd`.) - .. _compressed tag set: https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#compressed-tag-sets The version of the Stable ABI, both ``abi3`` and ``abi3t``, is indicated by @@ -685,7 +673,7 @@ free-threaded one. (*): Wheels with these tags cannot be built; see table below The following table summarizes which wheel tag should be used for an extension -built with a given interpreter and ``Py_LIMITED_API`` macro: +built with the given interpreter and defined macros: +-----------------------+-------------+--------------------+---------------------+-----------+ | To get the wheel tag… | Compile on… | ``Py_LIMITED_API`` | ``Py_TARGET_ABI3T`` | Note | @@ -761,7 +749,8 @@ This PEP combines several pieces, implemented individually: `python/cpython#137212 `__. - A check for older ``abi3`` was implemented in GitHub pull request `python/cpython#137957 `__. -- For wheel tags, there is no implementation yet. +- For wheel tags, a draft pull request is at + `pypa/packaging/pull#1099 `__. - A porting guide is not yet written. @@ -836,6 +825,24 @@ better as a transitional state before larger changes like :pep:`809`'s ``abi2026``. +``abi3+abi3t`` filename tag +--------------------------- + +Extensions that are compatible with both ``abi3`` and ``abi3t`` can +use a "compressed tag set" (``abi3.abi3t``) in wheel metadata, +but not in filenames (``.abi3.so``/``.abi3t.so``). + +We could add a dedicated tag for the combination -- for example, +``.abi3+abi3t.so`` -- but this would have no practical benefits over +omitting the tag altogether -- that is, using bare ``.so``. +Specifically, bare ``.so`` meets the needs of :pep:`3149` (which introduced +the filename tags): existing stable ABI extensions (``.abi3.so``) +and version-specific ABI extensions (``.cpNN-*.so``) can be installed +alongside the ``.so`` file, and will be preferred by interpreters that can +load them. + + + Reusing ``Py_GIL_DISABLED`` to enable the new ABI ------------------------------------------------- @@ -859,6 +866,28 @@ the ability to build for the GIL-only Stable ABI with no source changes was deemed to be worth an extra configuration macro (``Py_TARGET_ABI3T``). +Making ``abi3t`` compatible with GIL-enabled builds +--------------------------------------------------- + +Initially, ``abi3t`` will be compatible with ``abi3``. +It would be possible to promise that it will always be so. +This would make the ``abi3.abi3t`` tag unnecessary; +``abi3t`` could be used instead. + +In this case, installers would need to allow ``abi3t``-tagged wheels +for GIL-enabled builds of Python, needing a change in installers. +Since installers written in Python typically use/vendor the :pypi:`packaging` +library, this change would mostly be limited to one repository. + +The wheel tag ``abi3.abi3t`` would be valid in this scenario; +it could be used to support existing installers (which "know about" ``abi3`` +but not ``abi3t``), to experiment with supporting Python 3.14, and in similar +cases. + +However, this would prevent the possibility of ever making ``abi3`` +and ``abi3t`` diverge. + + Copyright ========= From 4c5d2f846cade030c6099151c571381105605178 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Mon, 23 Mar 2026 14:22:34 +0100 Subject: [PATCH 3/5] Partial revert --- peps/pep-0803.rst | 90 +++++++++++++++++++++-------------------------- 1 file changed, 40 insertions(+), 50 deletions(-) diff --git a/peps/pep-0803.rst b/peps/pep-0803.rst index 9e823ccf4cc..0533e90cb24 100644 --- a/peps/pep-0803.rst +++ b/peps/pep-0803.rst @@ -288,6 +288,20 @@ The tag ``abi3t`` is chosen to reflect the fact that this ABI is similar to uses the letter ``t`` in existing, version-specific ABI tags like ``cp314t``). +Filename tag +------------ + +On systems that use the ``abi3`` tag in filenames, a new filename tag +(``abi3t``) is added so that older stable ABI extensions +(:samp:`{name}.abi3.so`) can be installed in the same directory as ones that +support Stable ABI for free-threaded Python (:samp:`{name}.abi3t.so`). + +There can only be one ABI tag in a filename (there is no concept of "compressed +tag sets like in wheel tags), so extensions that are compatible with both ABIs +at once need to use *one* of the tags -- the new one (``abi3t``), which +is currently unused. + + Specification ============= @@ -470,27 +484,29 @@ should use a new `ABI tag`_: ``abi3t``. .. _ABI tag: https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#abi-tag On systems where filenames of Stable ABI extensions end with ``.abi3.so``, -extensions compatible with ``abi3t`` should instead use ``.abi3t.so``. -Extensions compatible with *both* ``abi3`` and ``abi3t`` should not use -a tag; they sould end in ``.so`` only. - -On these systems, free-threaded builds of CPython will load extensions -with the ``abi3t`` tag. - -For example, for ``x86_64-linux-gnu`` builds of CPython, -``importlib.machinery.EXTENSION_SUFFIXES`` will be: - -* for ``python3.15``: - ``['.cpython-315-x86_64-linux-gnu.so', '.abi3.so', '.so']`` -* for ``python3.15t``: +extensions that support free-threading should instead use ``abi3t.so``. +This includes extensions compatible with *both* ``abi3`` and ``abi3t``. + +On these systems, all builds of CPython -- GIL-enabled and free-threaded -- +will load extensions with the ``abi3t`` tag. +Free-threaded builds will -- unlike in 3.14 -- *not* load extensions +with the ``abi3`` tag. +If files are present with both tags, GIL-enabled builds will prefer +"their" ``*.abi3.so`` over ``*.abi3t.so``. + +Put another way, ``importlib.machinery.EXTENSION_SUFFIXES`` will be +(for ``x86_64-linux-gnu`` builds of CPython): + +* ``python3.15``: + ``['.cpython-315-x86_64-linux-gnu.so', '.abi3.so', '.abi3t.so', '.so']`` +* ``python3.15t``: ``['.cpython-315-x86_64-linux-gnu.so', '.abi3t.so', '.so']`` Recommendations for installers ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Package installers should treat the ``abi3t`` tag as completely separate -from ``abi3``. +Package installers should treat this tag as completely separate from ``abi3``. They should allow ``abi3t``-tagged wheels for free-threaded builds wherever they currently allow ``abi3``-tagged ones for (otherwise equal) non-free-threaded builds. @@ -510,29 +526,25 @@ Stable ABI (``abi3``): - defining ``Py_GIL_DISABLED`` (on Windows) - building with free-threaded CPython headers (elsewhere) - The filename of the resulting extension should not have a :pep:`3149` - tag: it should be named :samp:`{modulename}.so`, :samp:`{modulename}.pyd` - or similar. - - Wheels with such extensions should be tagged with the - `compressed tag set`_ ``abi3.abi3t``. + Such extensions should be tagged with the `compressed tag set`_ + ``abi3.abi3t``. - Compile extensions compatible with *only* ``abi3t``, by defining only :samp:`Py_TARGET_ABI3T={v}` and tagging the result with ``abi3t``. This will initially offer no advantages over the ``abi3.abi3t`` option above, but there is a possibility that it will become useful in the future. - On systems that use the :pep:`3149` tag ``abi3`` tag (e.g. on Linux, macOS, - and similar, which use :samp:`{modulename}.abi3.so`), - such extensions should use the tag ``abi3t`` (that is, the extension should - be named :samp:`{modulename}.abi3t.so` on those systems). - - Wheels with such extensions should be tagged with the ABI tag ``abi3t``. - In the above, :samp:`{v}` stands for a the lowest Python version with which the extension should be compatible, in :c:func:`Py_PACK_VERSION` format. In the cases above, this version must be set to 3.15 or higher. +In both cases, if a resulting extension's *filename* would use the ``abi3`` +tag (that is, :samp:`{modulename}.abi3.so` on Linux, macOS, and similar), +the ``abi3t`` tag should be used instead (that is, +:samp:`{modulename}.abi3t.so` on those systems). +If this tag is not used, there should be no change. (This is always the case +on Windows, where the filename should remain :samp:`{name}.pyd`.) + .. _compressed tag set: https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#compressed-tag-sets The version of the Stable ABI, both ``abi3`` and ``abi3t``, is indicated by @@ -866,28 +878,6 @@ the ability to build for the GIL-only Stable ABI with no source changes was deemed to be worth an extra configuration macro (``Py_TARGET_ABI3T``). -Making ``abi3t`` compatible with GIL-enabled builds ---------------------------------------------------- - -Initially, ``abi3t`` will be compatible with ``abi3``. -It would be possible to promise that it will always be so. -This would make the ``abi3.abi3t`` tag unnecessary; -``abi3t`` could be used instead. - -In this case, installers would need to allow ``abi3t``-tagged wheels -for GIL-enabled builds of Python, needing a change in installers. -Since installers written in Python typically use/vendor the :pypi:`packaging` -library, this change would mostly be limited to one repository. - -The wheel tag ``abi3.abi3t`` would be valid in this scenario; -it could be used to support existing installers (which "know about" ``abi3`` -but not ``abi3t``), to experiment with supporting Python 3.14, and in similar -cases. - -However, this would prevent the possibility of ever making ``abi3`` -and ``abi3t`` diverge. - - Copyright ========= From 64f7ec499ed63f61bc949f1d600eea3278dce055 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Mon, 23 Mar 2026 17:43:36 +0100 Subject: [PATCH 4/5] Rewordings --- peps/pep-0803.rst | 239 +++++++++++++++++++++++++++++----------------- 1 file changed, 152 insertions(+), 87 deletions(-) diff --git a/peps/pep-0803.rst b/peps/pep-0803.rst index 0533e90cb24..f958b0542c7 100644 --- a/peps/pep-0803.rst +++ b/peps/pep-0803.rst @@ -23,14 +23,15 @@ Python” (or ``abi3t`` for short). This will require users to migrate to new API for common tasks like defining modules and most classes. -At least initially, extensions built for ``abi3t`` will be compatible with -the existing Stable ABI (``abi3``). +Extensions built for ``abi3t`` will be compatible with ``abi3`` 3.15+, +and will signal this explicitly using the wheel tag +``abi3.abi3t``. Terminology =========== -This PEP uses “GIL-enabled build” as an antonym to “free-threaded build”, +This PEP uses "GIL-enabled build" as an antonym to “free-threaded build”, that is, an interpreter or extension built without ``Py_GIL_DISABLED``. @@ -38,7 +39,7 @@ Motivation ========== The Stable ABI is currently not available for free-threaded builds. -Extensions will fail to build for a both Limited API and +Extensions will fail to build for the Stable ABI on free-threaded Python (that is, when both :c:macro:`Py_LIMITED_API` and :c:macro:`Py_GIL_DISABLED` preprocessor macros are defined). Extensions built for GIL-enabled builds of CPython will fail to load @@ -66,19 +67,18 @@ With free-threading builds (:pep:`703`) being on track to eventually become the default (:pep:`779`), we need a way to make a Stable ABI available to those builds. -To build against a Stable ABI, the extension must use a *Limited API*, -that is, only a subset of the functions, structures, etc. that CPython -exposes. -Both the Limited API and the current Stable ABI are versioned, and building -against Stable ABI 3.X requires using only Limited API 3.X, and yields an -extension that is ABI-compatible with CPython 3.X and *any* later version +The current Stable ABI is versioned, and an extension built for Stable ABI +3.X is ABI-compatible with CPython 3.X and *any* later version (though bugs in CPython sometimes cause incompatibilities in practice). -The Limited API is not “stable”: newer versions may remove API that -were a part of older versions. +However, this forward compatibility is only guaranteed for +a subset of the API that CPython exposes (functions, structures, etc.). +Extensions that target the Stable ABI must limit themselves to this subset, +called the :ref:`Limited API `. -This PEP proposes additional API limitations, as required to be compatible -with both GIL-enabled and free-threaded builds of CPython. +This PEP proposes additional API limitations needed to compile extensions +compatible with both GIL-enabled and free-threaded builds of CPython 3.15+, +and naming/tagging schemes extensions can use to signal such compatibility. Ecosystem maintainers want decreased maintenance burden @@ -246,6 +246,17 @@ maintainer `David Woods said disaster since we're not getting rid of the regular compilation mode so people are free to pick their own personal trade-offs. +It should be noted that this PEP leaves some questions/work open. +Wenzel Jakob -- maintainer of ``nanobind``, a C++ binding generator -- +`noted `_ +a need for additional API that is left out of scope of *this* PEP: + + It's not clear to me how I can get a pointer to the *N*-th data entry + of [a ``PyVarObject``-derived object]. + + If that can be resolved then nanobind should be able to adopt this new + ``abi3t`` compilation target. + Rationale ========= @@ -258,8 +269,8 @@ Separate ABI However, it will be possible to compile a single extension module that supports both free-threaded and GIL-enabled builds. - This should involve no additional limitations, at least in the initial - implementation, and so it will be preferred over building + This should involve no additional limitations in the foreseeable future, + and so it will be preferred over building ``abi3t``-only extensions. No backwards compatibility now @@ -279,6 +290,8 @@ API changes are OK API, and compile separately for GIL-enabled builds and for specific versions of free-threaded builds. +See the :ref:`Rejected Ideas section ` for several +ideas that don't meet these constraints. Tag name -------- @@ -316,18 +329,17 @@ using major (3) and minor versions of Python interpreter. Extensions built for ``abi3t`` :samp:`3.{x}` will be compatible with CPython :samp:`3.{x}` and above. -To build a C/C++ extension for ``abi3t``, the extension will need to only -use *limited API for free-threaded builds*. -Like the existing Limited API, this will be a subset of the general CPython -C API. -Initially, it will also be a subset of the Limited API. -We will strive to keep it as a subset, but will not guarantee this. - -Since ``abi3t`` and ``abi3`` will overlap, it will be possible for a single -compiled extension to support both at once, and thus be compatible with -CPython 3.15+ (both free-threaded and GIL-enabled builds). -Initially, any extension compiled for ``abi3t`` will be compatible with +To build a C/C++ extension for ``abi3t``, the extension will need to limit +itself to only use API for which we can promise long-term support. +This *limited API for free-threaded builds* will be a subset of the +3.15 Limited API. + +Any extension compiled for ``abi3t`` will, in practice, be compatible with ``abi3`` as well. +However, we recommend that users and tool authors focus explicitly +signal supporting supporting *both at once* (that is, their "intersection") +rather than only ``abi3t``. +(See `a Rejected Idea <803-subset>`_ for rationale.) Choosing the target ABI @@ -448,12 +460,13 @@ models are possible. .. _platform compatibility tags: https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/ However, CPython will add a line of defense against outdated or misconfigured -tools, or human mistakes, in the form of a new *module slot* containing -basic ABI information. +tools, or human mistakes, in the form of a new *module slot*, ``Py_mod_abi``, +containing basic ABI information. This information will be checked when a module is loaded, and incompatible extensions will be rejected. The specifics are left to the C API working group -(see `issue 72 `__). +(see `issue 72 `__); +. This slot will become *mandatory* with the new export hook added in :pep:`793`. @@ -478,7 +491,7 @@ if the internal object layout needs to change. The ``abi3t`` wheel and filename tags ------------------------------------- -Wheels that use a stable ABI compatible with free-threading CPython builds +Wheels with extension modules compiled for Stable ABI for Free-Threaded Python should use a new `ABI tag`_: ``abi3t``. .. _ABI tag: https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#abi-tag @@ -502,37 +515,42 @@ Put another way, ``importlib.machinery.EXTENSION_SUFFIXES`` will be * ``python3.15t``: ``['.cpython-315-x86_64-linux-gnu.so', '.abi3t.so', '.so']`` +Making GIL-enabled builds load ``.abi3t.so`` files is purely a practical +choice: it this one case we break the conceptual purity of ``abi3`` and +``abi3t`` being separate ABIs. + Recommendations for installers ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Package installers should treat this tag as completely separate from ``abi3``. +Package installers should treat ``abi3t`` as completely separate from ``abi3``. They should allow ``abi3t``-tagged wheels for free-threaded builds wherever they currently allow ``abi3``-tagged ones for (otherwise equal) non-free-threaded builds. +Installers must handle the compressed ABI tag set ``abi3.abi3t`` as per +`PyPA specifications `_. + +Note that this PEP does not provide a way to target Stable ABI for +Free-threaded Python 3.14 (``cp314-abi3t``) and below. +This may change an the future, or with experimental build tools, so +*installers* should be prepared for such extensions. + + Recommendations for build tools ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Build tools should give users one or two additional options, in addition to +Build tools should give users an additional options, in addition to the existing CPython version-specific ABI (:samp:`cp3{nn}`) and Stable ABI (``abi3``): +Compile extensions that are compatible with *both* +``abi3`` and ``abi3t``, by either: -- Compile extensions compatible with *both* ``abi3`` and ``abi3t``, by either: - - - defining both :samp:`Py_LIMITED_API={v}` and :samp:`Py_TARGET_ABI3T={v}`, or - - defining :samp:`Py_LIMITED_API={v}` and: +- defining both :samp:`Py_LIMITED_API={v}` and :samp:`Py_TARGET_ABI3T={v}`, or +- defining :samp:`Py_LIMITED_API={v}` and: - - defining ``Py_GIL_DISABLED`` (on Windows) - - building with free-threaded CPython headers (elsewhere) - - Such extensions should be tagged with the `compressed tag set`_ - ``abi3.abi3t``. - -- Compile extensions compatible with *only* ``abi3t``, by defining only - :samp:`Py_TARGET_ABI3T={v}` and tagging the result with ``abi3t``. - This will initially offer no advantages over the ``abi3.abi3t`` option - above, but there is a possibility that it will become useful in the future. + - defining ``Py_GIL_DISABLED`` (on Windows) + - building with free-threaded CPython headers (elsewhere) In the above, :samp:`{v}` stands for a the lowest Python version with which the extension should be compatible, in :c:func:`Py_PACK_VERSION` format. @@ -545,20 +563,23 @@ the ``abi3t`` tag should be used instead (that is, If this tag is not used, there should be no change. (This is always the case on Windows, where the filename should remain :samp:`{name}.pyd`.) -.. _compressed tag set: https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#compressed-tag-sets +Wheels containing such extensions should be tagged with the +`compressed ABI tag set `_ ``abi3.abi3t``, and +the `Python tag`_ :samp:`cp{3yy}` corresponding to :samp:`{v}` above. -The version of the Stable ABI, both ``abi3`` and ``abi3t``, is indicated by -the `Python wheel tag`_. For example, a wheel tagged ``cp315-abi3.abi3t`` will be compatible with 3.15, 3.16, and later versions; ``cp317-abi3.abi3t`` will be compatible with 3.17+. -.. _Python wheel tag: https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#python-tag +.. _compressed tag set: https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#compressed-tag-sets -Note that this PEP does not provide a way to target Stable ABI for -Free-threaded Python 3.14 and below (``cp314-abi3t``). -This may change an the future, or with experimental build tools, so -*installers* should be prepared for such extensions. +.. _Python tag: https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#python-tag + +Alternately, or in addition, tools may allow compiling extensions compatible +with *only* ``abi3t``, by defining only :samp:`Py_TARGET_ABI3T={v}` and tagging +the resulting wheel with ``abi3t`` rather than ``abi3.abi3t``. +This will artificially limit the result to free-threaded interpreters only, +without many advantages. New API @@ -568,8 +589,8 @@ Implementing this PEP will make it possible to build extensions that can be successfully loaded on free-threaded Python, but not necessarily ones that are thread-safe without a GIL. -Limited API to allow thread-safety without a GIL -- presumably ``PyMutex``, -``PyCriticalSection``, and similar -- will be added via the C API working group, +Limited API to allow thread-safety without a GIL -- presumably +``PyCriticalSection`` and similar -- will be added via the C API working group, or in a follow-up PEP. @@ -584,10 +605,11 @@ Extension authors who cannot switch may continue to use the existing ``abi3``, that is, build on GIL-enabled Python without defining ``Py_GIL_DISABLED``. For compatibility with free-threaded builds, they can compile using version-specific ABI -- that is, compile ``abi3``-compatible source -on free-threaded CPython builds without defining ``Py_LIMITED_API``. +on free-threaded CPython builds without defining ``Py_TARGET_ABI3T``. -Limited API 3.15 for free-threading is a subset of the existing Limited API, -and as such, it will be forward-compatible with future versions of CPython 3.x. +Limited API 3.15 for free-threading is a subset of the "``abi3``" Limited +API 3.15, and as such, it will be forward-compatible with future versions +of CPython 3.x. Older versions of the Limited API (that is, 3.14 and below) will continue to be forward-compatible with GIL-enabled builds of CPython 3.x, starting with the version that introduced the given Limited API. @@ -718,6 +740,7 @@ In the “Compile on” column, *FT* means that the :c:macro:`Py_GIL_DISABLED` macro must be defined -- either explicitly or, on non-Windows platforms, by including CPython headers configured with :option:`--disable-gil`. *GIL* means that :c:macro:`Py_GIL_DISABLED` must *not* be defined. +Rows without this note apply to both cases. In the ``Py_LIMITED_API`` and ``Py_TARGET_ABI3T``, a dash means the macro must not be defined; a version means the macro must be set to the corresponding @@ -766,26 +789,31 @@ This PEP combines several pieces, implemented individually: - A porting guide is not yet written. +.. _pep803-rejected-ideas: + Rejected Ideas ============== -Make ``PyObject`` opaque in Limited API 3.15 +Make ``PyObject`` opaque in Stable ABI 3.15 -------------------------------------------- -It would be possible to make ``PyObject`` struct opaque in Limited API 3.15 -(that is, the new version of the existing Limited API), -rather than introduce a new variant of the Stable ABI and Limited API. +It would be possible to make ``PyObject`` struct opaque in Stable ABI +rather than introduce a new variant of the Stable ABI. This would mean that extension authors would need to adapt their code to the -new limitations, or abandon Limited API altogether, in order to use any C API +new limitations, or abandon Stable ABI altogether, in order to use any C API introduced in Python 3.15. -It would also not remove the need for a new wheel tag (``abi3t``), +It would also not fully remove the need for a new wheel tag (``abi3t``), which would be required to express that an extension is compatible with both GIL-enabled and free-threaded builds of CPython 3.14 or lower. +In the `PEP discussion `__, +the ability to build for the GIL-only Stable ABI with no source changes +was deemed to be worth an extra configuration macro (``Py_TARGET_ABI3T``). + .. _pep803-no-shim: @@ -819,6 +847,37 @@ extension to query the running interpreter, and for 3.14, use a ``struct`` definition corresponding to the detected build's ``PyModuleDef``. +.. _803-subset: + +Making ``abi3t`` compatible with ``abi3`` +----------------------------------------- + +It would be possible to teach all packaging tools that ``abi3t`` is a subset +of ``abi3``, and thus GIL-enabled interpreters are guaranteed to be compatible +with ``abi3t``-tagged builds. + +This would make the ``abi3.abi3t`` compressed tag set fully equivalent to +``abi3t``, and thus redundant. +However, tools would still need to output the compressed tag set, to support +"older installers", which will *not* install ``abi3t`` for GIL-enabled builds. + +Here, "older installers" include ones that use or vendor an existing version +of the :pypi:`packaging` package, which Python-based installers typically use +to implement wheel tag matching. +This project provides a *ordered set* of matching tags for a given platforms +(`packaging.tags.sys_tags `__). + +Merging ``abi3`` into ``abi3t`` would mean that for free-threaded builds, +```packaging.tags.sys_tags`` would include all tags with ``abi3t`` *and* +all tags with ``abi3t``, essentially doubling the size of the set. +This is undesirable. + +Less importantly, merging the ABIs would also remove an "escape hatch" of +possibly making ``abi3`` and ``abi3t`` diverge in the future. +(Such divergence could only affect a few niche APIs, which can be only added +to ``abi3`` or ``abi3t``, but excluded from the intersection.) + + Naming this ``abi4`` -------------------- @@ -840,19 +899,38 @@ better as a transitional state before larger changes like :pep:`809`'s ``abi3+abi3t`` filename tag --------------------------- +Filename ABI tags (as introduced in :pep:`3149`) allow extensions for several +ABIs to co-exist in a directory. + Extensions that are compatible with both ``abi3`` and ``abi3t`` can use a "compressed tag set" (``abi3.abi3t``) in wheel metadata, but not in filenames (``.abi3.so``/``.abi3t.so``). +We *could* add a dedicated tag for the combination -- for example, +``.abi3+abi3t.so``. -We could add a dedicated tag for the combination -- for example, -``.abi3+abi3t.so`` -- but this would have no practical benefits over -omitting the tag altogether -- that is, using bare ``.so``. -Specifically, bare ``.so`` meets the needs of :pep:`3149` (which introduced -the filename tags): existing stable ABI extensions (``.abi3.so``) -and version-specific ABI extensions (``.cpNN-*.so``) can be installed -alongside the ``.so`` file, and will be preferred by interpreters that can -load them. +There would be no need for ``.abi3+abi3t.so`` extensions to co-exist with +``.abi3t.so`` ones, so we opt to stay away from the path toward +`compressed tag sets `_ in filenames, +and make the practical choice to make ``.abi3t.so`` mean "abi3+abi3t", +that is "loadable by all builds". +Conceptually, filename tags do not "describe" or "name" an extension's ABI. +(The current ``.abi3`` tag is too weak for this, as it lacks a version.) + + +No filename tag (bare ``.so``) +------------------------------ + +It would be possible to drop the filename ABI tag altogether, +and using ``.so`` instead of ``.abi3t.so``. +Their behavior are very close. +``.so`` is loadable by *any* build of CPython. +``.abi3t.so`` will be loadable by any CPython *3.15 or above* -- but the +Stable ABI filename tag already lacks version information. + +They are different semantically, though. +Bare ``.so`` means "don't care"; an ``.abi3t.so`` extension is *intentionally* +compatible with the new ABI. Reusing ``Py_GIL_DISABLED`` to enable the new ABI @@ -865,19 +943,6 @@ This would require annoying fiddling with build flags, and make it especially cumbersome to target *both* ``abi3`` and ``abi3t`` at the same time. -Making the new version of ``abi3`` compatible with free-threading ------------------------------------------------------------------ - -It would be possible to make ``PyObject`` opaque in Limited API 3.15, -rather than add a new stable ABI. -This would make all extensions built for the Stable ABI 3.15 and above -compatible with both free-threading and GIL-enabled Python. - -In the `PEP discussion `__, -the ability to build for the GIL-only Stable ABI with no source changes -was deemed to be worth an extra configuration macro (``Py_TARGET_ABI3T``). - - Copyright ========= From 99ccd426205469a2238fd56a00c92b43fc9114d8 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Mon, 23 Mar 2026 17:46:57 +0100 Subject: [PATCH 5/5] More --- peps/pep-0803.rst | 366 +++++++++++++++++++++++++++++++--------------- 1 file changed, 247 insertions(+), 119 deletions(-) diff --git a/peps/pep-0803.rst b/peps/pep-0803.rst index f958b0542c7..7b331e3f415 100644 --- a/peps/pep-0803.rst +++ b/peps/pep-0803.rst @@ -23,15 +23,15 @@ Python” (or ``abi3t`` for short). This will require users to migrate to new API for common tasks like defining modules and most classes. -Extensions built for ``abi3t`` will be compatible with ``abi3`` 3.15+, -and will signal this explicitly using the wheel tag -``abi3.abi3t``. +In practice, ``abi3t`` 3.15 will be compatible with ``abi3`` 3.15. +Extension authors are encouraged to explicitly compile for both ABIs at once, +and signal compatibility using the wheel tag ``abi3.abi3t``. Terminology =========== -This PEP uses "GIL-enabled build" as an antonym to “free-threaded build”, +This PEP uses "GIL-enabled build" as an antonym to "free-threaded build", that is, an interpreter or extension built without ``Py_GIL_DISABLED``. @@ -52,8 +52,8 @@ for free-threading should be prepared and defined for Python 3.15”. This PEP proposes the Stable ABI for free-threading. -Background ----------- +Background & Summary +-------------------- Python's Stable ABI (``abi3`` for short), as defined in :pep:`384` and :pep:`652`, provides a way to compile extension modules that can be loaded @@ -75,10 +75,15 @@ However, this forward compatibility is only guaranteed for a subset of the API that CPython exposes (functions, structures, etc.). Extensions that target the Stable ABI must limit themselves to this subset, called the :ref:`Limited API `. +When the "opt-in" preprocessor macro ``Py_LIMITED_API`` is defined, +CPython headers will expose the Limited API only -This PEP proposes additional API limitations needed to compile extensions -compatible with both GIL-enabled and free-threaded builds of CPython 3.15+, -and naming/tagging schemes extensions can use to signal such compatibility. +This PEP proposes a *stable ABI for free-threading builds* +(``abi3t`` for short), which includes additional API limitations needed to +compile extensions compatible with both GIL-enabled and free-threaded builds +of CPython 3.15+, +a corresponding macro to opt in to these limitations, and naming/tagging +schemes that extensions should use to signal such compatibility. Ecosystem maintainers want decreased maintenance burden @@ -99,7 +104,7 @@ The cryptography project shipped 48 wheel files with their `most recent release somewhat unusual in that they ship 14 wheels each for both the ``cp38`` and ``cp311`` stable ABIs to enable optimizations available in newer limited API versions. They also ship 14 additional ``cp314t`` wheels and 6 wheels for -pypy. If there is no free-threaded stable ABI, then with Python 3.15, +PyPy. If there is no free-threaded stable ABI, then with Python 3.15, cryptography will be using roughly the same amount of space on PyPI to support two versions of the free-threaded build as *all* non-EOL versions of the GIL-enabled build. @@ -261,37 +266,74 @@ a need for additional API that is left out of scope of *this* PEP: Rationale ========= -The design in this PEP uses several constraints: +The design of ``abi3t`` involves several choices/assumptions/constraints: Separate ABI - The new ABI (``abi3t``) will conceptually be treated as separate from the - existing Stable ABI (``abi3``). +------------ + +The new ABI (``abi3t``) will conceptually be treated as separate from the +existing Stable ABI (``abi3``) – even though all extensions compatible with +``abi3t`` will, in practice, *also* be compatible with ``abi3``. + +(In more precise wording: ``abi3t``'s set of allowed APIs will be a +*subset* of ``abi3``'s; ``abi3t``'s set of compatible interpreters will +be a *superset* of ``abi3``'s. That makes one's head spin, which is part +of the reason to keep them separate.) + +Extensions should be explicitly compiled for *both* ``abi3t`` and ``abi3``, +and should explicitly signal that they support both at once +-- via packaging wheel tags (``abi3.abi3t``) and a runtime ABI check +(:external+py3.15:c:macro:`Py_mod_abi`). + +This explicitness has several advantages over having ``abi3t`` support +*imply* ``abi3`` support: - However, it will be possible to compile a single extension module that - supports both free-threaded and GIL-enabled builds. - This should involve no additional limitations in the foreseeable future, - and so it will be preferred over building - ``abi3t``-only extensions. +- The tags clearly show whether an extension is compatible with GIL-enabled + builds, and whether the existing backwards compatibility guarantees of + ``abi3`` apply. +- Implementation-wise, the set of tags a given free-threaded interpreter + supports (as returned from the :pypi:`packaging` function + ``packaging.tags.sys_tags``) It will be the same size as for a corresponding + GIL-enabled build. +- It allows the ABIs to diverge slightly in the future -- keeping + *both at once* as the preferred compilation target, but allowing + ``abi3t``-only extensions for special cases. + +One practical exception to keeping the ABIs conceptually separate is discussed +in the :ref:`803-filename-tag` section. + +See these Rejected Ideas sections for more on the alternatives: + +- :ref:`803-subset` +- :ref:`803-single` No backwards compatibility now - The new stable ABI variant will not support CPython 3.14 and below. - Projects that need this support can build separate extensions specifically - for the 3.14 free-threaded interpreter, and for older stable ABI versions. +------------------------------ + +CPython headers will not allow compiling for ``abi3t`` for CPython 3.14 +and earlier. +Projects that need this can build separate extensions specifically +for the 3.14 free-threaded interpreter, and for older ``abi3``. - However, we won't block the possibility of extending compatibility to - CPython 3.14 and below, and we recommend that package installation tools - prepare for such extensions. - See a :ref:`rejected idea ` for how this could work. +However, it *is* technically possible to build an extension compatible +with both free-threaded and GIL-enabled builds of CPython 3.14+. +To enable experiments in this area, we recommend that package installation +tools are prepared for such extensions. +See a :ref:`rejected idea ` for more details. -API changes are OK - The new Limited API variant may require extension authors to make - significant changes to their code. - Projects that cannot do this (yet) can continue using the existing Limited - API, and compile separately for GIL-enabled builds - and for specific versions of free-threaded builds. +Source changes are necessary in extensions +------------------------------------------ -See the :ref:`Rejected Ideas section ` for several -ideas that don't meet these constraints. +``abi3t`` will require extension authors to make +significant changes to their code. + +Projects that cannot do this (yet) can continue using ``abi3``, +and compile the same source for specific versions of free-threaded builds. +(Note that the APIs removed in ``api3t`` still are usable when compiling for +a specific version, including 3.15t.) + +See a Rejected Ideas sections for an alternative: +:ref:`803-freeze-pyobject` Tag name -------- @@ -300,6 +342,10 @@ The tag ``abi3t`` is chosen to reflect the fact that this ABI is similar to ``abi3``, with minimal changes necessary to support free-threading (which uses the letter ``t`` in existing, version-specific ABI tags like ``cp314t``). +See a Rejected Ideas sections for an alternative: +:ref:`803-abi4` + +.. _803-filename-tag: Filename tag ------------ @@ -311,8 +357,50 @@ support Stable ABI for free-threaded Python (:samp:`{name}.abi3t.so`). There can only be one ABI tag in a filename (there is no concept of "compressed tag sets like in wheel tags), so extensions that are compatible with both ABIs -at once need to use *one* of the tags -- the new one (``abi3t``), which -is currently unused. +at once need to use *one* of the tags -- the new one (``abi3t``), as the +existing one has existing meaning. + +See Rejected Ideas sections for alternatives: + +- :ref:`803-combined-filename-tag` +- :ref:`803-bare-so` + +Knob name +--------- + +This PEP specifies that the C preprocessor macro ``Py_TARGET_ABI3T`` +will enable compiling for ``abi3t`` (that is, practically: it will make +``Python.h`` only expose forward-compatible definitions). + +The corresponding "knob" for ``abi3`` is named ``Py_LIMITED_API``. +This name is problematic: + +- It describes what the macro's historical internal effect (limiting which + definitions are exposed), but not the intended benefit (forward + compatibility). +- It is increasingly a misnomer: for API like ``Py_TYPE``, it selects + a forward-compatible implementation (DLL function call rather than inline + pointer deference) rather than limiting the API. +- The pair of terms *Stable ABI* and *Limited API* is technically accurate, + but quite confusing. + Avoiding the term *Limited API*, and talking about "constraints necessary + for targeting a given ABI", tends to be clearer. + +The proposed macro name (``Py_TARGET_ABI3T``) emphasizes ``abi3t`` as a +*compilation target*, with API limitations as its implicit price -- and forward +compatibility as the implicit benefit. + +As for ``Py_LIMITED_API``, this PEP proposes no change, which means keeping +it for ``abi3``. +``abi3`` is expected to eventually become irrelevant *if* free-threaded builds +replace the GIL-enabled ones (see the +`PEP 703 acceptance notice `__ +for the tentative plan). +At that point, ``Py_LIMITED_API`` will likely remain user-visible, but as an +implementation detail. + +See a Rejected Ideas sections for an alternative -- reusing an existing "knob": +:ref:`803-knob-gildisabled` Specification @@ -327,7 +415,13 @@ builds*, or ``abi3t`` for short. As with the current Stable ABI (``abi3``), ``abi3t`` will be versioned using major (3) and minor versions of Python interpreter. Extensions built for ``abi3t`` :samp:`3.{x}` will be compatible with -CPython :samp:`3.{x}` and above. +free-threading builds of CPython :samp:`3.{x}` and above. + +This mirrors the compatibility promise for the existing Stable ABI, ``abi3``, +which was defined :pep:`384#abstract` and modified in +:pep:`703#backwards-compatibility`: +Extensions built for ``abi3`` :samp:`3.{x}` will be compatible +with *GIL-enabled* builds of CPython :samp:`3.{x}` and above. To build a C/C++ extension for ``abi3t``, the extension will need to limit itself to only use API for which we can promise long-term support. @@ -336,10 +430,10 @@ This *limited API for free-threaded builds* will be a subset of the Any extension compiled for ``abi3t`` will, in practice, be compatible with ``abi3`` as well. -However, we recommend that users and tool authors focus explicitly -signal supporting supporting *both at once* (that is, their "intersection") -rather than only ``abi3t``. -(See `a Rejected Idea <803-subset>`_ for rationale.) +However, we recommend that users and tools explicitly compile for +*both at the same time*, and signal this explicitly. +(In the PyPA packaging ecosystem, this signaling means using the wheel tag +``abi3.abi3t`` as detailed below). Choosing the target ABI @@ -353,11 +447,6 @@ tool-specific UI -- will select the target ABI using the following macros: * ``Py_TARGET_ABI3T=`` (proposed here): Compile for ``abi3t`` of the given version. -These two macros are functionally very similar. -In hindsight, ``Py_TARGET_ABI3`` (without the ``T``) would be a more fitting -name for :c:macro:`Py_LIMITED_API`. -We keep the existing name for backwards compatibility. - For ease of use and implementation simplicity, respectively, ``Python.h`` will set the configuration macros automatically in the following situations: @@ -444,6 +533,8 @@ They will, however, not be removed. - :c:func:`PyModule_FromDefAndSpec`, :c:func:`PyModule_FromDefAndSpec2` +.. _803-runtime-check: + Runtime ABI checks ------------------ @@ -465,8 +556,7 @@ containing basic ABI information. This information will be checked when a module is loaded, and incompatible extensions will be rejected. The specifics are left to the C API working group -(see `issue 72 `__); -. +(see `capi-workgroup issue 72 `__). This slot will become *mandatory* with the new export hook added in :pep:`793`. @@ -523,28 +613,23 @@ choice: it this one case we break the conceptual purity of ``abi3`` and Recommendations for installers ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Package installers should treat ``abi3t`` as completely separate from ``abi3``. -They should allow ``abi3t``-tagged wheels for free-threaded builds wherever -they currently allow ``abi3``-tagged ones for (otherwise equal) +Package installers should allow ``abi3t``-tagged wheels for free-threaded +builds wherever they currently allow ``abi3``-tagged ones for (otherwise equal) non-free-threaded builds. -Installers must handle the compressed ABI tag set ``abi3.abi3t`` as per -`PyPA specifications `_. - Note that this PEP does not provide a way to target Stable ABI for Free-threaded Python 3.14 (``cp314-abi3t``) and below. This may change an the future, or with experimental build tools, so -*installers* should be prepared for such extensions. +installers should be prepared for such extensions. Recommendations for build tools ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Build tools should give users an additional options, in addition to +Build tools should give users a new option: in addition to compiling for the existing CPython version-specific ABI (:samp:`cp3{nn}`) and -Stable ABI (``abi3``): -Compile extensions that are compatible with *both* -``abi3`` and ``abi3t``, by either: +Stable ABI (``abi3``), they should allow +compiling extensions for *both* ``abi3`` and ``abi3t`` at once, by either: - defining both :samp:`Py_LIMITED_API={v}` and :samp:`Py_TARGET_ABI3T={v}`, or - defining :samp:`Py_LIMITED_API={v}` and: @@ -552,34 +637,36 @@ Compile extensions that are compatible with *both* - defining ``Py_GIL_DISABLED`` (on Windows) - building with free-threaded CPython headers (elsewhere) -In the above, :samp:`{v}` stands for a the lowest Python version with which +In the above, :samp:`{v}` stands for the lowest Python version with which the extension should be compatible, in :c:func:`Py_PACK_VERSION` format. -In the cases above, this version must be set to 3.15 or higher. +This version must be 3.15 or higher. -In both cases, if a resulting extension's *filename* would use the ``abi3`` -tag (that is, :samp:`{modulename}.abi3.so` on Linux, macOS, and similar), -the ``abi3t`` tag should be used instead (that is, -:samp:`{modulename}.abi3t.so` on those systems). -If this tag is not used, there should be no change. (This is always the case -on Windows, where the filename should remain :samp:`{name}.pyd`.) +On systems that use ABI version tagged ``.so`` files as introduced in +:pep:`3149` (Linux, macOS, and similar), the extension should be named +:samp:`{modulename}.abi3t.so`. +Otherwise, there should be no change: on Windows, use :samp:`{name}.pyd`. Wheels containing such extensions should be tagged with the -`compressed ABI tag set `_ ``abi3.abi3t``, and +`compressed ABI tag set `_ +``abi3.abi3t``, and the `Python tag`_ :samp:`cp{3yy}` corresponding to :samp:`{v}` above. For example, a wheel tagged ``cp315-abi3.abi3t`` will be compatible with 3.15, 3.16, and later versions; ``cp317-abi3.abi3t`` will be compatible with 3.17+. -.. _compressed tag set: https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#compressed-tag-sets - .. _Python tag: https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#python-tag -Alternately, or in addition, tools may allow compiling extensions compatible -with *only* ``abi3t``, by defining only :samp:`Py_TARGET_ABI3T={v}` and tagging -the resulting wheel with ``abi3t`` rather than ``abi3.abi3t``. -This will artificially limit the result to free-threaded interpreters only, -without many advantages. +It is discouraged, but possible, to compile extensions compatible with *only* +``abi3t`` (by defining only :samp:`Py_TARGET_ABI3T={v}`, building with +GIL-enabled CPython, and tagging the resulting wheel with ``abi3t`` rather +than ``abi3.abi3t``). +This will limit the result to free-threaded interpreters only. + +Its is also possible to build ``abi3t`` extensions compatible with CPython 3.14 +(or even lower versions), but this is unsupported and would require +detailed understanding of the limitations and guarantees, +along with thorough testing. New API @@ -597,8 +684,9 @@ or in a follow-up PEP. Backwards and Forwards Compatibility ==================================== -Extensions targetting ``abi3t`` will not be backwards-compatible with older -CPython releases, due to the need to use new ``PyModExport`` API added +Extensions targeting ``abi3t`` will not be backwards-compatible with older +CPython releases, neither at the source level (API) nor in compiled form (ABI), +due to the need to avoid ``PyModuleDef`` and use new ``PyModExport`` hook added in :pep:`793`. Extension authors who cannot switch may continue to use the existing ``abi3``, @@ -607,12 +695,10 @@ For compatibility with free-threaded builds, they can compile using version-specific ABI -- that is, compile ``abi3``-compatible source on free-threaded CPython builds without defining ``Py_TARGET_ABI3T``. -Limited API 3.15 for free-threading is a subset of the "``abi3``" Limited -API 3.15, and as such, it will be forward-compatible with future versions -of CPython 3.x. -Older versions of the Limited API (that is, 3.14 and below) will continue -to be forward-compatible with GIL-enabled builds of CPython 3.x, starting with -the version that introduced the given Limited API. +As for the existing Stable ABI: ``abi3`` :samp:`3.{x}` continues to be +compatible with *GIL-enabled* CPython :samp:`3.{x}` or above, +as promised in :pep:`384#abstract` and amended in +:pep:`703#backwards-compatibility`. Compatibility Overview @@ -755,6 +841,7 @@ Values in the *Note* column: this PEP, but may be added in the future. Installers should be prepared to handle the tag. + Security Implications ===================== @@ -784,8 +871,10 @@ This PEP combines several pieces, implemented individually: `python/cpython#137212 `__. - A check for older ``abi3`` was implemented in GitHub pull request `python/cpython#137957 `__. -- For wheel tags, a draft pull request is at +- For wheel tag handling in installers, a draft pull request is at `pypa/packaging/pull#1099 `__. +- For build tools, several individual draft pull requests are open; + contact Nathan for details. - A porting guide is not yet written. @@ -794,9 +883,10 @@ This PEP combines several pieces, implemented individually: Rejected Ideas ============== +.. _803-single: -Make ``PyObject`` opaque in Stable ABI 3.15 --------------------------------------------- +Single new ABI: make ``PyObject`` opaque in Stable ABI 3.15 +----------------------------------------------------------- It would be possible to make ``PyObject`` struct opaque in Stable ABI rather than introduce a new variant of the Stable ABI. @@ -805,14 +895,15 @@ This would mean that extension authors would need to adapt their code to the new limitations, or abandon Stable ABI altogether, in order to use any C API introduced in Python 3.15. -It would also not fully remove the need for a new wheel tag (``abi3t``), +It would also not fully remove the case for a new wheel tag (``abi3t``), which would be required to express that an extension is compatible with both GIL-enabled and free-threaded builds of CPython 3.14 or lower. In the `PEP discussion `__, the ability to build for the GIL-only Stable ABI with no source changes -was deemed to be worth an extra configuration macro (``Py_TARGET_ABI3T``). +was deemed to be worth an extra configuration macro (now called +``Py_TARGET_ABI3T``). .. _pep803-no-shim: @@ -852,32 +943,33 @@ a ``struct`` definition corresponding to the detected build's ``PyModuleDef``. Making ``abi3t`` compatible with ``abi3`` ----------------------------------------- -It would be possible to teach all packaging tools that ``abi3t`` is a subset -of ``abi3``, and thus GIL-enabled interpreters are guaranteed to be compatible -with ``abi3t``-tagged builds. - -This would make the ``abi3.abi3t`` compressed tag set fully equivalent to +It would be possible to teach packaging tools that ``abi3t`` is a "subset" +of ``abi3``, that is, all GIL-enabled interpreters are guaranteed to be +compatible with ``abi3t``-tagged builds. +This would make the ``abi3.abi3t`` compressed tag set equivalent to ``abi3t``, and thus redundant. -However, tools would still need to output the compressed tag set, to support -"older installers", which will *not* install ``abi3t`` for GIL-enabled builds. - -Here, "older installers" include ones that use or vendor an existing version -of the :pypi:`packaging` package, which Python-based installers typically use -to implement wheel tag matching. -This project provides a *ordered set* of matching tags for a given platforms -(`packaging.tags.sys_tags `__). - -Merging ``abi3`` into ``abi3t`` would mean that for free-threaded builds, -```packaging.tags.sys_tags`` would include all tags with ``abi3t`` *and* -all tags with ``abi3t``, essentially doubling the size of the set. -This is undesirable. +However, tools would still need to output the compressed tag set to support +"older installers", which do *not* consider ``abi3t`` compatible with +GIL-enabled builds. + +Here, "older installers" include ones that use or vendor an version of the +:pypi:`packaging` library that wasn't updated for ``abi3t``. +(The ``packaging`` library is what Python-based installers typically use +to implement wheel tag matching.) + +Beyond installers, the ``abi3.abi3t`` tag allow mechanisms like the ABI filter +in PyPI file list (e.g. on ``__) +to match ``abi3`` without special-casing (assuming compressed tags are handled +according to standard). +Humans wondering about compatibility with ``abi3`` also get a more +explicit signal. Less importantly, merging the ABIs would also remove an "escape hatch" of possibly making ``abi3`` and ``abi3t`` diverge in the future. -(Such divergence could only affect a few niche APIs, which can be only added -to ``abi3`` or ``abi3t``, but excluded from the intersection.) +.. _803-abi4: + Naming this ``abi4`` -------------------- @@ -896,35 +988,48 @@ better as a transitional state before larger changes like :pep:`809`'s ``abi2026``. +.. _803-combined-filename-tag: + ``abi3+abi3t`` filename tag --------------------------- Filename ABI tags (as introduced in :pep:`3149`) allow extensions for several ABIs to co-exist in a directory. -Extensions that are compatible with both ``abi3`` and ``abi3t`` can -use a "compressed tag set" (``abi3.abi3t``) in wheel metadata, +Per this PEP, extensions that are compatible with both ``abi3`` and ``abi3t`` +will use a compressed tag set (``abi3.abi3t``) in wheel metadata, but not in filenames (``.abi3.so``/``.abi3t.so``). We *could* add a dedicated tag for the combination -- for example, ``.abi3+abi3t.so``. -There would be no need for ``.abi3+abi3t.so`` extensions to co-exist with -``.abi3t.so`` ones, so we opt to stay away from the path toward -`compressed tag sets `_ in filenames, -and make the practical choice to make ``.abi3t.so`` mean "abi3+abi3t", -that is "loadable by all builds". +But, there would be no need for ``.abi3+abi3t.so`` extensions to co-exist with +``.abi3t.so`` ones: free-threaded interpreters would always pick ``.abi3t.so``, +so the extension for GIL-enabled interpreters could just as well use +``.abi3.so``. +The only benefit would be clearer naming when an ``abi3.abi3t`` extension +is *not* installed together with its ``abi3``-only equivalent. + +Here, clearer naming is not worth the complexity. +We make the practical choice to make ``.abi3t.so`` mean +"abi3+abi3t", that is, "loadable by all builds". +This works for (discouraged) ``abi3t``-only extensions: on a GIL-enabled +interpreter, these will fail the mandatory :ref:`runtime ABI check <803-runtime-check>` +or, in the unlikely future where ``abi3t`` & ``abi3`` diverge, possibly fail +to load due to a missing linker symbol. Conceptually, filename tags do not "describe" or "name" an extension's ABI. -(The current ``.abi3`` tag is too weak for this, as it lacks a version.) +The current ``.abi3`` tag is already too weak for this, as it lacks a version. + +.. _803-bare-so: No filename tag (bare ``.so``) ------------------------------ It would be possible to drop the filename ABI tag altogether, -and using ``.so`` instead of ``.abi3t.so``. -Their behavior are very close. -``.so`` is loadable by *any* build of CPython. +and use ``.so`` instead of ``.abi3t.so``. +The practical meaning of these two tags is very close: +``.so`` is loadable by *any* build of CPython; ``.abi3t.so`` will be loadable by any CPython *3.15 or above* -- but the Stable ABI filename tag already lacks version information. @@ -933,6 +1038,8 @@ Bare ``.so`` means "don't care"; an ``.abi3t.so`` extension is *intentionally* compatible with the new ABI. +.. _803-knob-gildisabled: + Reusing ``Py_GIL_DISABLED`` to enable the new ABI ------------------------------------------------- @@ -940,7 +1047,28 @@ It would be possible to select ``abi3t`` (rather than ``abi3``) when the ``Py_GIL_DISABLED`` macro is defined together with ``Py_LIMITED_API``. This would require annoying fiddling with build flags, and make it -especially cumbersome to target *both* ``abi3`` and ``abi3t`` at the same time. +impossible to explicitly target both ``abi3`` and ``abi3t`` at the same time. + +.. _803-freeze-pyobject: + +Fully separate ABIs, keeping source compatibility +------------------------------------------------- + +It would be possible to make ``abi3`` and ``abi3t`` fully separate and +incompatible. +This would allow all current extensions to stay *source*-compatible with +``abi3t``: the ``PyObject`` struct could stay exposed, with each ABI +defining a different set of private fields, as in the version-specific +CPython ABI. + +However, exposed ``PyObject`` struct has been `noted `__ +as one of the main shortcomings of the existing Stable ABI. +It hindered or prevented optimizations and features such as immortalization +and free-threading itself. + +Exposing ``PyObject`` would mean repeating this mistake, "freezing" +its current free-threaded definition, and requiring +*yet another* variant of stable ABI if/when changes are needed. Copyright