Skip to content

Commit f00a2ae

Browse files
committed
Last bit of polish
1 parent a9bcaa7 commit f00a2ae

File tree

1 file changed

+68
-48
lines changed

1 file changed

+68
-48
lines changed

peps/pep-0829.rst

Lines changed: 68 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -47,8 +47,8 @@ support two functions:
4747
``import\\t``) are executed immediately by passing the source string
4848
to ``exec()``.
4949

50-
While there are valid use cases for each feature, the ``import`` lines
51-
support are the most problematic because:
50+
While there are valid use cases for both, the ``import`` line feature is the
51+
most problematic because:
5252

5353
* Code execution is a side effect of the implementation. Lines that
5454
start with ``import`` can be extended by separating multiple
@@ -77,49 +77,61 @@ This PEP proposes the following:
7777
unchanged. Specifically, absolute paths are used verbatim while relative
7878
paths are anchored at the directory in which the ``.pth`` file is located.
7979

80-
* A new file format called ``<name>.start`` files are added which names entry
81-
points conforming to the "colon-form" of :func:`pkgutil.resolve_name`
82-
arguments.
80+
* A new file format called ``<name>.start`` is added which names entry points
81+
conforming to the "colon-form" of :func:`pkgutil.resolve_name` arguments.
8382

8483
* During the deprecation period, the presence of a ``<name>.start`` file
8584
matching a ``<name>.pth`` file disables the execution of ``import`` lines in
8685
the ``<name>.pth`` file in favor of entry points from the ``<name>.start``
8786
file. This provides a migration path straddling Python versions which
88-
support this PEP and earlier versions which do not.
87+
support this PEP and earlier versions which do not. In this case, warnings
88+
about ``import`` lines are *not* printed.
8989

9090
During the deprecation period, for any ``<name>.pth`` file without a
91-
matching ``<name>.start`` file, the processing of the former is unchanged (a
92-
warning about ``import`` lines may be printed). After the deprecation
91+
matching ``<name>.start`` file, the processing of the former is unchanged,
92+
although a warning about ``import`` lines is issued. After the deprecation
9393
period ``import`` lines in ``<name>.pth`` files are ignored and a warning is
9494
issued, regardless of whether there is a matching ``<name>.start`` file or
9595
not.
9696

9797
See the :ref:`teach` section for specific migration guidelines.
9898

99+
.. _dash-S: https://docs.python.org/3/using/cmdline.html#cmdoption-S
100+
101+
Both ``<name>.pth`` and ``<name>.start`` files are processed by the
102+
``site.py`` module, just like current ``.pth`` files. This means that
103+
`disabling site.py processing <dash-S_>`__ with ``-S`` disables processing of
104+
both files.
105+
99106
``site.py`` start up code is divided into these explicit phases:
100107

101108
#. Find the ``<name>.pth`` files (see :ref:`discovery` for additional details)
102109
and sort them in alphabetical order by filename.
103110

104-
#. Parse the ``<name>.pth`` files in sorted order, keeping a global sorted
105-
list of all path extensions, ignoring duplicates.
111+
#. Parse the ``<name>.pth`` files in sorted order, keeping a global list of
112+
all path extensions, preserving order file-by-file and then by entry
113+
appearance. Duplicates are ignored.
106114

107115
#. *Future extension:* apply a :ref:`global policy filter <future>` on the
108-
sorted list of path extensions.
116+
list of path extensions.
109117

110-
#. Append path extensions in sorted order to ``sys.path``.
118+
#. Append path extensions to ``sys.path`` in the global preserved order.
111119

112120
#. List all ``<name>.start`` files (see :ref:`discovery` for additional
113121
details) and sort them in alphabetical order by filename.
114122

115-
#. Parse the ``<name>.start`` files in sorted order, keeping a global sorted
116-
list of all entry points, ignoring duplicates.
123+
#. Parse the ``<name>.start`` files in sorted order, keeping a global list of
124+
all entry points, preserving order file-by-file and then by entry
125+
appearance. Duplicates are :ref:`not ignored <duplicate-eps>`.
117126

118127
#. *Future extension:* apply a :ref:`global policy filter <future>` on the
119-
sorted list of entry points.
128+
list of entry points.
120129

121-
#. For each entry point, use :func:`pkgutil.resolve_name` to resolve the entry
122-
point into a callable. Call the entry point with no arguments.
130+
#. For each entry point in preserved order, use :func:`pkgutil.resolve_name`
131+
to resolve the entry point into a callable. Call the entry point with no
132+
arguments and any return value is discarded. The resolved object is not
133+
tested for callability before it is called (and thus any ``TypeError`` that
134+
might result is reported).
123135

124136
In both ``<name>.pth`` files and ``<name>.start`` files, comment lines
125137
(i.e. lines beginning with ``#`` as the first non-whitespace character) and
@@ -162,34 +174,32 @@ File Naming and Discovery
162174
``<name>.pth`` files are found today. The ``<name>.pth`` location stays the
163175
same.
164176

165-
* The discovery rules for ``<name>.start`` files is the same as ``<name>.pth``
166-
files today. File names that start with a single ``.`` (e.g. ``.start``)
167-
and files with OS-level hidden attributes (``UF_HIDDEN``,
177+
* The discovery rules for ``<name>.start`` files are the same as with
178+
``<name>.pth`` files today. File names that start with a single ``.``
179+
(e.g. ``.start``) and files with OS-level hidden attributes (``UF_HIDDEN``,
168180
``FILE_ATTRIBUTE_HIDDEN``) are excluded.
169181

170182

171183
Error Handling
172184
--------------
173185

174-
Errors are handled differently depending on the phase:
186+
During parsing, errors are generally skipped and only reported when ``-v``
187+
(verbose) flag is given to Python. Unlike with ``.pth`` files currently,
188+
processing does *not* abort for the entire file when an error is encountered.
189+
190+
#. If a ``<name>.pth`` or ``<name>.start`` file cannot be opened or read, it
191+
is skipped and processing continues to the next file.
175192

176-
Phase 1: Reading and Parsing
177-
If a ``<name>.pth`` or ``<name>.start`` file cannot be opened or read, it
178-
is skipped and processing continues to the next file. Errors are reported
179-
only when ``-v`` (verbose) is given.
193+
#. Invalid entry point specifications are skipped.
180194

181-
Phase 2: Execution
182-
Invalid or nonexistent file system paths are ignored, with errors printed
183-
to ``sys.stderr``. Processing continues to the next path entry.
195+
During execution, errors are printed to ``sys.stderr`` and processing
196+
continues.
184197

185-
Invalid entry point specifications are skipped. Exceptions during
186-
execution of the entry point are printed to ``sys.stderr``. Processing
187-
continues to the next entry point.
198+
#. Any ``sys.path`` extension directory pointing to an invalid or nonexistent
199+
path is ignored and processing continues to the next path entry.
188200

189-
Because of the 2-phase interpretation of ``<name>.pth`` and ``<name>.start``
190-
files, continued evaluation of paths and entry points is a deliberate
191-
improvement over current ``.pth`` behavior, which aborts processing the
192-
remainder of a file on the first error.
201+
#. Exceptions during execution of the entry point are printed and processing
202+
continues to the next entry point.
193203

194204

195205
.. _future:
@@ -246,8 +256,24 @@ for both ``sys.path`` extension and entry point invocation gives us a chance
246256
(:ref:`in the future <future>`) to design and implement global policies for
247257
explicitly controlling which path extensions and entry points are allowed (and
248258
by implication, deemed safe), without resorting to the heavy hammer of
249-
`disabling site.py processing completely
250-
<https://docs.python.org/3/using/cmdline.html#cmdoption-S>`_.
259+
`disabling site.py processing <dash-S_>`__ completely.
260+
261+
All valid ``sys.path`` extensions in all ``<name>.pth`` files found are
262+
processed *before* any entry points in ``<name>.start`` files are called.
263+
This is to ensure that all ``sys.path`` modifications required to import entry
264+
point modules are applied first.
265+
266+
.. _duplicate-eps:
267+
268+
Entry points are *not* de-duplicated, regardless of whether they're defined
269+
multiple times in the same ``<name>.start`` file or across more than one
270+
``<name>.start`` file. This means that if an entry point appears more than
271+
once it will get called more than once. Unlike with the de-duplication of
272+
``sys.path`` entries (where the appearance of a directory path later on
273+
``sys.path`` than its duplicate has no effect), users could -- however
274+
unlikely -- actually want multiple invocations of their entry points. This
275+
also avoids the complexity of defining de-duplicating entry point semantics
276+
across independently-authored ``<name>.start`` files.
251277

252278

253279
Backwards Compatibility
@@ -282,7 +308,7 @@ This PEP improves the security posture of interpreter startup.
282308
importable modules.
283309

284310
The overall attack surface is not eliminated -- a malicious package can still
285-
cause arbitrary code execution viaentry points, but the mechanism proposed in
311+
cause arbitrary code execution via entry points, but the mechanism proposed in
286312
this PEP is more structured, auditable, and amenable to future policy
287313
controls.
288314

@@ -294,7 +320,7 @@ How to Teach This
294320

295321
The :mod:`site` module documentation will be updated to describe the operation
296322
and best practices for ``<name>.pth`` and ``<name>.start`` files. The
297-
following guidelines for package authors will be include:
323+
following guidelines for package authors will be included:
298324

299325
If your package currently ships a ``<name>.pth`` file, analyze whether you are
300326
using it for ``sys.path`` extension or start up code execution. You can keep
@@ -363,9 +389,9 @@ Open Issues
363389
* As described in the :ref:`entry point syntax <ep-syntax>` section, this PEP
364390
proposes to use a narrow definition of the acceptable object reference
365391
syntax implemented by :func:`pkgutil.resolve_name`, i.e. specifically
366-
requiring the ``pkg.mod:callable`` syntax. This is because a) we don't want
367-
to encourage code execution by direct import side-effect (i.e. functionality
368-
at module scope level).
392+
requiring the ``pkg.mod:callable`` syntax. This is because we don't want to
393+
encourage code execution by direct import side-effect (i.e. functionality at
394+
module scope level).
369395

370396
Assuming this restriction is acceptable, how this is implemented is an open
371397
question. ``site.py`` could enforce it directly, but then that sort of
@@ -403,12 +429,6 @@ to its current form. Thanks also go to Emma Smith and Brett Cannon for their
403429
feedback and encouragement.
404430

405431

406-
Footnotes
407-
=========
408-
409-
None
410-
411-
412432
Copyright
413433
=========
414434

0 commit comments

Comments
 (0)