[Python-checkins] r56190 - peps/trunk/pep-0366.txt

Sun Jul 8 09:45:47 CEST 2007

Author: nick.coghlan
Date: Sun Jul 8 09:45:46 2007
New Revision: 56190
Modified:
 peps/trunk/pep-0366.txt
Log:
Major rewrite to simplify things and to incorporate python-dev feedback
Modified: peps/trunk/pep-0366.txt
==============================================================================

--- peps/trunk/pep-0366.txt	(original)
+++ peps/trunk/pep-0366.txt	Sun Jul 8 09:45:46 2007
@@ -7,8 +7,8 @@
 Type: Standards Track
 Content-Type: text/x-rst
 Created: 1-May-2007
-Python-Version: 2.6
-Post-History: 1-May-2007
+Python-Version: 2.6, 3.0
+Post-History: 1-May-2007, 4-Jul-2007, 7-Jul-2007
 
 
 Abstract
@@ -17,158 +17,95 @@
 This PEP proposes a backwards compatible mechanism that permits
 the use of explicit relative imports from executable modules within
 packages. Such imports currently fail due to an awkward interaction
-between PEP 328 and PEP 338 - this behaviour is the subject of at
-least one open SF bug report (#1510172)[1], and has most likely
-been a factor in at least a few queries on comp.lang.python (such
-as Alan Isaac's question in [2]).
-
-With the proposed mechanism, relative imports will work automatically
-if the module is executed using the ``-m`` switch. A small amount of
-boilerplate will be needed in the module itself to allow the relative
+between PEP 328 and PEP 338.
+
+By adding a new module level attribute, this PEP allows relative imports
+to work automatically if the module is executed using the ``-m``switch.
+A small amount of boilerplate in the module itself will allow the relative
 imports to work when the file is executed by name.
 
 
-Import Statements and the Main Module
-=====================================
+Proposed Change
+===============
+
+The major proposed change is the introduction of a new module level
+attribute, ``__package__``. When it is present, relative imports will
+be based on this attribute rather than the module ``__name__``
+attribute.
+
+As with the current ``__name__`` attribute, setting ``__package__`` will
+be the responsibility of the PEP 302 loader used to import a module.
+Loaders which use ``imp.new_module()`` to create the module object will
+have the new attribute set automatically to
+``__name__.rpartition('.')[0]``.
+
+``runpy.run_module`` will also set the new attribute, basing it off the
+``mod_name`` argument, rather than the ``run_name`` argument. This will
+allow relative imports to work correctly from main modules executed with
+the ``-m`` switch.
+
+When the main module is specified by its filename, then the
+``__package__`` attribute will be set to the empty string. To allow
+relative imports when the module is executed directly, boilerplate
+similar to the following would be needed before the first relative
+import statement:
+
+ if __name__ == "__main__" and not __package_name__:
+ __package_name__ = "<expected_pacakage_name>"
 
-(This section is taken from the final revision of PEP 338)
+Note that this boilerplate is sufficient only if the top level package
+is already accessible via ``sys.path``. Additional code that manipulates
+``sys.path`` would be needed in order for direct execution to work
+without the top level package already being importable.
 
-The release of 2.5b1 showed a surprising (although obvious in
-retrospect) interaction between PEP 338 and PEP 328 - explicit
-relative imports don't work from a main module. This is due to
-the fact that relative imports rely on ``__name__`` to determine
-the current module's position in the package hierarchy. In a main
-module, the value of ``__name__`` is always ``'__main__'``, so
-explicit relative imports will always fail (as they only work for
-a module inside a package).
-
-Investigation into why implicit relative imports *appear* to work when
-a main module is executed directly but fail when executed using ``-m``
-showed that such imports are actually always treated as absolute
-imports. Because of the way direct execution works, the package
-containing the executed module is added to sys.path, so its sibling
-modules are actually imported as top level modules. This can easily
-lead to multiple copies of the sibling modules in the application if
-implicit relative imports are used in modules that may be directly
-executed (e.g. test modules or utility scripts).
-
-For the 2.5 release, the recommendation is to always use absolute
-imports in any module that is intended to be used as a main module.
-The ``-m`` switch already provides a benefit here, as it inserts the
-current directory into ``sys.path``, instead of the directory containing
-the main module. This means that it is possible to run a module from
-inside a package using ``-m`` so long as the current directory contains
-the top level directory for the package. Absolute imports will work
-correctly even if the package isn't installed anywhere else on
-sys.path. If the module is executed directly and uses absolute imports
-to retrieve its sibling modules, then the top level package directory
-needs to be installed somewhere on sys.path (since the current directory
-won't be added automatically).
-
-Here's an example file layout::
-
- devel/
- pkg/
- __init__.py
- moduleA.py
- moduleB.py
- test/
- __init__.py
- test_A.py
- test_B.py
-
-So long as the current directory is ``devel``, or ``devel`` is already
-on ``sys.path`` and the test modules use absolute imports (such as
-``import pkg.moduleA`` to retrieve the module under test, PEP 338
-allows the tests to be run as::
+This approach also has the same disadvantage as the use of absolute
+imports of sibling modules - if the script is moved to a different
+package or subpackage, the boilerplate will need to be updated
+manually.
 
- python -m pkg.test.test_A
- python -m pkg.test.test_B
 
 Rationale for Change
 ====================
 
-In rejecting PEP 3122 (which proposed a higher impact solution to this
-problem), Guido has indicated that he still isn't particularly keen on
-the idea of executing modules inside packages as scripts [2]. Despite
-these misgivings he has previously approved the addition of the ``-m``
-switch in Python 2.4, and the ``runpy`` module based enhancements
-described in PEP 338 for Python 2.5.
-
-The philosophy that motivated those previous additions (i.e. access to
-utility or testing scripts without needing to worry about name clashes in
-either the OS executable namespace or the top level Python namespace) is
-also the motivation behind fixing what I see as a bug in the current
-implementation.
+The current inability to use explicit relative imports from the main
+module is the subject of at least one open SF bug report (#1510172)[1],
+and has most likely been a factor in at least a few queries on
+comp.lang.python (such as Alan Isaac's question in [2]).
 
 This PEP is intended to provide a solution which permits explicit
 relative imports from main modules, without incurring any significant
 costs during interpreter startup or normal module import.
 
+The section in PEP 338 on relative imports and the main module provides
+further details and background on this problem.
 
-Proposed Solution
-=================
 
-The heart of the proposed solution is a new module attribute
-``__package_name__``. This attribute will be defined only in
-the main module (i.e. modules where ``__name__ == "__main__"``).
-
-For a directly executed main module, this attribute will be set
-to the empty string. For a module executed using
-``runpy.run_module()`` with the ``run_name`` parameter set to
-``"__main__"``, the attribute will be set to
-``mod_name.rpartition('.')[0]`` (i.e., everything up to
-but not including the last period).
-
-In the import machinery there is an error handling path which
-deals with the case where an explicit relative reference attempts
-to go higher than the top level in the package hierarchy. This
-error path would be changed to fall back on the ``__package_name__``
-attribute for explicit relative imports when the importing module
-is called ``"__main__"``.
-
-With this change, explicit relative imports will work automatically
-from a script executed with the ``-m`` switch. To allow direct
-execution of the module, the following boilerplate would be needed at
-the top of the script::
+Reference Implementation
+========================
 
- if __name__ == "__main__" and not __package_name__:
- __package_name__ = "<expected_pkg_name>"
-
-Note that this boilerplate is sufficient only if the top level package
-is already accessible via sys.path. Additional code that manipulates
-sys.path would be needed in order for direct execution to work
-without the top level package already being on sys.path.
-
-This approach also has the same disadvantage as the use of absolute
-imports of sibling modules - if the script is moved to a different
-package or subpackage, the boilerplate will need to be updated
-manually.
-
-With this feature in place, the test scripts in the package above
-would be able to change their import lines to something along the
-lines of ``import ..moduleA``. The scripts could then be
-executed unmodified even if the name of the package was changed.
-
-(Rev 47142 in SVN implemented an early variant of this proposal
+Rev 47142 in SVN implemented an early variant of this proposal
 which stored the main module's real module name in the
 '__module_name__' attribute. It was reverted due to the fact
-that 2.5 was already in beta by that time.)
+that 2.5 was already in beta by that time.
+
+A new patch will be developed for 2.6, and forward ported to
+Py3k via svnmerge.
 
 
 Alternative Proposals
 =====================
 
 PEP 3122 proposed addressing this problem by changing the way
-the main module is identified. That's a huge compatibility cost
+the main module is identified. That's a significant compatibility cost
 to incur to fix something that is a pretty minor bug in the overall
-scheme of things.
+scheme of things, and the PEP was rejected [3].
 
 The advantage of the proposal in this PEP is that its only impact on
-normal code is the tiny amount of time needed at startup to set the extra
-attribute in the main module. The changes to the import machinery are all
-in an existing error handling path, so normal imports don't incur any
-performance penalty at all.
+normal code is the small amount of time needed to set the extra
+attribute when importing a module. Relative imports themselves should
+be sped up fractionally, as the package name is stored in the module
+globals, rather than having to be worked out again for each relative
+import.
 
 
 References
@@ -177,11 +114,12 @@
 .. [1] Absolute/relative import not working?
 (http://www.python.org/sf/1510172)
 
-.. [2] Guido's rejection of PEP 3122
+.. [2] c.l.p. question about modules and relative imports
+ (http://groups.google.com/group/comp.lang.python/browse_thread/thread/c44c769a72ca69fa/)
+
+.. [3] Guido's rejection of PEP 3122
 (http://mail.python.org/pipermail/python-3000/2007-April/006793.html)
 
-.. [3] c.l.p. question about modules and relative imports
- (http://groups.google.com/group/comp.lang.python/browse_thread/thread/c44c769a72ca69fa/)
 
 Copyright
 =========
