[Python-checkins] r59955 - peps/trunk/pep-0000.txt peps/trunk/pep-0369.txt peps/trunk/pep-0370.txt

Mon Jan 14 21:42:39 CET 2008

Author: christian.heimes
Date: Mon Jan 14 21:42:39 2008
New Revision: 59955
Added:
 peps/trunk/pep-0369.txt (contents, props changed)
 peps/trunk/pep-0370.txt (contents, props changed)
Modified:
 peps/trunk/pep-0000.txt
Log:
Added PEP 369 and PEP 370
Modified: peps/trunk/pep-0000.txt
==============================================================================

--- peps/trunk/pep-0000.txt	(original)
+++ peps/trunk/pep-0000.txt	Mon Jan 14 21:42:39 2008
@@ -94,6 +94,8 @@
 S 364 Transitioning to the Py3K Standard Library Warsaw
 S 365 Adding the pkg_resources module Eby
 S 368 Standard image protocol and class Mastrodomenico
+ S 369 Post import hooks Heimes
+ S 370 Per user site-packages directory Heimes
 S 3108 Standard Library Reorganization Cannon
 S 3134 Exception Chaining and Embedded Tracebacks Yee
 S 3135 New Super Spealman, Delaney
@@ -467,6 +469,8 @@
 SF 366 Main module explicit relative imports Coghlan
 SR 367 New Super Spealman, Delaney
 S 368 Standard image protocol and class Mastrodomenico
+ S 369 Post import hooks Heimes
+ S 370 Per user site-packages directory Heimes
 SR 666 Reject Foolish Indentation Creighton
 SR 754 IEEE 754 Floating Point Special Values Warnes
 P 3000 Python 3000 GvR
@@ -571,6 +575,7 @@
 Griffin, Grant g2 at iowegian.com
 Hammond, Mark mhammond at skippinet.com.au
 Harris, Peter scav at blueyonder.co.uk
+ Heimes, Christian christian at cheimes.de
 Heller, Thomas theller at python.net
 Hetland, Magnus Lie magnus at hetland.org
 Hettinger, Raymond python at rcn.com
Added: peps/trunk/pep-0369.txt
==============================================================================
--- (empty file)
+++ peps/trunk/pep-0369.txt	Mon Jan 14 21:42:39 2008
@@ -0,0 +1,235 @@
+PEP: 369
+Title: Post import hooks
+Version: $Revision$
+Last-Modified: $Date$
+Author: Christian Heimes <christian(at)cheimes(dot)de>
+Status: Draft
+Type: Standards Track
+Content-Type: text/x-rst
+Created: 02-Jan-2008
+Python-Version: 2.6, 3.0
+Post-History:
+
+
+Abstract
+========
+
+This PEP proposes enhancements for the import machinery to add 
+post import hooks. It is intended primarily to support the wider
+use of abstract base classes that is expected in Python 3.0.
+
+The PEP originally started as a combined PEP for lazy imports and 
+post import hooks. After some discussion on the python-dev mailing
+list the PEP was parted in two separate PEPs. [1]_
+
+
+Rationale
+=========
+
+Python has no API to hook into the import machinery and execute code
+*after* a module is successfully loaded. The import hooks of PEP 302 are
+about finding modules and loading modules but they were not designed to
+as post import hooks.
+ 
+
+Use cases
+=========
+
+A use case for a post import hook is mentioned in Nick Coghlan's initial
+posting [2]_. about callbacks on module import. It was found during the
+development of Python 3.0 and its ABCs. We wanted to register classes 
+like decimal.Decimal with an ABC but the module should not be imported
+on every interpreter startup. Nick came up with this example::
+
+ @imp.when_imported('decimal')
+ def register(decimal):
+ Inexact.register(decimal.Decimal)
+
+The function ``register`` is registered as callback for the module named
+'decimal'. When decimal is imported the function is called with the
+module object as argument.
+
+While this particular example isn't necessary in practice, (as
+decimal.Decimal will inherit from the appropriate abstract Number base
+class in 2.6 and 3.0), it still illustrates the principle.
+
+
+Existing implementations
+========================
+
+PJE's peak.util.imports [3]_ implements post load hooks. My
+implementation shares a lot with his and it's partly based on his ideas.
+
+
+Post import hook implementation
+===============================
+
+Post import hooks are called after a module has been loaded. The hooks
+are callable which take one argument, the module instance. They are 
+registered by the dotted name of the module, e.g. 'os' or 'os.path'.
+
+The callable are stored in the dict ``sys.post_import_hooks`` which
+is a mapping from names (as string) to a list of callables or None.
+
+
+States
+------
+
+No hook was registered
+''''''''''''''''''''''
+
+sys.post_import_hooks contains no entry for the module
+
+A hook is registered and the module is not loaded yet
+'''''''''''''''''''''''''''''''''''''''''''''''''''''
+
+The import hook registry contains an entry 
+sys.post_import_hooks["name"] = [hook1]
+
+A module is successfully loaded
+'''''''''''''''''''''''''''''''
+
+The import machinery checks if sys.post_import_hooks contains post import
+hooks for the newly loaded module. If hooks are found then the hooks are
+called in the order they were registered with the module instance as first
+argument. The processing of the hooks is stopped when a method raises an
+exception. At the end the entry for the module name is removed from
+sys.post_import_hooks, even when an error has occured.
+
+A module can't be loaded
+''''''''''''''''''''''''
+
+The import hooks are neither called nor removed from the registry. It
+may be possible to load the module later.
+
+A hook is registered but the module is already loaded
+'''''''''''''''''''''''''''''''''''''''''''''''''''''
+
+The hook is fired immediately. 
+
+
+C API
+-----
+
+New PyImport_* API functions
+''''''''''''''''''''''''''''
+
+``PyObject* PyImport_GetPostImportHooks(void)``
+ Returns the dict sys.post_import_hooks or NULL
+
+``PyObject* PyImport_NotifyModuleLoaded(PyObject *module)``
+ Notify the post import system that a module was requested. Returns the
+ module or NULL if an error has occured.
+
+``PyObject* PyImport_RegisterPostImportHook(PyObject *callable, PyObject *mod_name)``
+ Register a new hook ``callable`` for the module ``mod_name``
+
+The ``PyImport_PostImportNotify()`` method is called by
+``PyImport_ImportModuleLevel()``::
+
+ PyImport_ImportModuleLevel(...)
+ {
+ ...
+ result = import_module_level(name, globals, locals, fromlist, level);
+ result = PyImport_PostImportNotify(result);
+ ...
+ }
+
+
+Python API
+----------
+
+The import hook registry and two new API methods are exposed through the
+``sys`` and ``imp`` module.
+
+sys.post_import_hooks
+ The dict contains the post import hooks: {"name" : [hook1, hook2], ...}
+
+imp.register_post_import_hook(hook: "callable", name: str)
+ Register a new hook *hook* for the module *name*
+
+imp.notify_module_loaded(module: "module instance") -> module
+ Notify the system that a module has been loaded. The method is provided
+ for compatibility with existing lazy / deferred import extensions.
+
+The when_imported function decorator is also in the imp module,
+which is equivalent to::
+
+ def when_imported(name):
+ def register(hook):
+ register_post_import_hook(hook, name)
+ return register
+
+imp.when_imported(name) -> decorator function
+ for @when_imported(name) def hook(module): pass
+
+
+Open issues
+===========
+
+The when_imported decorator hasn't been written.
+
+The code contains several XXX comments. They are mostly about error
+handling in edge cases.
+
+
+Backwards Compatibility
+=======================
+
+The new features and API don't conflict with old import system of Python
+and don't cause any backward compatibility issues for most software.
+However systems like PEAK and Zope which implement their own lazy import
+magic need to follow some rules.
+
+The post import hooks carefully designed to cooperate with existing
+deferred and lazy import systems. It's the suggestion of the PEP author
+to replace own on-load-hooks with the new hook API. The alternative
+lazy or deferred imports will still work but the implementations must
+call the ``imp.notify_module_loaded`` function.
+
+
+Reference Implementation
+========================
+
+A reference implementation is already written and is available in the 
+*py3k-importhook* branch. [4]_ It still requires some cleanups,
+documentation updates and additional unit tests.
+
+
+Acknowledgments
+===============
+
+Nick Coghlan, for proof reading and the initial discussion
+Phillip J. Eby, for his implementation in PEAK and help with my own implementation
+
+
+Copyright
+=========
+
+This document has been placed in the public domain.
+
+
+References
+==========
+
+.. [1] PEP: Lazy module imports and post import hook
+ http://permalink.gmane.org/gmane.comp.python.devel/90949
+
+.. [2] Interest in PEP for callbacks on module import
+ http://permalink.gmane.org/gmane.comp.python.python-3000.devel/11126
+
+.. [3] peak.utils.imports
+ http://svn.eby-sarna.com/Importing/peak/util/imports.py?view=markup
+
+.. [4] py3k-importhook branch
+ http://svn.python.org/view/python/branches/py3k-importhook/
+
+
+..
+ Local Variables:
+ mode: indented-text
+ indent-tabs-mode: nil
+ sentence-end-double-space: t
+ fill-column: 70
+ coding: utf-8
+ End:
Added: peps/trunk/pep-0370.txt
==============================================================================
--- (empty file)
+++ peps/trunk/pep-0370.txt	Mon Jan 14 21:42:39 2008
@@ -0,0 +1,229 @@
+PEP: 370
+Title: Per user site-packages directory
+Version: $Revision$
+Last-Modified: $Date$
+Author: Christian Heimes <christian(at)cheimes(dot)de>
+Status: Draft
+Type: Standards Track
+Content-Type: text/x-rst
+Created: 11-Jan-2008
+Python-Version: 2.6, 3.0
+Post-History:
+
+
+Abstract
+========
+
+This PEP proposes a new a per user site-packages directory to allow
+users the local installation of Python packages in their home directory.
+
+
+Rationale
+=========
+
+Current Python versions don't have an unified way to install packages
+into the home directory of an user (except for Mac Framework
+builds). Users are either forced to ask the system administrator to
+install or update a package for them or to use one of the many
+workaround like Virtual Python [1]_, Working Env [2]_ or 
+Virtual Env [3]_.
+
+It's not the goal of the PEP to replace the tools or to implement
+isolated installations of Python. It only implements the most common
+use case of an additional site-packages directory for each user.
+
+The feature can't be implemented using the environment variable
+*PYTHONPATH*. The env var just inserts a new directory to the beginning
+of *sys.path* but it doesn't parse the pth files in the directory. A
+full blown site-packages path is required for several applications
+and Python eggs.
+
+
+Specification
+=============
+
+site directory (site-packages)
+
+ A directory in ``sys.path``. In contrast to ordinary directories the pth
+ files in the directory are processed, too.
+
+user site directory
+
+ A site directory inside the users' home directory. An user site
+ directory is specific to a Python version. The path contains
+ the version number (major and minor only).
+
+ Mac
+ ``~/Library/Python/2.6/site-packages``
+ Unix
+ ``~/.local/lib/python2.6/site-packages``
+
+ Windows
+ ``%APPDATA%/Python/Python26/site-packages``
+
+user configuration directory
+
+ Usually the parent directory of the user site directory. It's meant
+ for Python version specific data like config files.
+
+ Mac
+ ``~/Library/Python/2.6``
+ Unix
+ ``~/.local/lib/python2.6``
+ Windows
+ ``%APPDATA%/Python/Python26``
+
+user base directory
+
+ It's located inside the user's home directory. The user site and 
+ use config directory are inside the base directory. On some systems
+ the directory may be shared with 3rd party apps.
+
+ Mac
+ ``~/Library/Python``
+
+ Unix
+ ``~/.local``
+ Windows
+ ``%APPDATA%/Python``
+
+user script directory
+
+ A directory for binaries and scripts. [10]_ It's shared across Python
+ versions and the destination directory for scripts.
+
+ Mac
+ ``~/Library/Python/bin``
+ Unix
+ ``~/.local/bin``
+ Windows
+ ``%APPDATA%/Python/Scripts``
+
+
+On Windows ``APPDATA`` was chosen because it is the most logical place for
+application data. Microsoft recommands that software doesn't write to
+``USERPROFILE`` [5]_ and ``My Documents`` is not suited for application data,
+too. [8]_
+
+On Linux ``~/.local`` was chosen in favor over ``~/.python`` because the
+directory is already used by several other programs in analogy to
+``/usr/local``. [7]_
+
+
+Implementation
+==============
+
+The site module gets a new method ``adduserpackage()`` which adds the
+appropriate directory to the search path. The directory is not added if
+it doesn't exist when Python is started. However the location of the 
+user site directory and user base directory is stored in an internal 
+variable for distutils.
+
+The user site directory is added before the system site directories
+but after Python's search paths and ``PYTHONPATH``. This setup allows
+the user to install a different version of a package than the system
+administrator but it prevents the user from accidently overwriting a
+stdlib module. Stdlib modules can still be overwritten with
+``PYTHONPATH``.
+
+For security reasons the user site directory is *not* added to
+``sys.path`` when the effective user id is not equal to the process
+user id [9]_. It prevents users from injecting Python code into suid
+apps.
+
+The user site directory can be suppressed with a new option ``-s`` or
+the environment variable ``PYTHONNOUSERSITE``. The feature can be
+disabled globally by setting ``site.ENABLE_USER_SITE`` to the value
+``False``. It must be set by editing ``site.py``. It can't be altered
+``in sitecustomize.py`` or later.
+
+``distutils.command.install`` (setup.py install) gets a new argument
+``--user`` to install packages in the user site directory. The required
+directories are created on demand.
+
+``distutils.sysconfig`` will get methods to access the private variables
+of site. (not yet implemented)
+
+The Windows updater needs to be updated, too. It should create an menu
+item which opens the user site directory in a new explorer windows.
+
+
+Backwards Compatibility
+=======================
+
+TBD
+
+
+Open Questions
+==============
+
+* Are the directories for Windows, Mac and Unix fine?
+
+* Mac: Should framework and non-framework builds of Python use the
+ same directories?
+
+* The patch also adds a usecustomize hook to site. Is it useful and
+ should it stay?
+
+* Should the site package directory also be ignored if process gid !=
+ effective gid?
+
+* Should the Windows installer add ``%APPDATA%/Python/Scripts`` to
+ ``PATH``?
+
+
+Reference Implementation
+========================
+
+A reference implementation is available in the bug tracker. [4]_
+
+
+Copyright
+=========
+
+This document has been placed in the public domain.
+
+
+References
+==========
+
+.. [1] Virtual Python
+ http://peak.telecommunity.com/DevCenter/EasyInstall#creating-a-virtual-python
+
+.. [2] Working Env
+ http://pypi.python.org/pypi/workingenv.py
+ http://blog.ianbicking.org/workingenv-revisited.html
+
+.. [3] Virtual Env
+ http://pypi.python.org/pypi/virtualenv
+
+.. [4] reference implementation
+ http://bugs.python.org/issue1799
+
+.. [5] MSDN: CSIDL
+ http://msdn2.microsoft.com/en-us/library/bb762494.aspx
+
+.. [6] Initial suggestion for a per user site-packages directory
+ http://permalink.gmane.org/gmane.comp.python.devel/90902
+
+.. [7] Suggestion of ~/.local/
+ http://permalink.gmane.org/gmane.comp.python.devel/90925
+
+.. [8] APPDATA discussion
+ http://permalink.gmane.org/gmane.comp.python.devel/90932
+
+.. [9] Security concerns and -s option
+ http://permalink.gmane.org/gmane.comp.python.devel/91063
+
+.. [10] Discussion about the bin directory
+ http://permalink.gmane.org/gmane.comp.python.devel/91095
+
+
+..
+ Local Variables:
+ mode: indented-text
+ indent-tabs-mode: nil
+ sentence-end-double-space: t
+ fill-column: 70
+ coding: utf-8
+ End:
