homepage

Changes made to /Library/Python on macOSX Sierra cause the --with-ensurepip compiler flag to fail, and lead to failures of pip after installing Python.
The new file that causes the problem on Sierra is:
/Library/Python/2.7/site-packages/Extras.pth
The current version of site.py reads that .pth file, which results in sys.path containing the path:
/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
The latter directory (which is protected by SIP) contains many Python 2.7 packages, including easy_install, setuptools, six, py2app, numpy, pylab and pyOpenSSL. The effect of including this SIP-protected path in sys.path is this: installing or upgrade a package in /Library/Frameworks/Python.framework which is also installed as an "Extra" in the /System/Frameworks/Python.framework will cause pip to attempt to delete the "old" package from its SIP-protected directory, leading to a "Permission Denied" exception and a failed install.
Given that Apple has now tied /Library/Python to the system Python in this way, thereby making a separate Python framework in /Library/Frameworks unusable, the natural solution to this problem would be to stop including any /Library/Python paths in sys.path.
I am attaching a patch that removes the block of code in site.py which adds these paths.

This also affects the -m ensurepip installation of pip itself: the pip install fails trying to upgrade the Apple-supplied version of setuptools. The behavior of adding the system Python's site-packages directory to the search path of all framework-build Pythons was deliberately added as a result of Issue4865. I think experience has shown this was not a good idea because of the coupling it introduced between separate Python installations and some third-party distributors of Python on OS X already patch this code out. It's currently only an issue for 2.7.x since Apple has not yet shipped versions of Python 3.x but the code should be removed there, too.

Another reason to remove this feature: installing python 2.7.12 using the installer on www.python.org breaks the system install of Python, likely because of this issue.
I'm a bit sad that this feature has to go, but modern Python use (in particular virtualenv) has reduced the need for this feature.

New changeset a8a342b3fbc7 by Ned Deily in branch '2.7':
Issue #28440: No longer add /Library/Python/site-packages, the Apple-supplied
https://hg.python.org/cpython/rev/a8a342b3fbc7 

I've pushed the change to revert Issue4865 for the 2.7 branch for release with 2.7.13. I am leaving this open to push similar changes for 3.x branches in anticipation of Apple someday supplying system Python 3.x. It would also be good to see if the post-install script that runs ensurepip can be made to fail more noisily.

New changeset 763f094b1f0ce2a07768828e78afe9687e9ad3bb by Ned Deily in branch 'master':
bpo-28440: Don't add /Library/Python/3.x/site-packages to sys.path (#5445)
https://github.com/python/cpython/commit/763f094b1f0ce2a07768828e78afe9687e9ad3bb

In response to msg282272: That day has come, the beta for 10.15 contains Python 3.7.3.

> That day has come, the beta for 10.15 contains Python 3.7.3.
Yay! FTR, 763f094b1f0ce2a07768828e78afe9687e9ad3bb was released in 3.7.0.
The only reason I have left this issues open was that I also wanted to make sure that ensurepip failures in the post-install script do not fail silently.