homepage

Mac OS builds fail due to test_site in VSTS for 3.7 branches only
Ref : https://python.visualstudio.com/cpython/_build/results?buildId=17921&view=logs
2018年07月21日T07:01:08.5912930Z Traceback (most recent call last):
2018年07月21日T07:01:08.5927520Z File "/Users/vsts/agent/2.136.1/work/1/s/Lib/test/test_site.py", line 199, in test_s_option
2018年07月21日T07:01:08.5942120Z self.assertIn(usersite, sys.path)
2018年07月21日T07:01:08.5959870Z AssertionError: '/Users/vsts/.local/lib/python3.7/site-packages' not found in ['/Users/vsts/agent/2.136.1/work/1/s', '/usr/local/lib/python37.zip', '/Users/vsts/agent/2.136.1/work/1/s/Lib', '/Users/vsts/agent/2.136.1/work/1/s/build/lib.macosx-10.13-x86_64-3.7-pydebug', '/usr/local/lib/python3.7/site-packages']
2018年07月21日T07:01:08.5971110Z 
2018年07月21日T07:01:08.5985860Z ======================================================================
2018年07月21日T07:01:08.6000880Z FAIL: test_startup_imports (test.test_site.StartupImportTests)
2018年07月21日T07:01:08.6017050Z ----------------------------------------------------------------------
2018年07月21日T07:01:08.6033630Z Traceback (most recent call last):
2018年07月21日T07:01:08.6048270Z File "/Users/vsts/agent/2.136.1/work/1/s/Lib/test/test_site.py", line 487, in test_startup_imports
2018年07月21日T07:01:08.6063180Z self.assertFalse(modules.intersection(collection_mods), stderr)
2018年07月21日T07:01:08.6079990Z AssertionError: {'reprlib', 'heapq', 'collections', 'keyword', 'operator', 'functools', 'types'} is not false : import _frozen_importlib # frozen
I couldn't find any related issues in search and hence opened this. Feel free to close this if it's a duplicate or fixed in some other branch.
Thanks

It seems that the last successful build in 3.7 was for https://github.com/python/cpython/commit/339e0c1296c61c9dbc1f8f880c5c668bf4007e5e and all PRs after it had the same failure.
Sorry for not adding the relevant components during ticket creation.
Thanks

Adding @vstinner as a friendly reminder since buildbot related failures are notified. This issue causes all the PRs for 3.7 on Mac OS VSTS builds to fail since July 19 though other Mac related builds are passing.
Feel free to remove yourself or close this if it's irrelevant.
Thanks

Maybe, vsts machine installs Homebrew's Python.
Homebrew's Python installes /usr/local/lib/python3.7/sitecustomize.py
https://github.com/Homebrew/homebrew-core/blob/0ce9a5a481b3472608fe4e4188cb5d243deebb99/Formula/python.rb#L295-L329
When running tests, it is imported.
It causes test_startup_imports fail.
Maybe, we can use random prefix instead of default /usr/local to avoid this conflict.

New changeset 777cdd94b9bebd5b22df0fc293aa078d5537b988 by INADA Naoki in branch '3.6':
bpo-34177: vsts: Avoid conflict with Homebrew Python (GH-8430)
https://github.com/python/cpython/commit/777cdd94b9bebd5b22df0fc293aa078d5537b988

Thank you for the fix Naokia!

This error is happening in the x86-64 High Sierra 3.7 buildbot:
https://buildbot.python.org/all/#/builders/147/builds/174
======================================================================
FAIL: test_startup_imports (test.test_site.StartupImportTests)
----------------------------------------------------------------------
Traceback (most recent call last):
 File "/Users/buildbot/buildarea/3.7.billenstein-sierra/build/Lib/test/test_site.py", line 487, in test_startup_imports
 self.assertFalse(modules.intersection(collection_mods), stderr)
AssertionError: {'collections', 'heapq', 'types', 'functools', 'keyword', 'reprlib', 'operator'} is not false : import _frozen_importlib # frozen

Same fix can be applied to buildbot.
This is caused by (a) Homebrew installs Python to /usr/local with (hacky) sitecustomize, and (b) Python's default prefix is /usr/local.
I think test should be run without conflicting with another installation.

> Same fix can be applied to buildbot.
Sure. Go ahead :-) The buildbot configuration can be found at:
https://github.com/python/buildmaster-config/
Send a pull request, and Zachary Ware (or me) will review it. Zach is the expert here ;-)

I think this can be closed with https://github.com/python/buildmaster-config/commit/9a456462fb891328b5b8a170522d5f56f480fdfb. The buildbots are also green now with the failure reported in https://buildbot.python.org/all/#/builders/147/builds/174 fixed with the commit.
Thanks