homepage

With the current default branch, test_venv fails every time for me:
[1/1] test_venv
Traceback (most recent call last):
 File "C:\Code\Python\lib\runpy.py", line 160, in _run_module_as_main
 "__main__", fname, loader, pkg_name)
 File "C:\Code\Python\lib\runpy.py", line 73, in _run_code
 exec(code, run_globals)
 File "C:\Code\Python\lib\ensurepip\__main__.py", line 66, in <module>
 main()
 File "C:\Code\Python\lib\ensurepip\__main__.py", line 61, in main
 default_pip=args.default_pip,
 File "C:\Code\Python\lib\ensurepip\__init__.py", line 92, in bootstrap
 _run_pip(args + [p[0] for p in _PROJECTS], additional_paths)
 File "C:\Code\Python\lib\ensurepip\__init__.py", line 28, in _run_pip
 import pip
 File "C:\Users\Tim\AppData\Local\Temp\tmpjltqdgi8\pip-1.5.rc1-py2.py3-none-any.whl\pip\__init__.py", line 10
, in <module>
 File "C:\Users\Tim\AppData\Local\Temp\tmpjltqdgi8\pip-1.5.rc1-py2.py3-none-any.whl\pip\util.py", line 17, in
 <module>
 File "C:\Users\Tim\AppData\Local\Temp\tmpjltqdgi8\pip-1.5.rc1-py2.py3-none-any.whl\pip\_vendor\distlib\versi
on.py", line 14, in <module>
 File "C:\Users\Tim\AppData\Local\Temp\tmpjltqdgi8\pip-1.5.rc1-py2.py3-none-any.whl\pip\_vendor\distlib\compat.py", line 66, in <module>
ImportError: cannot import name 'HTTPSHandler'
test test_venv failed -- Traceback (most recent call last):
 File "C:\Code\Python\lib\test\test_venv.py", line 288, in test_with_pip
 self.run_with_capture(venv.create, self.env_dir, with_pip=True)
 File "C:\Code\Python\lib\test\test_venv.py", line 48, in run_with_capture
 func(*args, **kwargs)
 File "C:\Code\Python\lib\venv\__init__.py", line 359, in create
 builder.create(env_dir)
 File "C:\Code\Python\lib\venv\__init__.py", line 86, in create
 self._setup_pip(context)
 File "C:\Code\Python\lib\venv\__init__.py", line 242, in _setup_pip
 subprocess.check_output(cmd)
 File "C:\Code\Python\lib\subprocess.py", line 618, in check_output
 raise CalledProcessError(retcode, process.args, output=output)
subprocess.CalledProcessError: Command '['C:\\Users\\Tim\\AppData\\Local\\Temp\\tmpt0ca1aqn\\Scripts\\python_d
.exe', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1
1 test failed:
 test_venv
All virtual Greek to me.

Interesting - this isn't *quite* a duplicate of the buildbot failures in issue 19734 (at least, I don't think it is - the extra diagnostics I just checked in should tell us for sure).
Since pip isn't useful without HTTPS, we may want to add a check for ssl support to either venv or ensurepip itself.

Ah, I didn't even notice the "S" in "HTTPS"! I'm not building the SSL cruft on my box, so it's not surprising that anything requiring it would fail. It is surprising that this is the only test that _does_ fail without it ;-)

FYI, here's the new output:
[1/1] test_venv
test test_venv failed -- Traceback (most recent call last):
 File "C:\Code\Python\lib\test\test_venv.py", line 289, in test_with_pip
 self.run_with_capture(venv.create, self.env_dir, with_pip=True)
subprocess.CalledProcessError: Command '['C:\\Users\\Tim\\AppData\\Local\\Temp\\tmptw2_vda6\\Scripts\\python_d
.exe', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
 File "C:\Code\Python\lib\test\test_venv.py", line 295, in test_with_pip
 self.fail(msg)
AssertionError: Command '['C:\\Users\\Tim\\AppData\\Local\\Temp\\tmptw2_vda6\\Scripts\\python_d.exe', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1
**Subprocess Output**
Traceback (most recent call last):
 File "C:\Code\Python\lib\runpy.py", line 160, in _run_module_as_main
 "__main__", fname, loader, pkg_name)
 File "C:\Code\Python\lib\runpy.py", line 73, in _run_code
 exec(code, run_globals)
 File "C:\Code\Python\lib\ensurepip\__main__.py", line 66, in <module>
 main()
 File "C:\Code\Python\lib\ensurepip\__main__.py", line 61, in main
 default_pip=args.default_pip,
 File "C:\Code\Python\lib\ensurepip\__init__.py", line 92, in bootstrap
 _run_pip(args + [p[0] for p in _PROJECTS], additional_paths)
 File "C:\Code\Python\lib\ensurepip\__init__.py", line 28, in _run_pip
 import pip
 File "C:\Users\Tim\AppData\Local\Temp\tmpdkfwi7it\pip-1.5.rc1-py2.py3-none-any.whl\pip\__init__.py", line 10
, in <module>
 File "C:\Users\Tim\AppData\Local\Temp\tmpdkfwi7it\pip-1.5.rc1-py2.py3-none-any.whl\pip\util.py", line 17, in
 <module>
 File "C:\Users\Tim\AppData\Local\Temp\tmpdkfwi7it\pip-1.5.rc1-py2.py3-none-any.whl\pip\_vendor\distlib\version.py", line 14, in <module>
 File "C:\Users\Tim\AppData\Local\Temp\tmpdkfwi7it\pip-1.5.rc1-py2.py3-none-any.whl\pip\_vendor\distlib\compat.py", line 66, in <module>
ImportError: cannot import name 'HTTPSHandler'
1 test failed:
 test_venv

New changeset 9891ba920f3c by Nick Coghlan in branch 'default':
Issue #19744 (temp workaround): without ssl, skip pip test
http://hg.python.org/cpython/rev/9891ba920f3c 

Temporarily skipped the test to appease the build bots for the beta release, but this should be changed so that ensurepip refuses to bootstrap pip if SSL/TLS support is not available.
test_venv would then be updated to check for the appropriate behaviour (e.g. by inserting a dummy ssl.py that just raises import error into the venv)

Also noting that the reason for the dummy ssl in the venv would be to provoke the "SSL/TLS not available" behaviour when running the tests in a Python that actually has those pieces (since the buildbots will have them available unless something goes wrong with the build)

There's a ticket in pip to make pip work even when ssl isn't available. You wouldn't be able to install from PyPI but you would be able to install from local archives.

Is that likely to be in 1.5 or 1.5.1? Not needing to special case this in
ensurepip would be nice :)

I've stumbled upon what appears to be a related issue, but I'm not sure it deserves its own bug report.
I compiled 3.4 on my LMDE (so essentially Debian testing) system, and aside from not building tkinter, various compression modules, etc., all went well. I ran `make test` with no errors, including the success of test_venv. (There were many and various warnings about deprecated code and calling str() on bytes instances, but I'm pretty sure none of it is related).
I ran `sudo make altinstall` (to not nuke my current 3.3 from the repos), and to my surprise, it failed with the following error:
running install_scripts
copying build/scripts-3.4/pydoc3.4 -> /usr/local/bin
copying build/scripts-3.4/2to3-3.4 -> /usr/local/bin
copying build/scripts-3.4/idle3.4 -> /usr/local/bin
copying build/scripts-3.4/pyvenv-3.4 -> /usr/local/bin
changing mode of /usr/local/bin/pydoc3.4 to 755
changing mode of /usr/local/bin/2to3-3.4 to 755
changing mode of /usr/local/bin/idle3.4 to 755
changing mode of /usr/local/bin/pyvenv-3.4 to 755
rm /usr/local/lib/python3.4/lib-dynload/_sysconfigdata.py
rm -r /usr/local/lib/python3.4/lib-dynload/__pycache__
/usr/bin/install -c -m 644 ./Misc/python.man \
 /usr/local/share/man/man1/python3.4.1
if test "xupgrade" != "xno" ; then \
 case upgrade in \
 upgrade) ensurepip="--altinstall --upgrade" ;; \
 install|*) ensurepip="--altinstall" ;; \
 esac; \
 ./python -E -m ensurepip \
 $ensurepip --root=/ ; \
 fi
Traceback (most recent call last):
 File "/home/bill/py3.4/Python-3.4.0b1/Lib/runpy.py", line 160, in _run_module_as_main
 "__main__", fname, loader, pkg_name)
 File "/home/bill/py3.4/Python-3.4.0b1/Lib/runpy.py", line 73, in _run_code
 exec(code, run_globals)
 File "/home/bill/py3.4/Python-3.4.0b1/Lib/ensurepip/__main__.py", line 66, in <module>
 main()
 File "/home/bill/py3.4/Python-3.4.0b1/Lib/ensurepip/__main__.py", line 61, in main
 default_pip=args.default_pip,
 File "/home/bill/py3.4/Python-3.4.0b1/Lib/ensurepip/__init__.py", line 92, in bootstrap
 _run_pip(args + [p[0] for p in _PROJECTS], additional_paths)
 File "/home/bill/py3.4/Python-3.4.0b1/Lib/ensurepip/__init__.py", line 28, in _run_pip
 import pip
 File "/tmp/tmprwpsemxj/pip-1.5.rc1-py2.py3-none-any.whl/pip/__init__.py", line 10, in <module>
 File "/tmp/tmprwpsemxj/pip-1.5.rc1-py2.py3-none-any.whl/pip/util.py", line 17, in <module>
 File "/tmp/tmprwpsemxj/pip-1.5.rc1-py2.py3-none-any.whl/pip/_vendor/distlib/version.py", line 14, in <module>
 File "/tmp/tmprwpsemxj/pip-1.5.rc1-py2.py3-none-any.whl/pip/_vendor/distlib/compat.py", line 66, in <module>
ImportError: cannot import name 'HTTPSHandler'
make: *** [altinstall] Error 1
Note: I will certainly *not* be trying to `sudo make install`.
In any case, the executable and modules were installed fine, so other than reporting it here, it's not causing me problems (so far).

It probably can. I just need to figure out how to test it to make sure the PR that supposedly fixes it fixes it, and then figure out how to ensure it still works into the future.

That I can help with. Steal the "import_fresh_module helper function from test.support (or the gist of it anyway - you can likely leave out the stuff about deprecated imports):
http://hg.python.org/cpython/file/default/Lib/test/support/__init__.py#l192
Then do:
 pip_nossl = import_fresh_module("pip", blocked=["ssl"])

If the ssl import is actually in a submodule, you may need to list additional subpackages in the "fresh" parameter.
If you're wondering why this isn't in the importlib API - it's because it can go wrong in an impressively large number of ways, and we don't have any hope of documenting it well enough to make it usable by anyone that either: a) couldn't write it themselves; or b) isn't getting coached by someone that could write it themselves.
But when you know what you're doing and the modules involved don't break horribly when treated this way, it's a very neat trick for testing purposes.

Relevant pip issue: https://github.com/pypa/pip/issues/1165 

Can this be solved in ensurepip for now? I've been banging away at this but it's going to require some refactoring in pip to make it reasonably work. The move to distlib and requests made this harder to do than the old PR against pip could handle.

OK, since pip 1.5 will still have the SSL/TLS dependency, the approach I'll go with for 3.4 is to:
1. Have ensurepip refuse to bootstrap pip if the ssl module is not available (noting that we'll remove that restriction if pip 1.6 avoids the strict dependency)
2. Use import_fresh_module to check that behaviour
3. Ensure venv skips trying to bootstrap pip if the ssl module is not available (although the subprocess invocation in the venv tests could make that tricky to test when the ssl module actually *is* available)

New changeset f670d8db8ef3 by Nick Coghlan in branch 'default':
Issue #19744: improve ensurepip error when ssl is missing
http://hg.python.org/cpython/rev/f670d8db8ef3 

I ended up not implementing step 3 - if you don't have SSL/TLS built, and you pass with_pip to the venv module API, or use the default settings for pyvenv, you *will* get an error from ensurepip.
Instead, I just kept the test skip in test_venv. ensurepip and test_ensurepip have been updated to provide a better traceback when SSL/TLS support is missing, though.
Tim, could you poke around at the latest version in your local build and see if the new checks are triggering? (I've assumed the ssl module can't be imported if the necessary underlying bits aren't built)

This should be fixed, so I don't think it's a release blocker any more, but I also don't want to close it until Tim confirms it also works for him.

This upsets "make install" as well - currently with a traceback.

Attached patch tweaks the ensurepip CLI to check immediately for ssl support, and avoid printing a traceback if it is missing.
However, missing SSL support still fails the call, which fails installation.

This patch is probably a better bet - it just prints a message to stderr to say that we're ignoring the ensurepip failure during installation.
pip wouldn't work anyway in a Python without ssl built, but at least this way that Python can still be installed without the ensurepip invocation complaining.
Ned, if this approach sounds reasonable to you, I'll commit this one.

Note: I'm deliberately not worrying about ensurepip._uninstall here, since that only gets invoked implicitly in the Windows uninstaller, and that should always have a valid SSL to play with.
Always, since I worked out how to disable ssl in my local build (just a small tweak to setup.py), I was able to verify the correct behaviour of test_ensurepip and test_venv with SSL unavailable.

issue19744_ensurepip_install_ok_without_ssl.diff looks good to me. Note, though, that with it and with ssl support missing, test_ensurepip fails somewhat obscurely:
======================================================================
FAIL: test_bootstrap_version (test.test_ensurepip.TestBootstrappingMainFunction)
----------------------------------------------------------------------
Traceback (most recent call last):
 File "/py/dev/3x/root/uxd/lib/python3.4/test/test_ensurepip.py", line 293, in test_bootstrap_version
 ensurepip._main(["--version"])
AssertionError: SystemExit not raised
----------------------------------------------------------------------

New changeset 9f76adbac8b7 by Nick Coghlan in branch 'default':
Issue #19744: Handle missing SSL/TLS in ensurepip
http://hg.python.org/cpython/rev/9f76adbac8b7 

Thanks Ned - I fixed that test to only run if SSL/TLS is available, and added a new one to test that the command "succeeds" (with a warning printed to stderr) if SSL/TLS is missing.
Larry - over to you to decide whether or not to cherry pick this into the release clone. The remaining misbehaviour that was fixed in the last patch only affects custom source builds, so the beneficiaries would be people trying to build from the source tarball or release tag without SSL/TLS support.

Issue 20685 created to cover inclusion in 3.4.0, already committed to default, so closing this one.

New changeset cd39d4cab680 by Nick Coghlan in branch '3.4':
Issue #19744: Handle missing SSL/TLS in ensurepip
http://hg.python.org/cpython/rev/cd39d4cab680