Python3

Upgrading beyond 3.6.9 on Ubuntu bionic

The most straight-forward way is to use the deadsnakes PPA.

Other ansible roles for Python:

• https://github.com/mdklatt/ansible-python3-role
• https://github.com/fubarhouse/ansible-role-python

Testing the python installed from deadsnakes PPA on rosetta

Before any changes:

taha@rosetta:~
$ ls -l /usr/bin/python*
lrwxrwxrwx 1 root root 9 Apr 16 2018 /usr/bin/python -> python2.7
lrwxrwxrwx 1 root root 9 Apr 16 2018 /usr/bin/python2 -> python2.7
-rwxr-xr-x 1 root root 3633000 Feb 27 16:10 /usr/bin/python2.7
lrwxrwxrwx 1 root root 33 Feb 27 16:10 /usr/bin/python2.7-config -> x86_64-linux-gnu-python2.7-config
lrwxrwxrwx 1 root root 16 Apr 16 2018 /usr/bin/python2-config -> python2.7-config
lrwxrwxrwx 1 root root 9 Mar 9 16:10 /usr/bin/python3 -> python3.6
-rwxr-xr-x 2 root root 4526456 Jan 26 16:33 /usr/bin/python3.6
lrwxrwxrwx 1 root root 33 Jan 26 16:33 /usr/bin/python3.6-config -> x86_64-linux-gnu-python3.6-config
-rwxr-xr-x 2 root root 4526456 Jan 26 16:33 /usr/bin/python3.6m
lrwxrwxrwx 1 root root 34 Jan 26 16:33 /usr/bin/python3.6m-config -> x86_64-linux-gnu-python3.6m-config
lrwxrwxrwx 1 root root 16 Oct 25 2018 /usr/bin/python3-config -> python3.6-config
-rwxr-xr-x 1 root root 384 Apr 11 2018 /usr/bin/python3-futurize
lrwxrwxrwx 1 root root 10 Mar 9 16:10 /usr/bin/python3m -> python3.6m
lrwxrwxrwx 1 root root 17 Oct 25 2018 /usr/bin/python3m-config -> python3.6m-config
-rwxr-xr-x 1 root root 388 Apr 11 2018 /usr/bin/python3-pasteurize
lrwxrwxrwx 1 root root 16 Apr 16 2018 /usr/bin/python-config -> python2.7-config

Now ran this role with ppa_deadsnakes.enabled and version 3.8.

taha@rosetta:~
$ ls -l /usr/bin/python*
lrwxrwxrwx 1 root root 9 Apr 16 2018 /usr/bin/python -> python2.7
lrwxrwxrwx 1 root root 9 Apr 16 2018 /usr/bin/python2 -> python2.7
-rwxr-xr-x 1 root root 3633000 Feb 27 16:10 /usr/bin/python2.7
lrwxrwxrwx 1 root root 33 Feb 27 16:10 /usr/bin/python2.7-config -> x86_64-linux-gnu-python2.7-config
lrwxrwxrwx 1 root root 16 Apr 16 2018 /usr/bin/python2-config -> python2.7-config
lrwxrwxrwx 1 root root 9 Mar 9 16:10 /usr/bin/python3 -> python3.6
-rwxr-xr-x 2 root root 4526456 Jan 26 16:33 /usr/bin/python3.6
lrwxrwxrwx 1 root root 33 Jan 26 16:33 /usr/bin/python3.6-config -> x86_64-linux-gnu-python3.6-config
-rwxr-xr-x 2 root root 4526456 Jan 26 16:33 /usr/bin/python3.6m
lrwxrwxrwx 1 root root 34 Jan 26 16:33 /usr/bin/python3.6m-config -> x86_64-linux-gnu-python3.6m-config
-rwxr-xr-x 1 root root 5134032 May 5 05:01 /usr/bin/python3.8
lrwxrwxrwx 1 root root 33 May 5 05:01 /usr/bin/python3.8-config -> x86_64-linux-gnu-python3.8-config
lrwxrwxrwx 1 root root 16 Oct 25 2018 /usr/bin/python3-config -> python3.6-config
-rwxr-xr-x 1 root root 384 Apr 11 2018 /usr/bin/python3-futurize
lrwxrwxrwx 1 root root 10 Mar 9 16:10 /usr/bin/python3m -> python3.6m
lrwxrwxrwx 1 root root 17 Oct 25 2018 /usr/bin/python3m-config -> python3.6m-config
-rwxr-xr-x 1 root root 388 Apr 11 2018 /usr/bin/python3-pasteurize
lrwxrwxrwx 1 root root 16 Apr 16 2018 /usr/bin/python-config -> python2.7-config

At this point:

taha@rosetta:~
$ python3.8 --version
Python 3.8.10
$ python3 --version
Python 3.6.9
$ python --version
Python 2.7.17

Question is, then, should I symlink python3 to 3.8? Or would that break things? For now, let's not.

• https://tech.serhatteker.com/post/2019-12/upgrade-python38-on-ubuntu/
• https://tech.serhatteker.com/post/2019-12/how-to-install-python38-on-ubuntu/
• https://unix.stackexchange.com/questions/410579/change-the-python3-default-version-in-ubuntu/410580
• https://askubuntu.com/questions/1197683/how-do-i-install-python-3-8-in-lubuntu-18-04

mps-youtube

I have removed the patch from asks2, but mpsyt still crashes immediately after finishing playing a video. This appears to be a different issue than the one the patch was intended to fix.

Why we install pip packages as user, not root

I have moved away from installing pip as the system user (root) because that lead to really hard-to-overcome clashes between pip packages and apt packages, such as pycairo.

Attempting uninstall: pycairo
Found existing installation: pycairo 1.16.2
stderr: ERROR: Cannot uninstall 'pycairo'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall

What if we uninstall python3-cairo? This was explicitly requested by the screenkey play, but is a dependency of multiple other projects:

(master) $ sudo apt remove python3-cairo 
Reading package lists... Done
Building dependency tree 
Reading state information... Done
The following packages were automatically installed and are no longer required:
 fortune-mod fortunes-min gedit-common gir1.2-ayatanaappindicator3-0.1 gir1.2-gexiv2-0.10
 gir1.2-gtksource-3.0 libayatana-appindicator3-1 libayatana-indicator3-7 libgexiv2-2 libgspell-1-1
 libgspell-1-common libgtksourceview-3.0-1 libgtksourceview-3.0-common librecode0 python3-bs4
 python3-html5lib python3-lxml python3-olefile python3-pil python3-webencodings
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED
 gedit meld python3-cairo python3-cairo-dev python3-gi-cairo variety
0 to upgrade, 0 to newly install, 6 to remove and 21 not to upgrade.
After this operation, 7,605 kB disk space will be freed.
Do you want to continue? [Y/n] 

No, that would break too many other packages.

Options?

• simply power on by adding --ignore-installed to pip extra args, but that risks putting the entire system in an unmaintainable state
• would it help if we changed pip_user from root to {{ ansible_env.USER }}? (don't really need it to be root...)
• if that does not help, then what?
• install more pip packages in their own virtualenvs, instead of like now?
• install python from source (a so-called private python installation), would that help?

Good commentary on this subject:

• https://stackoverflow.com/questions/49932759/pip-10-and-apt-how-to-avoid-cannot-uninstall-x-errors-for-distutils-packages
• https://stackoverflow.com/questions/53807511/pip-cannot-uninstall-package-it-is-a-distutils-installed-project/53807588#53807588

While working on this issue, I deleted /usr/lib/python3 which broke the entire system. Had to reinstall the OS from scratch...

Places where Python places its packages

And ditto for Python 2:

/usr/lib/python3/
/usr/lib/python3.6/
/usr/local/lib/python3.6/dist-packages/
~/.local/lib/python3.6/site-packages/