How do I get Python 3 to work again on my Raspberry Pi?

Question

I am running Raspbian on a Pi 3 that has one job: it runs a clock on a Sense Hat, from Python 3 source code. Recently I found I was unable to run my clock program, which generated an error stack that ended this way:

 File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
 File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
 File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
 File "<frozen importlib._bootstrap>", line 1129, in _exec
 File "<frozen importlib._bootstrap>", line 1467, in exec_module
 File "<frozen importlib._bootstrap>", line 1570, in get_code
 File "<frozen importlib._bootstrap>", line 656, in _compile_bytecode
EOFError: marshal data too short

I found a number of possible solutions, none of which worked. Any attempt to uninstall/reinstall modules with pip3 generated the same error stack. I read instructions to delete all the .pyc files in the Python3 tree and recompile them, but after that any attemtp to load a module just gave me

"defining __gutstate__ cannot be picKled")

and I was unable to recreate the .pyc files.

Essentially everything I did that involved touching Python files, whether through apt-get or pip3, failed on some Python error, either the "marshal data too short" or the "cannot be picKled". I can't even remove or purge Python3 itself - I get Python errors that won't let me reinstall Python.

Is there a solution for this? My clock actually runs under Python 2.7, but I'd like to be able to run a supported version of Python if possible. I could reinstall Raspbian but there's no guarantee it wouldn't fail the same way and put me right back where I am.

• Jamie Lindsey: I would suggest manually deleting python and reinstall it built from source. It would take a long time on an rpi but I have done it before (took around 8 hours). Failing that just reinstall raspian like you said, the probl ms would probably not exist in a fresh install of raspian and that way you can update to python3.9 or whatever is in the raspian repos.
• Dmitry Grigoryev: @JamieLindsey Reinstalling Raspbian takes much less than 8 hours and has higher chances of success.

Answer 1

I found the suggestion to delete pycache in another thread and deleted all the pycache directories in the /usr/lib/python3 and /usr/lib/python3.4 directories. All that happened was that python3 stopped working completely and I had to restore the system from backup. I also tried apt-get purge python3, and numerous components failed to uninstall with exactly the same "marshal data too short" error that happened when I tried to load a library. In fact, the whole stacktrace was identical:

Traceback (most recent call last):
 File "/usr/bin/py3clean", line 24, in <module>
 import logging
 File "/usr/lib/python3.4/logging/__init__.py", line 26, in <module>
 import sys, os, time, io, traceback, warnings, weakref, collections
 File "/usr/lib/python3.4/traceback.py", line 3, in <module>
 import linecache
 File "/usr/lib/python3.4/linecache.py", line 10, in <module>
 import tokenize
 File "/usr/lib/python3.4/tokenize.py", line 29, in <module>
 import collections
 File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
 File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
 File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
 File "<frozen importlib._bootstrap>", line 1129, in _exec
 File "<frozen importlib._bootstrap>", line 1467, in exec_module
 File "<frozen importlib._bootstrap>", line 1570, in get_code
 File "<frozen importlib._bootstrap>", line 656, in _compile_bytecode
EOFError: marshal data too short

• Ingo: This is not an answer to your question. If you have additional information then edit your question and add it there. Please take the short Tour and visit the Help Center to get an idea how things work here.
• Dmitry Grigoryev: @Ingo It is an answer: the OP tried to regenerate the pycache, the situation got worse, and they ultimately restored the system from a backup. Not a great solution perhaps, but I actually would have suggested the same thing.
• Ingo: @DmitryGrigoryev Really... hard to find the subordinate clause within all the error messages :-) I have upvoted now.
• Seamus: @DmitryGrigoryev: If this is an answer, it may be the best disguised answer in the history of SE :) At least your comment will alert future readers that this is the answer - even if it doesn't look like an answer.

Answer 2

You should delete all the __pycache__ directories in any python library directories that are part of your program, including your program directory. So if you import requests then delete the __pycache__ for in the main requests lib folder (should be something like /usr/lib/pythonX.X/site-packages/requests)

It would seem the .pyc compiled files have been automatically updated with some corrupt data.

• mobilityguy: I've seen this solution suggested before, and tried deleting all the pycache directories in the /usr/lib/python3 and /usr/lib/python3.4 directory trees. After that, Python3 didn't even start, and anything I did to try to fix it simply crashed out. The only way to get Python3 back was to restore the system from backup.