Which standard library modules are required to run the Python 3.5 interpreter?

Question

Here's a CPython program that tries to initialize the interpreter with an empty sys.path:

#include <Python.h>
int main(int argc, char** argv)
{
 wchar_t* program = NULL;
 wchar_t* sys_path = NULL;
 Py_NoSiteFlag = 1;
 program = Py_DecodeLocale(argv[0], NULL);
 Py_SetProgramName(program);
 sys_path = Py_DecodeLocale("", NULL);
 Py_SetPath(sys_path);
 Py_Initialize();
 PyMem_RawFree(program); 
 PyMem_RawFree(sys_path);
 Py_Finalize();
}

Executing the program above raises the following error:

Fatal Python error: Py_Initialize: Unable to get the locale encoding
ImportError: No module named 'encodings'
Current thread 0x00007ffff7fc6700 (most recent call first):
Signal: SIGABRT (Aborted)

So which of the packages and modules in the Python 3.5 standard library, besides the encodings package, are absolutely required to run the Python 3.5 interpreter? This information seems to me absent from the documentation.

• Charles Duffy: You might test by running the interpreter and then looking at the dictionary of imported modules to see what it contains.

Answer 1

These are packages/modules that are used during interpreter start-up (as, @Charles Duffy noted in a comment, by looking in sys.modules).

The result depends on whether you have site enabled or not (your Py_NoSiteFlag = 1; implies this isn't the case but anyway, I'll give both options :-)).

site drags a couple of additional modules with it when you use it like _sitebuiltins and stat, in total you could run Python using only the following:

abc.py encodings os.py _sitebuiltins.py sysconfig.py
codecs.py genericpath.py posixpath.py site.py _collections_abc.py 
io.py stat.py _weakrefset.py

with site disabled, you're stripped down to the following 6:

abc.py codecs.py encodings io.py os.py _weakrefset.py

when invoked through C with Py_Initialize() (or through Windows based on your comment) I'm guessing os.py might not be actually needed.

• bzrr: What do you mean by build? I was able to run the interpreter with no more than the encodings package, and the _weakrefset, abc, codecs, and io modules in sys.path.
• bzrr: There's not much it can do with those, but still.
• Dimitris Fasarakis Hilliard: Ah, I thought you meant about the whole process (i.e getting the source for CPython, building it and then also running it). Yes, for simply running the interpreter you need a small subset of the modules/packages I listed. Since you have found the required modules I guess you've answered your own question :-) @Jovito
• bzrr: Maybe it would be better if you removed the part about building the Python interpreter since that's not what the question is about, or at least move the part about actually running it to the top of your answer :)
• bzrr: Not on Windows, no.

Answer 2

If you run the interpreter as Charles Duffy suggests in his comment, you'll load packages like readline. It has been a decade since I did this but IIRC you don't need that module if you use python as an extension to your C program, as there is no commandline interaction. The same might hold for other modules.

The quickest way to determine what is really needed with only a slight chance of getting in too much is by putting all of lib/python3.5 where your program can find it, and in the program print out sys.modules, that will give you a list of what your program actually loaded, not what the interpreter might need to start up. After that remove everything not on that list.

Answer 3

Here's another approach - asking the Python interpreter what modules are loaded:

$ python3.5 -v -S -c '' |& grep SourceFileLoader | sort 
import 'abc' # <_frozen_importlib_external.SourceFileLoader object at 0x7f4b1d12e860>
import '_bootlocale' # <_frozen_importlib_external.SourceFileLoader object at 0x7f4b1d1367b8>
import 'codecs' # <_frozen_importlib_external.SourceFileLoader object at 0x7f4b1d187fd0>
import 'encodings.aliases' # <_frozen_importlib_external.SourceFileLoader object at 0x7f4b1d11eac8>
import 'encodings' # <_frozen_importlib_external.SourceFileLoader object at 0x7f4b1d187be0>
import 'encodings.latin_1' # <_frozen_importlib_external.SourceFileLoader object at 0x7f4b1d12e3c8>
import 'encodings.utf_8' # <_frozen_importlib_external.SourceFileLoader object at 0x7f4b1d12c898>
import 'io' # <_frozen_importlib_external.SourceFileLoader object at 0x7f4b1d12e5f8>
import '_weakrefset' # <_frozen_importlib_external.SourceFileLoader object at 0x7f4b1d135080>

_bootlocale is not required but recommended. It's used for initializing the best encoding for sys.stdin/sys.stdout/sys.stderr. See https://hg.python.org/cpython/rev/fbbf8b160e8d

sys.modules can lie as it's mutable.