On Mon, Jan 25, 2021 at 7:51 AM Ivan Pozdeev via Python-Dev <[email protected]
<mailto:[email protected]>> wrote:
Just _names_? There's a recurring error case when a 3rd-party module
overrides a standard one if it happens to have the same name. If you
filter such a module out, you're shooting yourself in the foot...
Would another use case be to support issuing a warning if a third-party module is imported whose name matches a standard one? A related
use case would be to build on this and define a function that accepts an already imported module and return whether it is from the
standard library. Unlike, the module_names attribute, this function would reflect the reality of the underlying module, and so not have
false positives as with doing a name check alone.
—Chris
On 25.01.2021 16:03, Victor Stinner wrote:
> Hi,
>
> I just added a new sys.module_names attribute, list (technically a
> frozenset) of all stdlib module names:
> https://bugs.python.org/issue42955 <https://bugs.python.org/issue42955>
>
> There are multiple use cases:
>
> * Group stdlib imports when reformatting a Python file,
> * Exclude stdlib imports when computing dependencies.
> * Exclude stdlib modules when listing extension modules on crash or
> fatal error, only list 3rd party extension (already implemented in
> master, see bpo-42923 ;-)).
> * Exclude stdlib modules when tracing the execution of a program using
> the trace module.
> * Detect typo and suggest a fix: ImportError("No module named maths.
> Did you mean 'math'?",) (test the nice friendly-traceback project!).
>
> Example:
>
>>>> 'asyncio' in sys.module_names
> True
>>>> 'numpy' in sys.module_names
> False
>
>>>> len(sys.module_names)
> 312
>>>> type(sys.module_names)
> <class 'frozenset'>
>
>>>> sorted(sys.module_names)[:10]
> ['__future__', '_abc', '_aix_support', '_ast', '_asyncio', '_bisect',
> '_blake2', '_bootsubprocess', '_bz2', '_codecs']
>>>> sorted(sys.module_names)[-10:]
> ['xml.dom', 'xml.etree', 'xml.parsers', 'xml.sax', 'xmlrpc', 'zipapp',
> 'zipfile', 'zipimport', 'zlib', 'zoneinfo']
>
> The list is opinionated and defined by its documentation:
>
> A frozenset of strings containing the names of standard library
> modules.
>
> It is the same on all platforms. Modules which are not available on
> some platforms and modules disabled at Python build are also listed.
> All module kinds are listed: pure Python, built-in, frozen and
> extension modules. Test modules are excluded.
>
> For packages, only sub-packages are listed, not sub-modules. For
> example, ``concurrent`` package and ``concurrent.futures``
> sub-package are listed, but not ``concurrent.futures.base``
> sub-module.
>
> See also the :attr:`sys.builtin_module_names` list.
>
> The design (especially, the fact of having the same list on all
> platforms) comes from the use cases list above. For example, running
> isort should produce the same output on any platform, and not depend
> if the Python stdlib was splitted into multiple packages on Linux
> (which is done by most popular Linux distributions).
>
> The list is generated by the Tools/scripts/generate_module_names.py
script:
>
https://github.com/python/cpython/blob/master/Tools/scripts/generate_module_names.py
<https://github.com/python/cpython/blob/master/Tools/scripts/generate_module_names.py>
>
> When you add a new module, you must run "make regen-module-names,
> otherwise a pre-commit check will fail on your PR ;-) The list of
> Windows extensions is currently hardcoded in the script (contributions
> are welcomed to discover them, since the list is short and evolves
> rarely, I didn't feel the need to spend time that on that).
>
> Currently (Python 3.10.0a4+), there are 312 names in sys.module_names,
> stored in Python/module_names.h:
> https://github.com/python/cpython/blob/master/Python/module_names.h
<https://github.com/python/cpython/blob/master/Python/module_names.h>
>
> It was decided to include "helper" modules like "_aix_support" which
> is used by sysconfig. But test modules like _testcapi are excluded to
> make the list shorter (it's rare to run the CPython test suite outside
> Python).
>
> There are 83 private modules, name starting with an underscore
> (exclude _abc but also __future__):
>
>>>> len([name for name in sys.module_names if not name.startswith('_')])
> 229
>
> This new attribute may help to define "what is the Python stdlib" ;-)
>
> Victor
--
Regards,
Ivan
_______________________________________________
Python-Dev mailing list -- [email protected]
<mailto:[email protected]>
To unsubscribe send an email to [email protected]
<mailto:[email protected]>
https://mail.python.org/mailman3/lists/python-dev.python.org/
<https://mail.python.org/mailman3/lists/python-dev.python.org/>
Message archived at
https://mail.python.org/archives/list/[email protected]/message/KCJDHKOKCN5343VVA3DC7RAGNUGWNKZY/
<https://mail.python.org/archives/list/[email protected]/message/KCJDHKOKCN5343VVA3DC7RAGNUGWNKZY/>
Code of Conduct: http://python.org/psf/codeofconduct/
<http://python.org/psf/codeofconduct/>