This issue tracker has been migrated to GitHub ,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2020年05月11日 22:12 by vstinner, last changed 2022年04月11日 14:59 by admin. This issue is now closed.
| Pull Requests | |||
|---|---|---|---|
| URL | Status | Linked | Edit |
| PR 23683 | closed | shihai1991, 2020年12月07日 18:11 | |
| PR 23699 | closed | corona10, 2020年12月08日 16:51 | |
| PR 23763 | merged | vstinner, 2020年12月14日 11:59 | |
| Messages (17) | |||
|---|---|---|---|
| msg368665 - (view) | Author: STINNER Victor (vstinner) * (Python committer) | Date: 2020年05月11日 22:12 | |
When a C extension module is created with PyModuleDef_Init(), it becomes possible to create more than one instance of the module. It would take significant effort to modify some extensions to make their code fully ready to have two isolated module. For example, the atexit module calls _Py_PyAtExit() to register itself into the PyInterpreterState. If the module is created more than once, the most recently created module wins, and calls registered on other atexit instances are ignore: see bpo-40288. One simple option would be to simply disallow loading the module more than once per interpreter. Also, some extensions are not fully compatible with subinterpreters. It may be interesting to allow to load them in a subinterpreter if it's not already loaded in another interpreter, like another subinterpreter or the main interpreter. It would be only load it once per Python *process*. For example, numpy would be a good candidate for such option. I'm not sure fow a module should announced in its definition that it should not be loaded more than once. |
|||
| msg368994 - (view) | Author: Terry J. Reedy (terry.reedy) * (Python committer) | Date: 2020年05月16日 01:10 | |
Title clarified. Leaving subinterpreters aside, only one instance, AFAIK, is true for stdlib and python modules unless imported with different names, as can happen with main module (which is a nuisance if not a bug). So only once per interpreter seems like a bugfix. Only once per process, if once per interpreter otherwise becomes normal, seems like an enhancement, even if a necessary option. |
|||
| msg376524 - (view) | Author: mohamed koubaa (koubaa) * | Date: 2020年09月07日 19:07 | |
What about a new PyModuleDef_Slot function?
```
static int my_can_create(/*need any arg??, InterpreterState, etc?*/) {
if (test_global_condition()) {
return -1; //Don't allow creation
}
return 0; //Allow creation
};
static PyModuleDef_Slot signal_slots[] = {
{Py_mod_exec, my_exec},
{Py_mod_can_create, my_can_create},
{0,0}
};
```
|
|||
| msg376674 - (view) | Author: Dong-hee Na (corona10) * (Python committer) | Date: 2020年09月10日 08:58 | |
One of my opinions is that Since module object supports detecting error during Py_mod_exec, The only thing we have to do is return -1 when the module has already existed. we should define new exception type and don't have to define new slots. The pros of this method is that we don't have to modify module object and no needs to write the new PEP but the cons of this method is that we should promise the standard exception when try to create multiple instances which is not allowed. ref: https://github.com/python/cpython/blob/788b79fa7b6184221e68d4f1a3fbe0b3270693f6/Objects/moduleobject.c#L399 On the other side, defining a Py_mod_exec_once that supports execution for just once can be a way. Although the usage is little, it will be fine because the use case will exist. Please point out what I missed :) |
|||
| msg376676 - (view) | Author: STINNER Victor (vstinner) * (Python committer) | Date: 2020年09月10日 10:00 | |
One option is to get the behavior before multi-phase initialization. We store extensions in a list. Once it's load, it cannot be unloaded before we exit Python. See _PyState_AddModule() and _PyState_AddModule(). Calling PyInit_xxx() the second time would simply return the existing module object. When we exit Python, the clear and/or free function of the module is called. |
|||
| msg376685 - (view) | Author: mohamed koubaa (koubaa) * | Date: 2020年09月10日 13:48 | |
Something like this?
```
static PyObject *def;
PyMODINIT_FUNC
PyInit_mymod(void)
{
if (def == NULL) {
def = PyModuleDef_Init(&mymod);
}
return def;
}
```
Then add a flag to PyModuleDef to indicate it is already exec?
|
|||
| msg382665 - (view) | Author: Hai Shi (shihai1991) * (Python triager) | Date: 2020年12月07日 18:08 | |
>On the other side, defining a Py_mod_exec_once that supports execution > >for just once can be a way. >Although the usage is little, it will be fine because the use case will >exist. IMHO, `Py_mod_exec_once` is more like a slot control flag. MAYBE we need add a module flag in `PyModuleDef`. |
|||
| msg382668 - (view) | Author: Hai Shi (shihai1991) * (Python triager) | Date: 2020年12月07日 18:14 | |
> MAYBE we need add a module flag in `PyModuleDef`. I created a demo in PR 23683. |
|||
| msg382737 - (view) | Author: Petr Viktorin (petr.viktorin) * (Python committer) | Date: 2020年12月08日 13:52 | |
Is it really necessary to add a slot/flag for this?
It can be done in about 10 lines as below, without complications like a global (or per-interpreter) registry of singleton modules.
Are there many modules that need to be per-interpreter singletons, but may be loaded in multiple interpreters? In my experience, it is really hard to ensure modules behave that way; if (if!) we need to add a dedicated API for this, I'd go for once-per-process.
static int loaded = 0;
static int
exec_module(PyObject* module)
{
if (loaded) {
PyErr_SetString(PyExc_ImportError,
"cannot load module more than once per process");
return -1;
}
loaded = 1;
// ... rest of initialization
}
|
|||
| msg382742 - (view) | Author: STINNER Victor (vstinner) * (Python committer) | Date: 2020年12月08日 15:15 | |
> static int loaded = 0; I would like to limit an extension to once instance *per interpreter*. See the atexit module for an example. |
|||
| msg382745 - (view) | Author: Petr Viktorin (petr.viktorin) * (Python committer) | Date: 2020年12月08日 15:46 | |
Are there any other examples? In my view, atexit is very special, and very closely tied to interpreter. I don't think it's good to design general API for one module. |
|||
| msg382747 - (view) | Author: STINNER Victor (vstinner) * (Python committer) | Date: 2020年12月08日 16:27 | |
I'm not 100% sure that preventing to create multiple module instances is needed. For the atexit module, another solution is to move the "module state" into the interpreter, as it has been done for other modules like _warnings. |
|||
| msg382748 - (view) | Author: Dong-hee Na (corona10) * (Python committer) | Date: 2020年12月08日 16:31 | |
> another solution is to move the "module state" into the interpreter, I am +1 on this solution if this module is a very special case. |
|||
| msg382757 - (view) | Author: Hai Shi (shihai1991) * (Python triager) | Date: 2020年12月08日 18:11 | |
> another solution is to move the "module state" into the interpreter, OK, I am agree victor's solution too. It's a more simpler way. |
|||
| msg382983 - (view) | Author: STINNER Victor (vstinner) * (Python committer) | Date: 2020年12月14日 12:03 | |
I created bpo-42639 and PR 23763 to move the atexit module to PyInterpreterState and so allow to have more than one atexit instance. |
|||
| msg383059 - (view) | Author: Petr Viktorin (petr.viktorin) * (Python committer) | Date: 2020年12月15日 14:07 | |
Thanks! Indeed, that's an even better solution than I had in mind. It follows PEP 630 quite nicely: https://www.python.org/dev/peps/pep-0630/#managing-global-state I will close this issue and PRs. I don't agree with adding a general API for disallowing multiple modules, but do let me know if you see a need for it again. |
|||
| msg383062 - (view) | Author: STINNER Victor (vstinner) * (Python committer) | Date: 2020年12月15日 14:43 | |
> Thanks! Indeed, that's an even better solution than I had in mind. > It follows PEP 630 quite nicely: https://www.python.org/dev/peps/pep-0630/#managing-global-state The atexit module was my main motivation for this issue. So I'm fine with closing it. |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022年04月11日 14:59:30 | admin | set | github: 84780 |
| 2020年12月15日 14:43:35 | vstinner | set | messages: + msg383062 |
| 2020年12月15日 14:07:33 | petr.viktorin | set | status: open -> closed resolution: not a bug messages: + msg383059 stage: patch review -> resolved |
| 2020年12月14日 12:03:34 | vstinner | set | messages: + msg382983 |
| 2020年12月14日 11:59:09 | vstinner | set | pull_requests: + pull_request22619 |
| 2020年12月08日 18:11:46 | shihai1991 | set | messages: + msg382757 |
| 2020年12月08日 16:51:05 | corona10 | set | pull_requests: + pull_request22566 |
| 2020年12月08日 16:31:31 | corona10 | set | messages: + msg382748 |
| 2020年12月08日 16:27:02 | vstinner | set | messages: + msg382747 |
| 2020年12月08日 15:46:31 | petr.viktorin | set | messages: + msg382745 |
| 2020年12月08日 15:15:55 | vstinner | set | messages: + msg382742 |
| 2020年12月08日 13:52:42 | petr.viktorin | set | nosy:
+ petr.viktorin messages: + msg382737 |
| 2020年12月07日 18:14:19 | shihai1991 | set | messages: + msg382668 |
| 2020年12月07日 18:11:49 | shihai1991 | set | keywords:
+ patch stage: test needed -> patch review pull_requests: + pull_request22547 |
| 2020年12月07日 18:08:38 | shihai1991 | set | messages:
+ msg382665 versions: + Python 3.10, - Python 3.9 |
| 2020年11月19日 11:48:30 | shihai1991 | set | nosy:
+ shihai1991 |
| 2020年09月10日 13:48:12 | koubaa | set | messages: + msg376685 |
| 2020年09月10日 10:00:12 | vstinner | set | messages: + msg376676 |
| 2020年09月10日 08:58:38 | corona10 | set | messages: + msg376674 |
| 2020年09月07日 19:07:03 | koubaa | set | nosy:
+ koubaa messages: + msg376524 |
| 2020年05月18日 14:03:05 | vstinner | set | components: + Subinterpreters |
| 2020年05月16日 01:10:16 | terry.reedy | set | nosy:
+ terry.reedy title: Add an option to disallow creating more than one instance of a module -> Add option to disallow > 1 instance of an extension module messages: + msg368994 stage: test needed |
| 2020年05月11日 22:12:29 | vstinner | create | |