homepage

Found by Hartmut Ring:
#include <Python.h>
int main() {
 for (int i=0; i<2; i++) {
 Py_Initialize();
 PyRun_SimpleString("a = u'\xe4'.encode('Latin-1')");
 Py_Finalize();
 }
 return 0;
}
First try runs fine, second try gives:
Traceback (most recent call last):
 File "<string>", line 1, in ?
LookupError: no codec search functions registered:
can't find encoding

Logged In: YES 
user_id=38388
This is due to a flag used in codecs.c:
/* Flag used for lazy import of the standard encodings
package */
static int import_encodings_called = 0;
The solution is to reset this flag in Py_Finalize().

Logged In: YES 
user_id=21627
I think the various global variables maintained in codecs.c
(_PyCodec_SearchCache, import_encodings_called, ...) need to
become part of the interpreter state. Otherwise, a codec
registered in one interpreter will be found in another.
Also, it appears that import_encodings_called can be
eliminated. Instead, a NULL value of _PyCodec_SearchPath
could be used to indicate whether initialization has taken
place. 

Logged In: YES 
user_id=38388
Good suggestion. Do you have time to cook up a patch ?

Logged In: YES 
user_id=7887
As discussed in python-dev, I'll work on that.

Logged In: YES 
user_id=7887
Here is a proposed solution.
It has been tested with the standard regression tests, and
in an environment with multiple interpreters.

Logged In: YES 
user_id=6380
I can't review this beyond "it doesn't break the tests for
me"; I suggest to assign to MAL for review. One note: I got
a compiler warning
 implicit declaration of function `_PyCodecRegistry_Init'
to fix, add
 static int _PyCodecRegistry_Init(void); /* Forward */
somewhere to the top of codecs.c.

Logged In: YES 
user_id=38388
Same comments as Guido, plus:
* in the interpreter state struct you should add the new 
 fields to the end, not in the middle
* the finalization phase is moved to a later point in
 the process; this could cause problems when finalizing
 codecs -- I guess we'll just have to see whether we
 get bug reports related to this; perhaps it's better
 to ZAP the dicts before ZAPping builtins etc. ?!
Great work, Gustavo. Please check the patch in with 
the above changes.
Thanks.

Logged In: YES 
user_id=7887
No problems.
If you don't mind, I'd just to have a confirmation: are you
sure I should move the new fields to the end, and leave a
conditional member in the middle?

Logged In: YES 
user_id=38388
Oh, I see what you mean. In that case, it's OK to add 
them just before the optional parts.

Logged In: YES 
user_id=7887
Commited as:
Include/pystate.h: 2.23
Misc/NEWS: 1.699
Python/codecs.c: 2.21
Python/pystate.c: 2.24
Python/pythonrun.c: 2.182
Thanks for your support!