homepage

Le 09/12/2011 22:12, Stefan Krah a écrit :
> The bottleneck in _decimal is (res is ascii):
>
> PyUnicode_FromString(res);
>
> PyUnicode_DecodeASCII(res) has the same performance.
>
>
> With this function ...
>
> static PyObject*
> unicode_fromascii(const char* s, Py_ssize_t size)
> {
> PyObject *res;
> res = PyUnicode_New(size, 127);
> if (!res)
> return NULL;
> memcpy(PyUnicode_1BYTE_DATA(res), s, size);
> return res;
> }
>
> ... I get the same performance as with Python 2.7 (5.85s)!
The problem is that unicode_fromascii() is unsafe: it doesn't check 
that the string is pure ASCII. That's why this function is private.
Because of the PEP 383, ASCII and UTF-8 decoders (PyUnicode_DecodeASCII 
and PyUnicode_FromString) have to first scan the input to check for 
errors, and then do a fast memcpy. The scanner of these two decoders is 
already optimized to process the input string word by word (word=the C 
long type), instead of byte by byte, using a bit mask.
--
You can write your own super fast ASCII decoder using two lines:
 res = PyUnicode_New(size, 127);
 memcpy(PyUnicode_1BYTE_DATA(res), s, size);
(this is exactly what unicode_fromascii does)
> I think it would be really beneficial for C-API users to have
> more ascii low level functions that don't do error checking and
> are simply as fast as possible.
It is really important to ensure that a ASCII string doesn't contain 
characters outside [U+0000; U+007F] because many operations on ASCII 
string are optimized (e.g. UTF-8 pointer is shared with the ASCII pointer).
I prefer to not expose such function or someone will use it without 
understanding exactly how dangerous it is.
Martin and other may disagree with me.
Do you know Murphy's Law? :-)
http://en.wikipedia.org/wiki/Murphy%27s_law