homepage

The second parameter of the PyUnicode_AsWideCharString() function
 wchar_t* PyUnicode_AsWideCharString(PyObject *unicode, Py_ssize_t *size)
is a pointer to Py_ssize_t. The size of created wchar_t array is saved on this pointer if it is not NULL. If NULL is passed as the second argument, the only way to determine the size of the wchar_t string is using wcslen(). But if the string contains the null characters, it looks truncated for wcslen() and other C API functions.
Reliable code should always pass the non-NULL second argument and check that wcslen() is equal to the returned string size. See for example the code in Modules/_io/winconsoleio.c. Passing NULL as the second argument is unsafe. But most code doesn't do such check (see all other usages of PyUnicode_AsWideCharString(..., NULL)). And this check complicates the callers code.
I propose to make the check for null characters inside of PyUnicode_AsWideCharString() if NULL is passes as the second argument. This will fix all unsafe usages of PyUnicode_AsWideCharString() and allow to simplify the reliable code.
This issue fixes the part of issue13617.

This change needs changing one ctypes test, and can break third-party tests or even a code. That is why it is targeted only for 3.7. I'm going to backport the change as a private function for using in CPython internally since this can fix vulnerabilities.

Could anyone please make a review of this PR? Especially the documentation part. This PR is a part of a set of PRs that fix potential vulnerabilities (issue13617, issue30730, and yet few issues planned).

New changeset e613e6add5f07ff6aad5802924596b631b707d2a by Serhiy Storchaka in branch 'master':
bpo-30708: Check for null characters in PyUnicode_AsWideCharString(). (#2285)
https://github.com/python/cpython/commit/e613e6add5f07ff6aad5802924596b631b707d2a

New changeset 0edffa3073b551ffeca34952529e7b292f1bd350 by Serhiy Storchaka in branch '3.6':
[3.6] bpo-30708: Check for null characters in PyUnicode_AsWideCharString(). (GH-2285) (#2443)
https://github.com/python/cpython/commit/0edffa3073b551ffeca34952529e7b292f1bd350

New changeset 94b169fe48bc7ea76b926823885d1b12c2c381fa by Serhiy Storchaka in branch '3.5':
[3.5] bpo-30708: Add private C API function _PyUnicode_AsWideCharString(). (GH-2285) (GH-2443) (#2448)
https://github.com/python/cpython/commit/94b169fe48bc7ea76b926823885d1b12c2c381fa

Wrong commit message in 3.6. Should be the same as in 3.5.
This functionality was backported as a private function _PyUnicode_AsWideCharString().