[Python-Dev] ctypes: Configurable bitfield allocation strategy

Sat Jun 25 21:09:24 CEST 2011

On 6/25/2011 12:33 PM, Vlad Riscutia wrote:
> I recently started looking at some ctypes issues. I dug a bit into
> http://bugs.python.org/issue6069 and then I found
> http://bugs.python.org/issue11920. They both boil down to the fact that
> bitfield allocation is up to the compiler, which is different in GCC and
> MSVC. Currently we have hard-coded allocation strategy based on paltform
> in cfields.c:
>>> if (bitsize /* this is a bitfield request */
>>> && *pfield_size /* we have a bitfield open */
>> #ifdef MS_WIN32
>> /* MSVC, GCC with -mms-bitfields */
>> && dict->size * 8 == *pfield_size
>> #else
>> /* GCC */
>> && dict->size * 8<= *pfield_size
>> #endif
>> && (*pbitofs + bitsize)<= *pfield_size) {
>> /* continue bit field */
>> fieldtype = CONT_BITFIELD;
>> #ifndef MS_WIN32
>> } else if (bitsize /* this is a bitfield request */
>> && *pfield_size /* we have a bitfield open */
>> && dict->size * 8>= *pfield_size
>> && (*pbitofs + bitsize)<= dict->size * 8) {
>> /* expand bit field */
>> fieldtype = EXPAND_BITFIELD;
>> #endif
>> So when creating a bitfield structure, it's size can be different on
> Linux vs Windows.
>> class MyStructure(ctypes.BigEndianStructure):
> _pack_ = 1 # aligned to 8 bits, not ctypes default of 32
> _fields_ = [
> ("Data0", ctypes.c_uint32, 32),
> ("Data1", ctypes.c_uint8, 3),
> ("Data2", ctypes.c_uint16, 12),
> ]
>> sizeof for above structure is 6 on GCC build and 7 on MSVC build. This
> leads to some confusion and issues, because we can't always interop
> correctly with code compiled with different compiler than the one Python
> is compiled with on the platform.

Just curious, are you saying that this is the 'cause' of the two bug 
reports, or 'just' something you discovered while investigating them?
 > Short term solution is to add a warning in the documentation about this.
For 2.7/3.2, yes.
 > Longer term though, I think it
> would be better to add a property on the Structure class for
> configurable allocation strategy, for example Native (default), GCC,
> MSVC and when allocating the bitfield, use given strategy. Native would
> behave similar to what happens now, but we would also allow GCC-style
> allocation on Windows for example and the ability to extend this if we
> ever run into similar issues with other compilers. This shouldn't be too
> difficult to implement, will be backwards compatible and it would
> improve interop. I would like to hear some opinions on this.

If this would allow the MSVC-compilied Python to better access dlls 
compiled with gcc (cygwin) on Windows, definitely -- in 3.3.
If the new feature is (currently) only useful on Windows, doc should say so.
-- 
Terry Jan Reedy