homepage

Consider the following:
import ctypes
class struct1( ctypes.Structure ):
 _pack_ = 1
 _fields_ = [
 ( "first", ctypes.c_uint8, 1 ),
 ( "second", ctypes.c_uint8, 1 ),
 ( "third", ctypes.c_uint8, 1 ),
 ( "fourth", ctypes.c_uint8, 1 ),
 ( "fifth", ctypes.c_uint8, 1 ),
 ( "pad", ctypes.c_uint16, 11 ),
 ]
 
s1 = struct1()
print ctypes.sizeof( s1 )
class struct2( ctypes.Structure ):
 _pack_ = 1
 _fields_ = [
 ( "first", ctypes.c_uint16, 1 ),
 ( "second", ctypes.c_uint16, 1 ),
 ( "third", ctypes.c_uint16, 1 ),
 ( "fourth", ctypes.c_uint16, 1 ),
 ( "fifth", ctypes.c_uint16, 1 ),
 ( "pad", ctypes.c_uint16, 11 ),
 ]
 
s2 = struct2()
print ctypes.sizeof( s2 )
The output is:
3
2
I'm generating python code from real c code. The compiler I'm using for the real c code packs both of these structures into two bytes. I need a way to make the first example work in python like the compiler without having to modify the source code.
Is this possible?

The output is: "3 2" on windows. It is "2 2" on the linux 64bit I tried.
This is consistent with what the usual compilers do on these platforms: MSVC on windows, and gcc on linux.
The specification of the C language specifies that """If an adjacent bitfield will not fit into the remainder of the unit, the implementation defines whether bitfields are allowed to span units or whether another unit is allocated for the second bitfield"""
Are you using gcc on windows? In this case I suggest to always use the largest type to declare the bitfields.

What compilers were used to build your Python distro and the native structure?
I found out in _ctypes/cfield.c (lns. 76-95):
 if (bitsize /* this is a bitfield request */
 && *pfield_size /* we have a bitfield open */
#ifdef MS_WIN32
 /* MSVC, GCC with -mms-bitfields */
 && dict->size * 8 == *pfield_size
#else
 /* GCC */
 && dict->size * 8 <= *pfield_size
#endif
 && (*pbitofs + bitsize) <= *pfield_size) {
 /* continue bit field */
 fieldtype = CONT_BITFIELD;
#ifndef MS_WIN32
 } else if (bitsize /* this is a bitfield request */
 && *pfield_size /* we have a bitfield open */
 && dict->size * 8 >= *pfield_size
 && (*pbitofs + bitsize) <= dict->size * 8) {
 /* expand bit field */
 fieldtype = EXPAND_BITFIELD;
#endif
So the allocation of the extra byte for the structure seems to depend on Python's compiler. To make sure, I compiled a native structure using MSVC:
#pragma pack(1)
typedef struct _struct1
{
 UINT8 first : 1;
 UINT8 second : 1;
 UINT8 third : 1;
 UINT8 fourth : 1;
 UINT8 fifth : 1;
 UINT16 pad : 11;
} struct1;
And I got the same value (sizeof == 3).

So, knowing there's a potential cross platform inconsistency here, is there
a proposed way to deal with this that doesn't involve modifying the real c
code I'm interfacing with? That's not always an option.
On Mon, Apr 25, 2011 at 2:49 PM, Santoso Wijaya <report@bugs.python.org>wrote:
>
> Santoso Wijaya <santoso.wijaya@gmail.com> added the comment:
>
> What compilers were used to build your Python distro and the native
> structure?
>
> I found out in _ctypes/cfield.c (lns. 76-95):
>
> if (bitsize /* this is a bitfield request */
> && *pfield_size /* we have a bitfield open */
> #ifdef MS_WIN32
> /* MSVC, GCC with -mms-bitfields */
> && dict->size * 8 == *pfield_size
> #else
> /* GCC */
> && dict->size * 8 <= *pfield_size
> #endif
> && (*pbitofs + bitsize) <= *pfield_size) {
> /* continue bit field */
> fieldtype = CONT_BITFIELD;
> #ifndef MS_WIN32
> } else if (bitsize /* this is a bitfield request */
> && *pfield_size /* we have a bitfield open */
> && dict->size * 8 >= *pfield_size
> && (*pbitofs + bitsize) <= dict->size * 8) {
> /* expand bit field */
> fieldtype = EXPAND_BITFIELD;
> #endif
>
> So the allocation of the extra byte for the structure seems to depend on
> Python's compiler. To make sure, I compiled a native structure using MSVC:
>
> #pragma pack(1)
> typedef struct _struct1
> {
> UINT8 first : 1;
> UINT8 second : 1;
> UINT8 third : 1;
> UINT8 fourth : 1;
> UINT8 fifth : 1;
> UINT16 pad : 11;
> } struct1;
>
> And I got the same value (sizeof == 3).
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue11920>
> _______________________________________
>

Opened http://bugs.python.org/issue12528 to address this.

Hi Steve,
There is currently no way to deal with this situation. Vlad has opened issue12528 with a proposal to make the allocation strategy configurable from the 'ctypes' API. Please follow that issue if you are still interested. I am closing this issue.