homepage

Steps to reproduce:
Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53) 
[GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import * 
>>> fields = [('a', c_short, 4), ('b', c_short, 4), ('c', c_long, 24)]
>>> class Foo(Structure):
... _fields_ = fields
... 
>>> Foo.a
<Field type=c_short, ofs=0:0, bits=4>
>>> Foo.b
<Field type=c_short, ofs=0:4, bits=4>
>>> Foo.c
<Field type=c_long, ofs=-2:8, bits=24> # Wrong!
More about my machine:
>>> sizeof(c_short)
2
>>> sizeof(c_long)
4
This particular example comes from a 32-bit Mac OS X Intel machine. The 
bug has been reproduced on Linux as well, but could not be reproduced on 
Windows XP.

Confirmed in HEAD for Python 2.6 and 3.0, on Linux.
Python 2.6b2+ (trunk:65708, Aug 16 2008, 15:04:13) 
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
Python 3.0b2+ (py3k:65708, Aug 16 2008, 15:09:19) 
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
I was also able to simplify the test case. I get this issue just using
one c_short and one c_long, with nonstandard bit lengths. eg:
fields = [('a', c_short, 16), ('b', c_long, 16)]
(sizeof(c_short) == 2 and sizeof(c_long) == 4).
But it's somewhat sporadic under which conditions it happens and which
it doesn't.
One might imagine this was a simple calculation. But the _ctypes module
is so big (5000 lines of C); at an initial glance I can't find the code
responsible! Any hints? (Modules/_ctypes/ctypes.c presumably is where
this takes place).

Here is a patch, with test, that fixes this problem.

Updated patch.

Updated the unittest so that it works on Windows, too.

Make this a release blocker so hopefully someone will review it.

Looks fine to me, except for the comment in the test suite. Should
+ # MS compilers do NOT combine c_short and c_int into
+ # one field, gcc doesn't.
perhaps be
+ # MS compilers do NOT combine c_short and c_int into
+ # one field, gcc do.
?
Is using explicit tests for MSVC vs. GCC a good idea, btw? What about 
other compilers? Can the test be changed to accept either value?

Looks reasonable, though I'm no ctypes maven. Trivial little nit:
In the comment just before the start of CField_FromDesc it says,
in part:
 * Expects the size, index and offset for the current field in *psize and
 * *poffset, stores the total size so far in *psize, the offset for the next
Note that it identifies three values (size, index, offset) as stored
in two locations (*psize and *poffset). Seems like some sort of
mismatch to me.

Fredrik Lundh schrieb:
> Looks fine to me, except for the comment in the test suite. Should
> 
> + # MS compilers do NOT combine c_short and c_int into
> + # one field, gcc doesn't.
> 
> perhaps be
> 
> + # MS compilers do NOT combine c_short and c_int into
> + # one field, gcc do.
Sure. But isn't this correct (or better) english, instead?
 ^^^^
> Is using explicit tests for MSVC vs. GCC a good idea, btw? What about 
> other compilers? Can the test be changed to accept either value?
Well, MSVC and GCC are the only compilers that I use (and that are tested
on the buildbots, afaik). If a cygwin compiled Python, for example, fails
this test then of course the test must be changed.
Thanks.

Skip Montanaro schrieb:
> Looks reasonable, though I'm no ctypes maven. Trivial little nit:
> In the comment just before the start of CField_FromDesc it says,
> in part:
> 
> * Expects the size, index and offset for the current field in *psize and
> * *poffset, stores the total size so far in *psize, the offset for the next
> 
> Note that it identifies three values (size, index, offset) as stored
> in two locations (*psize and *poffset). Seems like some sort of
> mismatch to me.
Unfortunately this is not the only comment that is out of date in the
ctypes sources. I wanted to touch as little code as possible in this patch.
Thanks.

^^^^
"Do" should be "does", right. Not enough coffee today :)

test_mixed_4 fail on:
Python 2.6rc2+ (trunk:66617M, Sep 25 2008, 16:32:44) 
[GCC 3.4.5 (mingw special)] on win32
Type "help", "copyright", "credits" or "license" for more information.
sizeof(X) return 12.

Does the following patch fix the test failure with MingW?
<patch>
Index: cfield.c
===================================================================
--- cfield.c	(revision 66611)
+++ cfield.c	(working copy)
@@ -65,10 +65,10 @@
 	}
 	if (bitsize /* this is a bitfield request */
 	 && *pfield_size /* we have a bitfield open */
-#if defined(MS_WIN32) && !defined(__MINGW32__)
-	 && dict->size * 8 == *pfield_size /* MSVC */
+#if defined(MS_WIN32)
+	 && dict->size * 8 == *pfield_size /* Windows */
 #else
-	 && dict->size * 8 <= *pfield_size /* GCC, MINGW */
+	 && dict->size * 8 <= *pfield_size /* GCC */
 #endif
 	 && (*pbitofs + bitsize) <= *pfield_size) {
 		/* continue bit field */
<end patch>
Also, can you please post the output of the following code snippet?
<test script>
from ctypes import *
class X(Structure):
 _fields_ = [("a", c_short, 4),
 ("b", c_short, 4),
 ("c", c_int, 24),
 ("d", c_short, 4),
 ("e", c_short, 4),
 ("f", c_int, 24)]
print X.a
print X.b
print X.c
print X.d
print X.e
print X.f
<end test script>

Yes this extra define was the problem.
Instead hint /* Windows */ what about /* MSVC, GCC(with -mms-bitfields) */ ?
The option -mms-bitfields is available for GCC compiler on mingw and
cygwin targets.
About test_bitfields.py:
- comment in test_mixed_4: if GCC compiler has option -mms-bitfields it
will produce code compatible with MSVC. May be comment has to include
this information. 
- may be method test_mixed_1 need similar comment as test_mixed_4, but
even without comment is fine with me.

Ok, here is an additional patch bitfields-mingw.patch.
It fixes the problem reported by rpetrov, and updates the comments.
Please review again ;-).
The previous patch bitfields-3.patch has already been applied.

Flag start only with one minus: -mms-bitfields
Fine with me.

Committed as SVN rev 66683 (trunk), 66684 (py3k), and 66685
(release25-maint).