Struct variables overlap (Python C API)

Question

Here's a strange one for you: I have been following the framework of this tutorial, in an attempt to write a Python module that will return a struct containing the results of a C function.

The C struct is defined as such:

typedef struct my_struct
{
    // Initialize refcount and pointer to type objects (no ';' )
    PyObject_HEAD

    // Components of the structure go here.
    uint32_t width; // unsigned long
    uint32_t height; // unsigned long
    uint8_t numcomps; // unsigned char
    uint8_t bitspercomp; // unsigned char
    uint8_t bytespercomp; // unsighed char
    uint32_t total_data_len; // unsigned long
    int data;

} my_struct;

The issue is that the members of the struct seem to overlap in memory when accessed at the Python level! For input of the form:

my_struct.My_Struct(unsigned_long 1, unsigned_long 2, unsigned_char 3, unsigned_char 4, unsigned_char 5, unsigned_long 6, int 7)

the values stored in the returned locations are clearly overlapping. This can most easily be seen when they are represented as hex values:

In [1]: import my_struct as ms

In [2]: op = ms.My_Struct(1,2,3,4,5,6,7)

In [3]: myfi.width
Out[3]: 8589934593L  ***should be "1"***

In [4]: "%x"%op.width
Out[4]: '200000001'

In [5]: "%x"%op.height
Out[5]: '5040300000002'

In [6]: "%x"%op.numcomps
Out[6]: '3'

In [7]: "%x"%op.bitspercomp
Out[7]: '4'

In [8]: "%x"%op.bytespercomp
Out[8]: '5'

In [9]: "%x"%op.total_data_len
Out[9]: '700000006'

In [10]: "%x"%op.data
Out[10]: '7'

The allocation of the input arguments happens in an init function, which calls the method Py_Arg_ParseTupleAndKeywords():

static int
my_struct_init(my_struct *self, PyObject *args, PyObject *kwds)
{
    static char *kwlist[] = 
    {"width","height","numcomps","bitspercomp","bytespercomp","total_data_len","data", NULL};

    // Increase reference count before parsing
    Py_INCREF(self);

    if (! PyArg_ParseTupleAndKeywords(args, kwds, "kkbbbki"
                                      , kwlist, &self->width, &self->height,
                                      &self->numcomps, &self->bitspercomp, 
                                      &self->bytespercomp,
                                      &self->total_data_len, &self->data))
        return -1;

    return 0;
};

I have been unable to find similar problems, please help!

The rest of the script follows the format of the example. There is a "new" function that sets each struct member value to zero, as well as a destructor. Possibly relevant are the module member definitions:

static PyMemberDef my_struct_members[] = {
    /*member name, type, offset, access flags, documentation string*/
    {"width", T_ULONG, offsetof(my_struct, width), 0,
     "words and such."},
    {"height", T_ULONG, offsetof(my_struct, height), 0,
     "words and such."},
    {"numcomps", T_UBYTE, offsetof(my_struct, numcomps), 0,
     "words and such."},
    {"bitspercomp", T_UBYTE, offsetof(my_struct, bitspercomp), 0,
     "words and such."},
    {"bytespercomp", T_UBYTE, offsetof(my_struct, bytespercomp), 0,
     "words and such."},
    {"total_data_len", T_ULONG, offsetof(my_struct, total_data_len), 0,
     "words and such."},
    {"data", T_UINT, offsetof(my_struct, data), 0,
     "words and such."},

    {NULL}  /* Sentinel */
};

Answer 1

Looks like you are working on a system (probably a 64 machine and OS) where unsigned long is a 64 bit type, so there is a mismatch between defining those field as uint32_t and accessing them via T_ULONG.

Changing to T_UINT is exactly what you needed to do; you now access 32 bit types with the appropriate 32 bit data type (unsigned int) instead of the 64 bit T_ULONG.