Python ctypes pointer to pointer to structure

Question

I'm having problems getting a pointer to a pointer to a structure working. This is the code I have which throws the exception "ArgumentError: argument 1: : expected LP_LP_List instance instead of pointer to LP_LP_List".

class List(Structure):
 _fields_ = (
 ('head', POINTER(Node)),
 ('tail', POINTER(Node)),
 ('current', POINTER(Node)),
 ('saved', POINTER(Node)),
 ('infosize', c_int),
 ('listsize', c_ulong),
 ('current_index', c_ulong),
 ('save_index', c_ulong),
 ('modified', c_bool),
 ('search_origin', c_int),
 ('search_dir', c_int),
 )
list_p = POINTER(List)
create = lib.DLL_CreateList
create.argtypes = [POINTER(POINTER(List)),]
create.restype = POINTER(List)
mem = POINTER(list_p)()
retval = create(byref(mem))

This seems to follow the recommended way to do this, but doesn't work.

Thanks for any help.

For those who don't want to read all the details below to find the solution, the last part should read like this:

#list_p = POINTER(List) # Not needed
create = lib.DLL_CreateList
create.argtypes = [POINTER(POINTER(List)),]
create.restype = POINTER(List)
control = POINTER(List)()
list_p = create(byref(control))

• dstromberg: In what way does it "not work"?
• cnobile: I edited the above with the full exception minus the traceback.
• PaulMcG: Why pass the list pointer by reference if you are returning the created list as the function return value? What does "create" actually do?
• cnobile: I'm writing unit tests in python for a Link List API I wrote over 12 years ago and is used by thousands of people and is still being downloaded weekly. The DLL_CreateList(List **list) function creates a central control structure that all linklist nodes can be control from. The idea is to pass it an empty pointer to a pointer of the control struct. It then creates the linklist control struct in memory. The next call will then attache the data struct that you need to the control struct which can then be duplicated by other calls. tetrasys-design.net/download/Linklist/linklist-1.2.1.tar.gz

Answer 1

The API sounds like the pointer passed in by reference will be modified by create, so it has to be passed by reference, with the return value be the newly created list.

If this were written in "C", I'm guessing you would have some code like this:

List *create(List **control);
List *control;
List *plist;
plist = create(&control);

With ctypes/Python, this would map to:

create.argtypes = [POINTER(POINTER(List)),]
create.restype = POINTER(List)
control = POINTER(List)()
newlist = create(byref(control))

Does this work better?

EDIT: Since create both modifies the passed-in parameter and returns the created list, I'd forget about the return and just keep the argument:

create.argtypes = [POINTER(POINTER(List)),]
control = Pointer(List)()
create(byref(control))

For completeness of your Python API, you might consider writing a context manager, to take care of calling Create (when the context manager's __enter__ method is called by the with statement to initialize the list control block), and automatically calling Destroy (when the context manager's __exit__ method is called at the end of the with statement's scope). Then your Python code could look like:

with listManager as ListManager():
 # do stuff with list managed by list manager
# end of with statement, listManager.__exit__ is automatically called, so that Destroy always
# gets called, even if an exception is raised - this is easier for API users than expecting them
# to write their own try-finally code, and put Destroy in the finally block

EDIT: - I built your DLL on Ubuntu and used the above Python code, it appears that it makes a list after all.

~/dev/python$ python
Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) 
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import *
>>> 
>>> class List(Structure): pass
... 
>>> lib = CDLL('./libdll.so')
>>> create = lib.DLL_CreateList
>>> create.argtypes = [POINTER(POINTER(List)),] 
>>> create.restype = POINTER(List) 
>>> 
>>> control = POINTER(List)() 
>>> create(byref(control))
<__main__.LP_List object at 0x7fdc0c607e60>

EDIT - I wrote a pyparsing utility to read a C header file and output placeholder Structure subclasses and function argtypes and restype definitions. Here's what I got for your lib - it's untested, but might give you a jumpstart in your API testing:

from ctypes import *
ll = CDLL("./libdll.so")
# user defined types
class ll_Info(Structure): pass
class ll_DLL_Boolean(Structure): pass
class ll_DLL_SrchOrigin(Structure): pass
class sys_select_fd_set(Structure): pass
class ll_List(Structure): pass
class sys_time_timeval(Structure): pass
class ll_DLL_SrchDir(Structure): pass
class ll_DLL_Return(Structure): pass
class ll_DLL_SearchModes(Structure): pass
class ll_DLL_InsertDir(Structure): pass
# functions
ll.DLL_CreateList.restype = POINTER(ll_List)
ll.DLL_CreateList.argtypes = (POINTER(POINTER(ll_List)),)
ll.DLL_DestroyList.restype = None
ll.DLL_DestroyList.argtypes = (POINTER(POINTER(ll_List)),)
ll.DLL_Version.restype = c_char_p
ll.DLL_Version.argtypes = ()
ll.DLL_IsListEmpty.restype = ll_DLL_Boolean
ll.DLL_IsListEmpty.argtypes = (POINTER(ll_List),)
ll.DLL_IsListFull.restype = ll_DLL_Boolean
ll.DLL_IsListFull.argtypes = (POINTER(ll_List),)
ll.DLL_CurrentPointerToHead.restype = ll_DLL_Return
ll.DLL_CurrentPointerToHead.argtypes = (POINTER(ll_List),)
ll.DLL_CurrentPointerToTail.restype = ll_DLL_Return
ll.DLL_CurrentPointerToTail.argtypes = (POINTER(ll_List),)
ll.DLL_DecrementCurrentPointer.restype = ll_DLL_Return
ll.DLL_DecrementCurrentPointer.argtypes = (POINTER(ll_List),)
ll.DLL_DeleteCurrentRecord.restype = ll_DLL_Return
ll.DLL_DeleteCurrentRecord.argtypes = (POINTER(ll_List),)
ll.DLL_DeleteEntireList.restype = ll_DLL_Return
ll.DLL_DeleteEntireList.argtypes = (POINTER(ll_List),)
ll.DLL_FindNthRecord.restype = ll_DLL_Return
ll.DLL_FindNthRecord.argtypes = (POINTER(ll_List),POINTER(ll_Info),c_ulong,)
ll.DLL_GetCurrentRecord.restype = ll_DLL_Return
ll.DLL_GetCurrentRecord.argtypes = (POINTER(ll_List),POINTER(ll_Info),)
ll.DLL_GetNextRecord.restype = ll_DLL_Return
ll.DLL_GetNextRecord.argtypes = (POINTER(ll_List),POINTER(ll_Info),)
ll.DLL_GetPriorRecord.restype = ll_DLL_Return
ll.DLL_GetPriorRecord.argtypes = (POINTER(ll_List),POINTER(ll_Info),)
ll.DLL_InitializeList.restype = ll_DLL_Return
ll.DLL_InitializeList.argtypes = (POINTER(ll_List),c_size_t,)
ll.DLL_IncrementCurrentPointer.restype = ll_DLL_Return
ll.DLL_IncrementCurrentPointer.argtypes = (POINTER(ll_List),)
ll.DLL_InsertRecord.restype = ll_DLL_Return
ll.DLL_InsertRecord.argtypes = (POINTER(ll_List),POINTER(ll_Info),ll_DLL_InsertDir,)
ll.DLL_RestoreCurrentPointer.restype = ll_DLL_Return
ll.DLL_RestoreCurrentPointer.argtypes = (POINTER(ll_List),)
ll.DLL_SaveList.restype = ll_DLL_Return
ll.DLL_SaveList.argtypes = (POINTER(ll_List),c_char_p,)
ll.DLL_SetSearchModes.restype = ll_DLL_Return
ll.DLL_SetSearchModes.argtypes = (POINTER(ll_List),ll_DLL_SrchOrigin,ll_DLL_SrchDir,)
ll.DLL_StoreCurrentPointer.restype = ll_DLL_Return
ll.DLL_StoreCurrentPointer.argtypes = (POINTER(ll_List),)
ll.DLL_SwapRecord.restype = ll_DLL_Return
ll.DLL_SwapRecord.argtypes = (POINTER(ll_List),ll_DLL_InsertDir,)
ll.DLL_UpdateCurrentRecord.restype = ll_DLL_Return
ll.DLL_UpdateCurrentRecord.argtypes = (POINTER(ll_List),POINTER(ll_Info),)
ll.DLL_GetSearchModes.restype = POINTER(ll_DLL_SearchModes)
ll.DLL_GetSearchModes.argtypes = (POINTER(ll_List),POINTER(ll_DLL_SearchModes),)
ll.DLL_GetCurrentIndex.restype = c_ulong
ll.DLL_GetCurrentIndex.argtypes = (POINTER(ll_List),)
ll.DLL_GetNumberOfRecords.restype = c_ulong
ll.DLL_GetNumberOfRecords.argtypes = (POINTER(ll_List),)

• cnobile: Keeping the passed in argument is what people have usually done, however, I had requests to pass it out of the function also, but it is usually thrown away. I have already tried your suggestion, but it didn't work either. newlist = Pointer(List)() <-- where did this come from?
• cnobile: OK, I must have done something wrong the first time around. I moved some code around and got it working. control = POINTER(List)() newlist = create(byref(control)) works now. Thanks
• cnobile: Yes it does, as I mentioned above I must have done something wrong the first time I tried it, but it works now. Thanks again for the help.
• cnobile: Wow, thanks a lot that may very well help out. There has been a recent surge in popularity of this API at FreshMeat/freecode over the last few months. I tend to do mostly Python these days and not so must C anymore. However, somebody recently found a 12 year old bug in the DLL_AddRecord() function which had made me want to write the unit tests. Thanks again.