How to fill a matrix with lists of different sized arrays on Python?

Question

I'm working on my first python project where my code extracts and draws contours of a growing blob in a video, and my goal is to create a matrix to fill with all of the contour pixel values for use as an input for a predictive ML model. My idea is to initialize an empty matrix of unspecified size, and append each contour array to it in each step of my for loop, but I've read that this is not a good approach as appending grows the matrix in each loop, and so numpy has to find a new contiguous block of RAM that can fit the new array each run through the loop. Is there a better approach for when I do not know the dimensions in advance? My loop runs through each contour with: for c in contours: and it seems that contours is a list filled with different c's which are ndarrays.

At one period of time,

In: c.shape
Out: (179, 1, 2)

and

In: contours[0]
Out: 
array([[[339, 211]],
 [[338, 212]],
 [[336, 212]],
...
 [[345, 211]]], dtype=int32)

Thanks in advance for any suggestions!

• hpaulj: "an empty matrix of unspecified size" - no such beast in numpy (or Python). Appending to a list is relatively cheap - it just adds a reference in-place. Growing an array requires making a new array and copying all values (old and new) to it.
• liveFreeOrπHard: I appreciate it; that's what I ended up doing and thankfully it didn't noticeably hinder my performance.

Answer 1

I do not know of any dynamically sized array in numpy, but you should just append the lists to a list, and convert it to a numpy.array in a second step, potentially using np.stack, np.hstack, np.vstack, or np.block.

• liveFreeOrπHard: Thank you, I created an empty list and appended the current value at every step in the loop, and it didn't negatively impact my performance at all! Is there a benefit to converting my list of lists into an array of lists?
• Dorian: there shouldn't be. Stick to the lists as long as you can, in the last step, if you need to, convert to a numpy.array. Array of lists doesn't work AFAIK, and list of arrays is useless since you need to convert it back anyways.