Combining two clock tick vectors with NumPy

Question

I have two long (7683200 length) 1D arrays corresponding to two clock cycles, that I wish to combine into single 1D array. The clock_ticks_fine cycles between 0-255, and corresponds to a coarse division of 7.5 msec, with the reset to 0 signified by a 1 in the clock_ticks_coarse array.

I would like this to be much much faster (I have many of these files), but I can't work out how to vectorise the code. I don't care about the clock_ticks_fine value when clock_ticks_coarse is equal to 1.

First 100 datapoints in both arrays using a basic for loop:

corse_div = 7.5 # in msec
fine_div = 7.5/255.0
clock_ticks_fine = np.array([ 36, 1, 11, 22, 39, 66, 71, 78, 107, 121, 137, 142, 163,
 190, 211, 212, 239, 246, 36, 7, 13, 47, 56, 73, 78, 110,
 128, 143, 146, 175, 192, 203, 204, 239, 36, 0, 13, 19, 43,
 58, 74, 81, 111, 130, 139, 139, 175, 189, 208, 209, 238, 36,
 2, 19, 22, 39, 66, 76, 79, 101, 130, 145, 146, 163, 184,
 201, 214, 227, 255, 36, 9, 20, 45, 64, 75, 78, 103, 122,
 143, 144, 175, 184, 205, 208, 235, 250, 36, 14, 17, 46, 62,
 71, 72, 103, 119, 143, 144, 175, 186, 200])
clock_ticks_coarse = np.array([1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0])
count = -1
for i,boolean in enumerate(clock_ticks_coarse):
 if boolean == 1:
 count +=1
 clock_ticks_coarse[i] = count
 else:
 clock_ticks_coarse[i] = boolean+count
fine_time_array = clock_ticks_fine*fine_div
coarse_time_array = clock_ticks_coarse*corse_div
time_array = fine_time_array+coarse_time_array
np.set_printoptions(precision=3, suppress = True)
print time_array

Output:

[ 1.059 0.029 0.324 0.647 1.147 1.941 2.088 2.294 3.147
 3.559 4.029 4.176 4.794 5.588 6.206 6.235 7.029 7.235
 8.559 7.706 7.882 8.882 9.147 9.647 9.794 10.735 11.265
 11.706 11.794 12.647 13.147 13.471 13.5 14.529 16.059 15.
 15.382 15.559 16.265 16.706 17.176 17.382 18.265 18.824 19.088
 19.088 20.147 20.559 21.118 21.147 22. 23.559 22.559 23.059
 23.147 23.647 24.441 24.735 24.824 25.471 26.324 26.765 26.794
 27.294 27.912 28.412 28.794 29.176 30. 31.059 30.265 30.588
 31.324 31.882 32.206 32.294 33.029 33.588 34.206 34.235 35.147
 35.412 36.029 36.118 36.912 37.353 38.559 37.912 38. 38.853
 39.324 39.588 39.618 40.529 41. 41.706 41.735 42.647 42.971
 43.382]

• plonser: What is boolean doing in clock_ticks_coarse[i] = boolean+count? It should be zero, shouldn't it?
• Cornspierre: yes, you are correct - the if else is unnecessary.

Answer 1

Everything apart from the for loop in your code is already vectorized which leaves only the for loop to be optimized.

You define a counter which is raised by 1 every time the entry of clock_ticks_coarse is 1. There exists a function cumsum in numpy which sums all entries of the argument like this:

import numpy as np
x = np.array([1,3,0,5,7])
np.cumsum(x)
# array([ 1, 4, 4, 9, 16])

It is a vectorized function, so it is very fast. Furthermore, it allows you to replace the for loop completely which should significantly boost the speed of your program. The if else condition can entirely be droped (if boolean is always zero in the else statement).

# This line replaces your whole for-loop and is vectorized
clock_ticks_coarse = np.cumsum(clock_ticks_coarse)-1
# the following lines are already vectorized, so they are fine
fine_time_array = clock_ticks_fine*fine_div
coarse_time_array = clock_ticks_coarse*corse_div
time_array = fine_time_array+coarse_time_array
np.set_printoptions(precision=3, suppress = True)
print time_array
# output
[ 1.059 0.029 0.324 0.647 1.147 1.941 2.088 2.294 3.147
 3.559 4.029 4.176 4.794 5.588 6.206 6.235 7.029 7.235
 8.559 7.706 7.882 8.882 9.147 9.647 9.794 10.735 11.265
 11.706 11.794 12.647 13.147 13.471 13.5 14.529 16.059 15.
 15.382 15.559 16.265 16.706 17.176 17.382 18.265 18.824 19.088
 19.088 20.147 20.559 21.118 21.147 22. 23.559 22.559 23.059
 23.147 23.647 24.441 24.735 24.824 25.471 26.324 26.765 26.794
 27.294 27.912 28.412 28.794 29.176 30. 31.059 30.265 30.588
 31.324 31.882 32.206 32.294 33.029 33.588 34.206 34.235 35.147
 35.412 36.029 36.118 36.912 37.353 38.559 37.912 38. 38.853
 39.324 39.588 39.618 40.529 41. 41.706 41.735 42.647 42.971
 43.382]