Consider the following scenario:

>>> iou_matrix
array([[0.18, 0.29],
 [0.43, 0.45]])
>>> matched_indices
array([[0, 1],
 [1, 0]])

First detection gets the second tracker assigned and second detection gets the first tracker. But as we later discard the first detection anyway, it shouldn't participate in assignment at all, resulting second detection to be assigned to second tracker.

In my experiments this caused identity switches and even though such cases are extremely rare for a low threshold of 0.3, they may get more common for larger values.