Remove duplicates nested list Python dependend on position

Question

in a nested list like the one below, I'd like to remove duplicates depending on the position.

[[1,2,3,4], [2,1,3,4], [1,2,3,4], [1,3,2,4]]

So every sub-list contains the same numbers, but in different order. If the order of the numbers is the same, I would like to delete this duplicate. So the list above should look like:

[[1,2,3,4], [2,1,3,4], [1,3,2,4]]

I have tried to write some code by myself, but since I am a beginner, there is no working result. I have tried:

result = []
for i in test_list:
 if i not in result:
 result.append(i)
return result

Or

tpls = [tuple(x) for x in test_list]
dct = list(dict.fromkeys(tpls))
dup_free = [list(x) for x in test_list]
return dup_free

Thanks!

EDIT2: Sorry everbody, the input was wrong so the code just could not work...

• charon25: In your second solution, you wrote test_list instead of dct.
• Sharim09: Why return statement is here.

Answer 1

lst = [[1,2,3,4], [2,1,3,4], [1,2,3,4], [1,3,2,4]]
new_lst = []
for a in lst:
 if a not in new_lst: # check if current element already in the new_lst or not.
 new_lst.append(a)
print(new_lst)

OUTPUT

[[1, 2, 3, 4], [2, 1, 3, 4], [1, 3, 2, 4]]

• Ic3Cub3: I also have tried this code, however it does not seem to work
• Sharim09: @Ic3Cub3 But it's working.
• Sharim09: @Ic3Cub3 Can you copy this code and run on your computer?
• Ic3Cub3: I copied the code, ran it on my computer with the full list, but it only returns [[3, 4, 1, 2]]
• Ic3Cub3: yeah sorry my input list was just wrong. My bad...

Answer 2

You can use pandas for this purpose.

import pandas as pd
aaa = {'date': ['2022-05-02', '2022-04-29', ' 2022年04月29日', '2022-04-28 ', '2022-04-28 ', '2022-04-28 '],
 'id': ['A', 'A', 'B', 'A', 'B', 'C'], 'number': [397177, 53876, 191214, 75824, 483860, 51731]}
df = pd.DataFrame(aaa)
df = df.drop_duplicates().values

Output

[[1 2 3 4]
 [2 1 3 4]
 [1 3 2 4]]

Answer 3

First one is working run this code :

X = [[1,2,3,4], [2,1,3,4], [1,2,3,4], [1,3,2,4]]
def Rem_Dup(L) :
 result = []
 for i in L:
 if i not in result:
 result.append(i)
 return result
print(Rem_Dup(X))

Answer 4

For each sublist sublst in the original nested list, convert into tuple for lookup in the dictionary dct. Add to the resulting list without dups nodups only if we have not seen it:

lst = [[1,2,3,4], [2,1,3,4], [1,2,3,4], [1,3,2,4]]
nodups = []
dct = {}
for sublst in lst:
 tup = tuple(sublst)
 if tup not in dct:
 dct[tup] = 1
 nodups.append(sublst)
 else:
 pass
print(nodups)
# [[1, 2, 3, 4], [2, 1, 3, 4], [1, 3, 2, 4]]

Answer 5

You can do it in one line with a list comprehension:

a = [[1,2,3,4], [2,1,3,4], [1,2,3,4], [1,3,2,4]]
[j for i, j in enumerate(a) if j not in a[i+1:]]

Output:

[[2, 1, 3, 4], [1, 2, 3, 4], [1, 3, 2, 4]]

Note: This will choose the last occurrence of that sublist in the original list if you care about the ordering

Edit: If you care about ordering, you can go through the list backwards and then reverse the result:

[j for i, j in enumerate(reversed(a)) if j not in list(reversed(a))[i+1:]][::-1]

Output:

[[1, 2, 3, 4], [2, 1, 3, 4], [1, 3, 2, 4]]