Python removing duplicates in a list

Question

I want to remove duplicated in a python list in a way that allows me to alter another corresponding list in the same way. In the below example original is the list I want to de-duplicate. Each element in key that shares the same index as original corresponds to each other:

original = [a,a,a,3,4,5,b,2,b]
key = [h,f,g,5,e,6,u,z,t]

So I want to remove duplicates in original such that whatever element I delete from original I delete the corresponding element (of the same index) in key. Results I want:

deduplicated_original = [a,3,4,5,b,2]
deduplicated_key = [h,5,e,6,u,z]

I can get deduplicated_original using list(set(original)) however I cannot get the corresponding deduplicated_key

• cs95: Is ['g', 5, 'e', 6, 't', 'z'] okay? removal of all but the last?
• Chris: Can you please show any attempts you've made? Where exactly are you stuck?

Answer 1

You can use a set to keep track of duplicates and enumerate() to iterate over indexes/values of the original list:

seen = set()
lst = []
for i, v in enumerate(original):
 if not v in seen:
 lst.append(key[i])
 seen.add(v)
print(lst)

Answer 2

maybe less elegant, less easy to follow the list revesal, index slicing

the inner list comp walks the input list org backwards, asking if there is a prior matching element, if so record the index of this duplicate

[len(org) - 1 - i
 for i, e in enumerate(org[::-1]) if e in org[:-i-1]]

then the outer list comp uses .pop() to modify org, ky as a side effect

nested list comprehension 'dups', a 'one liner' (with line breaks):

org = ['a','a','a',3,4,5,'b',2,'b']
ky = ['h','f','g',5,'e',6,'u','z','t']
dups = [(org.pop(di), ky.pop(di))
 for di in [len(org) - 1 - i
 for i, e in enumerate(org[::-1]) if e in org[:-i-1]]]
org, ky, dups
Out[208]: 
(['a', 3, 4, 5, 'b', 2],
 ['h', 5, 'e', 6, 'u', 'z'],
 [('b', 't'), ('a', 'g'), ('a', 'f')]) 

of course you don't actually have to assign the list comp result to anything to get the side effect of modifying the lists

Answer 3

You can manually get all the indices of duplicates like this:

indices = []
existing = set()
for i, item in enumerate(original):
 if item in existing:
 indices.append(i)
 else:
 existing.add(item)

and then remove those indices from your key list, in reverse because deleting a key changes the indices of further items:

for i in reversed(indices):
 del key[i]