removing duplicates from a list python

Question

I have two lists:

list1 = [IM12345, IM12346, IM12347, IM12348]
list2 = [ID300, ID404, ID300, ID601]

list2 associates with the corresponding list1 values. list1 has unique values where as list2 has duplicates.

I want to make list2 unique corresponding associated value will add in the that list2 value.

Dict= {ID300: {IM12345, IM12347}, ID404: IM12346, ID601: IM12348}

Above pattern can be in list, set or dictionary.

Which algorithm in python should I use to get the above result?

Answer 1

You could try collections.defaultdict:

from collections import defaultdict
d = defaultdict(set)
list1 = ['IM12345', 'IM12346', 'IM12347', 'IM12348']
list2 = ['ID300', 'ID404', 'ID300', 'ID601']
for key, value in zip(list2, list1):
 d[key].add(value)

Demo:

>>> d
defaultdict(<class 'set'>, {'ID300': {'IM12345', 'IM12347'}, 'ID404': {'IM12346'}, 'ID601': {'IM12348'}})
>>>
>>>
>>> for i, j in d.items():
... print(i, j)
... 
... 
ID601 {'IM12348'}
ID300 {'IM12345', 'IM12347'}
ID404 {'IM12346'}

Answer 2

You can create a dict to save the dataset

list1 = ["IM12345", "IM12346", "IM12347", "IM12348"]
list2 = ["ID300", "ID404", "ID300", "ID601"]
dictResult=dict()
i=0
for item in list2:
 print item
 if dictResult.has_key(item):
 dictResult[item].append(list1[i])
 else:
 dictResult[item]=[list1[i]]
 i=i+1
print dictResult

Result:

{'ID404': ['IM12346'], 'ID300': ['IM12345', 'IM12347'], 'ID601': ['IM12348']}

Answer 3

Might not completely be a pythonic way but - here it goes: Map the input:

map = dict(zip(list1, list2))

Now you can do an inverse mapping:

inv_map = {}
for k, v in map.iteritems():
 inv_map[v] = inv_map.get(v, [])
 inv_map[v].append(k)

Result for the example above:

>>> inv_map
{'ID404': ['IM12346'], 'ID300': ['IM12345', 'IM12347'], 'ID601': ['IM12348']}

Answer 4

Another way of doing it could be with list operations.

yourList = ["1","2","3","4","1","2"] 
newList = [] 
for f in yourList:
 if f not in newList:
 newList.append(f)

Answer 5

Simple solution

from collections import defaultdict
list1 = ['IM12345', 'IM12346', 'IM12347', 'IM12348']
list2 = ['ID300', 'ID404', 'ID300', 'ID601']
d = defaultdict(list)
for n in range(len(list2)):
 d[list2[n]].append(list1[n])
print d.items()

Result:

[('ID404', ['IM12346']), ('ID300', ['IM12345', 'IM12347']), ('ID601', ['IM12348'])]

Python2.7 Documentation----defaultdict