Python: Remove Duplicate Items from Nested list

Question

mylist = [[1,2],[4,5],[3,4],[4,3],[2,1],[1,2]]

I want to remove duplicate items, duplicated items can be reversed. The result should be :

mylist = [[1,2],[4,5],[3,4]]

How do I achieve this in Python?

• Volatility: Does the order matter?
• Abhijit: -1: refrain from naming variables to built-ins
• Sanjivani: @Volatility, no oder does not matter.
• Sanjivani: @abhijit, it is just an example, not real code.
• Volatility: Even so, you shouldn't be using list as a variable name.

Answer 1

lst=[[1,2],[4,5],[3,4],[4,3],[2,1],[1,2]]
fset = set(frozenset(x) for x in lst)
lst = [list(x) for x in fset]

This won't preserve order from your original list, nor will it preserve order of your sublists.

>>> lst=[[1,2],[4,5],[3,4],[4,3],[2,1],[1,2]]
>>> fset = set(frozenset(x) for x in lst)
>>> lst = [list(x) for x in fset]
>>> lst
[[1, 2], [3, 4], [4, 5]]

• Oleg: Not exactly, assume you have [0,0] in your lst, this will be collapsed to [0].
• mgilson: That's true. If you're gauranteed that the inputs always have only 2 items and you want to preserve that in the output, it's pretty easy to post-process to turn the 1-element lists into 2-element lists :)

Answer 2

If the Order Matters you can always use OrderedDict

>>> unq_lst = OrderedDict()
>>> for e in lst:
 unq_lst.setdefault(frozenset(e),[]).append(e)
>>> map(list, unq_lst.keys())
[[1, 2], [4, 5], [3, 4]]

Answer 3

If order is not important:

def rem_dup(l: List[List[Any]]) -> List[List[Any]]:
 tuples = map(lambda t: tuple(sorted(t)), l)
 return [list(t) for t in set(tuples)]