Removing duplicate strings from python nested list present in any order

Question

[["hello", "bye", "start"], ["bye", "start", "hello"], ["john", "riya", "tom"], ["riya","john", "tom"].....]

I have got a list like this. I want to remove duplicate elements from nested list in Python where elements should be in any order.

Output should be:-

[["hello", "bye", "start"], ["john", "riya", "tom"]]

3 strings should be present only once in any list. How to achieve this?

• pawamoy: What did you try? Iterate on nested list in a for loop and remove the duplicate elements.
• SpoonMeiser: This isn't entirely clear - given your description, I would have expected two empty lists in your output. So is it that you want to remove entire sub-lists if they are the same as a previous list, disregarding order (in which case a quick and dirty solution might be to use sets), or do you want to remove duplicate strings and also remove any lists that are empty afterwards?

Answer 1

Simply covert in to set it removes items automatically.

a = [list(i) for i in {frozenset(k) for k in a}]

• pylover: Excellent answer.

Answer 2

You can use enumerate:

s = [["hello", "bye", "start"], ["bye", "start", "hello"], ["john", "riya", "tom"], ["riya","john", "tom"]]
new_s = [a for i, a in enumerate(s) if not any(all(c in h for c in a) for h in s[:i])]

Output:

[['hello', 'bye', 'start'], ['john', 'riya', 'tom']]

Answer 3

Try this:-

a = [["hello", "bye", "start"], ["bye", "start", "hello"], ["john", "riya", "tom"], ["riya","john", "tom"]]
ls = []
for i in a:
 i = sorted(i)
 if i not in ls:
 ls.append(i)
print(ls)

As u said want to have same output like list then try this tricky method however it won't be pythonic way :-

ls = []
ind = []
for i,j in enumerate(a):
 j = sorted(j)
 if j not in ls:
 ind.append(i)
 ls.append(j)
ls1 = [a[x] for x in ind]
print(ls1)

Output:-

[['hello', 'bye', 'start'], ['john', 'riya', 'tom']]

• Ranjeet: No need of list on list(sorted(i))-> sorted(i).
• Aisha: Output should be [["hello", "bye", "start"], ["john", "riya", "tom"]] or [["bye", "start", "hello"],["riya","john", "tom"]]. Sorting is not what i wanted. I want to remove duplicate list which contains duplicate strings in whatever order it may be. But output should not change the order, in output it should either be ["hello", "bye", "start"] or ["bye", "start", "hello"]

Answer 4

a simple and naive solution

arr = [["hello", "bye", "start"], ["bye", "start", "hello"], ["john", "riya", "tom"], ["riya","john", "tom"]]
seen, tmp, result = list()
for i in arr:
 for j in i:
 if j not in seen:
 seen.append(j)
 tmp.append(j)
 if len(tmp) == 3:
 result.append(tmp)
 tmp = list()
print result

keep in mind this will only append list with 3 elements, so some cases may cause absent elements in the output list (as per your example and instructions given)

Answer 5

I think this is a pretty simple solution:

a = [['hello', 'bye', 'start'], ['bye', 'start', 'hello'], ['john', 'riya', 'tom'], ['riya', 'john', 'tom']]
for i in range(len(a)):
 a[i].sort()
for i in a:
 if i not in b:
 b.append(i)

it outputs:

 [['bye', 'hello', 'start'], ['john', 'riya', 'tom']]

• Aisha: I don't want to sort the list. I want to output the list element in exact same manner, either ['hello', 'bye', 'start'] or ['bye', 'start', 'hello'].