Python List of list, remove duplicates

Question

Trying to remove duplicates in list of list and print same without duplicates.

Original List

a = [['country',['America_1','America_2','America_3','America_4','England_5','England_6'],['apple_1_more','orange_1_more']],['country',['Brazil_2','Brazil_3','Brazil_1','Brazil_4','Mexico_1','Mexico_3','Mexico_2'],['grapes_1_less','banana_1_more']]]

looking for output:

[['country', ['America', 'England'], ['orange_more', 'apple_more']], ['country', ['Mexico', 'Brazil'], ['grapes_less', 'banana_more']]]

but getting:

[['country', ['America', 'England'], ['orange_more', 'apple_more']], ['country', ['America', 'England', 'Mexico', 'Brazil'], ['orange_more', 'grapes_less', 'banana_more', 'apple_more']]]

code::

 a = [['country',['America_1','America_2','America_3','America_4','England_5','England_6'],['apple_1_more','orange_1_more']],['country',['Brazil_2','Brazil_3','Brazil_1','Brazil_4','Mexico_1','Mexico_3','Mexico_2'],['grapes_1_less','banana_1_more']]]
aa ={}
aaa=[]
aaaa=[]
aaaaa=[]
for i in a:
 for j in i[1]:
 j=j.split('_',1)[0]
 aaa.append(j)
 for k in i[2]:
 k=k.split('_',2)[0]+'_'+k.split('_',2)[2]
 aaaa.append(k)
 aa['country'] = [i[0],list(set(aaa)),list(set(aaaa))]
 aaaaa.append(aa['country'])
print (aaaaa)

• John Ellmore: You'll have a much easier time if you use meaningful variable names.

Answer 1

Using a list comprehension, converting the second item in each sublist to and from a set():

a = [['country',['America','America','America','America','England','England']],['country',['Brazil','Brazil','Brazil','Brazil','Mexico','Mexico','Mexico']]]
a = [[i, list(set(j))] for i, j in a]
print(a)

Output:

[['country', ['England', 'America']], ['country', ['Brazil', 'Mexico']]]

This may not preserve the order of the inner list, as sets are unordered, so you may need to account for this.

Answer 2

Use this recursive function to remove duplicate item in multi level array:

def dup(input_):
 if isinstance(input_, list):
 try:
 input_ = list(set([i.split('_')[0] if not isinstance(i, list) else i for i in input_]))
 except TypeError:
 pass
 for child in input_:
 input_[input_.index(child)] = dup(child)
 return input_

• GRNearth: Traceback (most recent call last): dup(a) input_ = list(set([i.split('')[0] for i in input])) input_ = list(set([i.split('')[0] for i in input])) AttributeError: 'list' object has no attribute 'split'

Answer 3

This is how I would go about it.

country_list1 = [a[0[0]]]
country_list2 = [a[1[0]]]
duplicates = [country for country in country_list1 in country_list2]
non_duplicates = [country for country in country_list1 not in country_list2]

This will give you both the duplicated ones and non-duplicated This is considering case sensitiveness of the names in both

Answer 4

You can try this approach :

a = [['country',['America','America','America','America','England','England']],['country',['Brazil','Brazil','Brazil','Brazil','Mexico','Mexico','Mexico']]]
print(list(map(lambda x:[x[0],list(set(x[1:][0]))],a)))

output:

[['country', ['England', 'America']], ['country', ['Mexico', 'Brazil']]]

Your variables names are very confusing , Still i tried new approach , you can try this:

a = [['country',['America_1','America_2','America_3','America_4','England_5','England_6'],['apple_1_more','orange_1_more']],['country',['Brazil_2','Brazil_3','Brazil_1','Brazil_4','Mexico_1','Mexico_3','Mexico_2'],['grapes_1_less','banana_1_more']]]
final_data=[]
for i in a:
 sub_data=[]
 for j in i[1:]:
 d = {}
 for m in j:
 data=m.split('_')[0]
 d[data]=data
 sub_data.append(list(d.keys()))
 final_data.append(['country',*sub_data])
print(final_data)

output:

[['country', ['America', 'England'], ['orange', 'apple']], ['country', ['Brazil', 'Mexico'], ['banana', 'grapes']]]

If your data format is always like this then you can try this:

update

a = [['country',['America_1','America_2','America_3','America_4','England_5','England_6'],['apple_1_more','orange_1_more']],['country',['Brazil_2','Brazil_3','Brazil_1','Brazil_4','Mexico_1','Mexico_3','Mexico_2'],['grapes_1_less','banana_1_more']]]
final_data=[]
for i in a:
 sub_data=[]
 sub_extra=[]
 for j in i[1:2]:
 sub_extra.append(i[2])
 d = {}
 for m in j:
 data=m.split('_')[0]
 d[data]=data
 sub_data.extend([list(d.keys()),*sub_extra])
 final_data.append(['country',*sub_data])
print(final_data)

output:

[['country', ['America', 'England'], ['apple_1_more', 'orange_1_more']], ['country', ['Mexico', 'Brazil'], ['grapes_1_less', 'banana_1_more']]]

• GRNearth: Thanks, this worked perfectly...Thank @Ayodhyankit Paul... updated list of list but unable to use lambda
• GRNearth: updated.. with split for updated list of list ..getting error..print(list(map(lambda x:[x[0],list(set(x[1:][0].split('',1)[0])),list(set(x[1:][1].split('',2)[0]+'_'+[2]))],a)))
• GRNearth: AttributeError: 'list' object has no attribute 'split'
• GRNearth: - [['country', ['America', 'England'], ['apple', 'orange']], ['country', ['Brazil', 'Mexico'], ['grapes', 'banana']]] but looking for [['country', ['America', 'England'], ['orange_more', 'apple_more']], ['country', ['Mexico', 'Brazil'], ['grapes_less', 'banana_more']]]