Print or remove duplicates in list
How to quickly print out or remove duplicated items from given list in python.
Remove duplicates only
# simply change list to set, e.g.
list_with_dups = [1, 2, 3, 1]
list_without_dups = list(set(list_with_dups))
Find out duplicates only
import collections
list_with_dups = [1, 2, 3, 1]
print [item for item, count in collections.Counter(list_with_dups).items() if count > 1]
Find out and remove duplicates
list_with_dups = [1, 2, 3, 1]
dups = []
uniq = []
for item in list_with_dups:
if item not in uniq:
uniq.append(item)
else:
dups.append(item)
# print dups
print dups
# print unique list with dups removed
print uniq
Also see this stackoverflow post for reference.