Print or remove duplicates in list

Mar 2, 2017

How to quickly print out or remove duplicated items from given list in python.

Remove duplicates only

# simply change list to set, e.g.
list_with_dups = [1, 2, 3, 1]
list_without_dups = list(set(list_with_dups))

Find out duplicates only

import collections
list_with_dups = [1, 2, 3, 1]
print [item for item, count in collections.Counter(list_with_dups).items() if count > 1]

Find out and remove duplicates

list_with_dups = [1, 2, 3, 1]
dups = []
uniq = []
for item in list_with_dups:
    if item not in uniq:
        uniq.append(item)
    else:
        dups.append(item)
# print dups
print dups
# print unique list with dups removed
print uniq

Also see this stackoverflow post for reference.

python