Python – Ways to remove duplicates from list

List is an important container and used almost in every code of day-day programming as well as web-development, more it is used, more is the requirement to master it and hence knowledge of its operations is necessary. This article focuses on one of the operations of getting the unique list from a list that contains a possible duplicated. Remove duplicates from list operation has large number of applications and hence, its knowledge is good to have.

How to Remove Duplicates From a Python List

Method 1 : Naive method

In Naive method, we simply traverse the list and append the first occurrence of the element in new list and ignore all the other occurrences of that particular element.

# Using Naive methods:

Using Naive method

Output :

Remove duplicates from list using Naive method output

Method 2 : Using list comprehension

List comprehensions are Python functions that are used for creating new sequences (such as lists, tuple, etc.) using previously created sequence. This makes code more efficient and easy to understand. This method has working similar to the above method, but this is just a one-liner shorthand of longer method done with the help of list comprehension.

# Using list comprehension

Remove duplicates from list using list comprehension
Output :

Remove duplicates from list using list comprehension output

Method 3 : Using set():

We can remove duplicates from a list using an inbuilt function called set(). The set() always return distinct elements. Therefore, we use the set() for removing duplicates.But the main and notable drawback of this approach is that the ordering of the element is lost in this particular method.

# Using set()

Remove duplicates from list using set method
Output :

Remove duplicates from list using set method output

Method 4 : Using list comprehension + enumerate():

Enumerate can also be used for removing duplicates when used with the list comprehension.It basically looks for already occurred elements and skips adding them. It preserves the list ordering.

# Using list comprehension + enumerate()

Using list comprehension + enumerate()
Output :

Using list comprehension + enumerate() output
Method 5 : Using collections.OrderedDict.fromkeys():

This is fastest method to achieve the particular task. It first removes the duplicates and returns a dictionary which has to be converted to list. This works well in case of strings also.

# Using collections.OrderedDict.fromkeys()

Using collections.OrderedDict.fromkeys()
Output :

Using collections.OrderedDict.fromkeys() output

Conclusion :

In conclusion, nowyou may know “how to remove duplicates from a list in python?“. There are different ways but the using collections.OrderedDict.fromkeys() method is the best in accordance with the programming efficiency of the computer.

Leave a Comment