Duplicate elements in a list in pythons can be removed using different ways In This Python Tutorial we will discuss different ways to do so.
List In Python
A list in python is very important for Data structure, As a list allows us to insert different types of data in a single list.
Although a dictionary also gives the same property dictionary requires more space to store as it has to store its keys as well to each element it is not there in the case of a list.
As there are chances of getting some duplicate values in the list So here we will discuss different ways to remove the duplicate values from list
You can learn more about List from Python List Problems with Answers
Different Ways to Remove Duplicate Values From the List
As removing duplicate data is a very important step in data cleaning in deep learning to use the data and get the best prediction the data should be unique So foe the same There are a number of ways to remove duplicate Values And few of we will discuss here with examples.
Using Set To Remove Duplicates
We can remove all the duplicate values from our list By storing all the elements of a list in a set, Where it is the property of sets that it never stores any duplicate values. So here we will use this property of set to remove all the duplicate entries.
We can perform like.
t = [1, 2, 3, 1, 2, 3, 5, 6, 7, 8] list(set(t))
It will give – [1, 2, 3, 5, 6, 7, 8]
Drawback Of Using This
One thing we need to keep in mind, in this case, is that it does consider the order, which means the order of the elements stored maybe got changed, and maybe there will not be any order all the elements will be stored randomly.
If the order of data is important then we can not use this method in our program. For keeping the order the same we have another method.
Maintaining Order While Removing Duplicates
For removing Duplicates from the list and maintaining the order we use the OrderedDict function which keeps the order and removes the duplicate entries
from collections import OrderedDict t = [1, 2, 3, 1, 2, 3, 5, 6, 7, 8] list(OrderedDict.fromkeys(t))
It will give output as [1, 2, 3, 5, 6, 7, 8]. It is the method for python 2.7 but for new versions, we can simply use Dicts function for Unique values of the list or to remove the duplicate values.
s showed in the code we need to import the library of OrderedDict from Collection to use it as we can see in the output the order is as it is but all the duplicate elements are being removed
Removing Without Using Any Function
We can also remove the duplicate values from the list without using any function, only by using a set of code by using a for loop although it requires a set of code to write to we do not require to import any library for it. To do so follow the Code as.
t = [1, 2, 3, 1, 2, 5, 6, 7, 8]
s = []
for i in t:
if i not in s:
s.append(i)
s
Here we are checking all the elements individually with all the existing elements in the list whether the element is already there or not if is not there it will be added to a new list and if it is already there it will be passed to the next element.
The OUTPUT of this Code will be – [1, 2, 3, 4, 5, 6, 7, 8]
The drawback of Checking Individual Elements
As It is using a for loop so the complexity of the Program will be O(n^2) which would make the program slow in case of a large list. So we need to keep this thing in mind.
Learn More about Removing duplicates from a list at Duplicate Values in Python
Also visit How To Reverse A String In Python?
