How to check for duplicates in a list and remove them in Python

A list is a collection of data which can be the same as each other. Python list is a powerful data structure that can be used to store different types of items. The key feature of this list is its ability to be modified, which makes it very useful in many practical applications such as web development or machine learning. In addition, Python lists have some features that make them more efficient than other similar data structures like arrays.

It is possible for instance to iterate over the elements in the sequence without using any additional memory allocation operations, and without having to worry about indexing errors due to out-of-bound occurrences (as would happen with C). These are just some of the benefits that make Python lists so great

One of the most common tasks in computing is checking for duplicates in a list. Removing those duplicates can be done quickly with Python. This blog post will give an example of how to do so, using one list as input and another as output.

There are many methods to remove the duplicates.

Table of Contents

Use set()

The set() method creates a set in Python, which doesn’t accept duplicated entries. Our approach is to use set() to convert a list to a set, then compare the length of original list with the new set.

numbers = [0, 1, 2, 3, 1, 3, 5, 9]
numbers_set = set(numbers)
if len(numbers) == len(numbers_set):
	print('No duplicate')
else:
	print('There are ' + str(len(numbers) - len(numbers_set)) + ' duplicates')

Use any()

The any() function takes an iterable (list, string, dictionary etc.) and returns True if any element of the iterable is True. If not, it returns False.

numbers = [7, 2, 5, 2, 1, 5, 5, 8]
dup = any(numbers.count(element) > 1 for element in numbers)

Leave a Comment

Your email address will not be published. Required fields are marked *

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.

Close