Java Programming Tutorials

Java programming tutorials with many code examples!

Remove duplicates from List in Java

We show how to remove duplicates from List in Java. Choose implementation depending on circumstances and with expected performance.

Remove duplicates using Set

The simplest solution is to pass a List to Set as a parameter and rely on the property of Sets – no two elements, with correctly implemented equals/hashcode pair, can be in it:

List<Integer> numbers = asList(1, 1, 2, 1, 2, 3, 5);
System.out.println("Numbers: " + numbers);

// Use LinkedHashSet if you want to preserve order
Set<Integer> unique = new HashSet<>(numbers);
numbers = new ArrayList<>(unique);
System.out.println("Unique: " + numbers);

This is the preferred solution in most cases.

The above code produces the following output:

Numbers: [1, 2, 1, 3, 2, 5]
Unique: [1, 2, 3, 5]

Remove duplicates from List manually

This solution doesn’t require additional objects and removes duplicates from given List instead of creating a new one. Note that this implementation does the job in O(n^2) time, so it’s rather slow. However, it’s good to know about it.

List<Integer> numbers =
        new ArrayList<>(asList(1, 1, 2, 1, 2, 3, 5));
System.out.println("Numbers: " + numbers);

ListIterator<Integer> it = numbers.listIterator();
while (it.hasNext()) {
    int i = it.nextIndex();
    Integer current = it.next();
    for (int j = 0; j < i; ++j) {
        if (current.equals(numbers.get(j))) {
            it.remove();
            break;
        }
    }
}
System.out.println("Unique: " + numbers);

We use iterator here, because removal of elements from a List moves further elements to the left and we would need to handle current index by ourselves. The iterator does the job for us.
Also note that we use ListIterator implementation. This is because it gives use access to the index of the current element and we need that to know how many previous elements to check.

The result is the same:

Numbers: [1, 1, 2, 1, 2, 3, 5]
Unique: [1, 2, 3, 5]

Remove duplicates from sorted List

When we want to remove duplicates from sorted List it’s enough to go through all the elements once, because we can find duplicates by comparison with the previous element:

List<Integer> numbers =
         new ArrayList<>(asList(1, 1, 1, 2, 2, 3, 5, 5));
System.out.println("Numbers: " + numbers);

if (numbers.size() > 1) {
    Iterator<Integer> it = numbers.iterator();
    Integer prev = it.next();
    while (it.hasNext()) {
        Integer current = it.next();
        if (prev.equals(current)) {
            it.remove();
        } else {
            prev = current;
        }
    }
}
System.out.println("Unique: " + numbers);

And here are results of running the code:

Numbers: [1, 1, 1, 2, 2, 3, 5, 5]
Unique: [1, 2, 3, 5]

References:

Share with the World!