Skip to content

Yet another programming solutions log

Sample bits from programming for the future generations.

Technologies Technologies
  • Algorithms and Data Structures
  • Java Tutorials
  • JUnit Tutorial
  • MongoDB Tutorial
  • Quartz Scheduler Tutorial
  • Spock Framework Tutorial
  • Spring Framework
  • Bash Tutorial
  • Clojure Tutorial
  • Design Patterns
  • Developer’s Tools
  • Productivity
  • About
Expand Search Form

Remove duplicates from List in Java

farenda 2016-12-22 0

We show how to remove duplicates from List in Java. Choose implementation depending on circumstances and with expected performance.

Remove duplicates using Set

The simplest solution is to pass a List to Set as a parameter and rely on the property of Sets – no two elements, with correctly implemented equals/hashcode pair, can be in it:

List<Integer> numbers = asList(1, 1, 2, 1, 2, 3, 5);
System.out.println("Numbers: " + numbers);

// Use LinkedHashSet if you want to preserve order
Set<Integer> unique = new HashSet<>(numbers);
numbers = new ArrayList<>(unique);
System.out.println("Unique: " + numbers);

This is the preferred solution in most cases.

The above code produces the following output:

Numbers: [1, 2, 1, 3, 2, 5]
Unique: [1, 2, 3, 5]

Remove duplicates from List manually

This solution doesn’t require additional objects and removes duplicates from given List instead of creating a new one. Note that this implementation does the job in O(n^2) time, so it’s rather slow. However, it’s good to know about it.

List<Integer> numbers =
        new ArrayList<>(asList(1, 1, 2, 1, 2, 3, 5));
System.out.println("Numbers: " + numbers);

ListIterator<Integer> it = numbers.listIterator();
while (it.hasNext()) {
    int i = it.nextIndex();
    Integer current = it.next();
    for (int j = 0; j < i; ++j) {
        if (current.equals(numbers.get(j))) {
            it.remove();
            break;
        }
    }
}
System.out.println("Unique: " + numbers);

We use iterator here, because removal of elements from a List moves further elements to the left and we would need to handle current index by ourselves. The iterator does the job for us.
Also note that we use ListIterator implementation. This is because it gives use access to the index of the current element and we need that to know how many previous elements to check.

The result is the same:

Numbers: [1, 1, 2, 1, 2, 3, 5]
Unique: [1, 2, 3, 5]

Remove duplicates from sorted List

When we want to remove duplicates from sorted List it’s enough to go through all the elements once, because we can find duplicates by comparison with the previous element:

List<Integer> numbers =
         new ArrayList<>(asList(1, 1, 1, 2, 2, 3, 5, 5));
System.out.println("Numbers: " + numbers);

if (numbers.size() > 1) {
    Iterator<Integer> it = numbers.iterator();
    Integer prev = it.next();
    while (it.hasNext()) {
        Integer current = it.next();
        if (prev.equals(current)) {
            it.remove();
        } else {
            prev = current;
        }
    }
}
System.out.println("Unique: " + numbers);

And here are results of running the code:

Numbers: [1, 1, 1, 2, 2, 3, 5, 5]
Unique: [1, 2, 3, 5]

References:

  • Acquaint with method in Java Collections!
Share with the World!
Categories Java Tags algorithms, java, java-util
Previous: Clojure transients – fast mutations in persistent world
Next: GCD in Java using Euclidean Algorithm

Recent Posts

  • Java 8 Date Time concepts
  • Maven dependency to local JAR
  • Caesar cipher in Java
  • Java casting trick
  • Java 8 flatMap practical example
  • Linked List – remove element
  • Linked List – insert element at position
  • Linked List add element at the end
  • Create Java Streams
  • Floyd Cycle detection in Java

Pages

  • About Farenda
  • Algorithms and Data Structures
  • Bash Tutorial
  • Bean Validation Tutorial
  • Clojure Tutorial
  • Design Patterns
  • Java 8 Streams and Lambda Expressions Tutorial
  • Java Basics Tutorial
  • Java Collections Tutorial
  • Java Concurrency Tutorial
  • Java IO Tutorial
  • Java Tutorials
  • Java Util Tutorial
  • Java XML Tutorial
  • JUnit Tutorial
  • MongoDB Tutorial
  • Quartz Scheduler Tutorial
  • Software Developer’s Tools
  • Spock Framework Tutorial
  • Spring Framework

Tags

algorithms bash bean-validation books clojure design-patterns embedmongo exercises git gof gradle groovy hateoas hsqldb i18n java java-basics java-collections java-concurrency java-io java-lang java-time java-util java-xml java8 java8-files junit linux lists log4j logging maven mongodb performance quartz refactoring regex rest slf4j solid spring spring-boot spring-core sql unit-tests

Yet another programming solutions log © 2021

sponsored