On This Page
Seven scans in, three parcels out
A depot scanner sent seven scan events for three parcels. Two of them were sent twice. Here's what the finished program prints when I ask for the exact duplicates to go, and then for one row per parcel:
Raw feed: 7 scans
Exact duplicates removed (LinkedHashSet):
Scan[parcelId=PK-1001, depot=LEEDS, status=RECEIVED, scannedAt=2026-10-26T08:00:00Z]
Scan[parcelId=PK-1002, depot=LEEDS, status=RECEIVED, scannedAt=2026-10-26T08:01:00Z]
Scan[parcelId=PK-1003, depot=DERBY, status=RECEIVED, scannedAt=2026-10-26T08:05:00Z]
Scan[parcelId=PK-1001, depot=DERBY, status=IN_TRANSIT, scannedAt=2026-10-26T11:30:00Z]
Scan[parcelId=PK-1001, depot=DERBY, status=DELIVERED, scannedAt=2026-10-26T15:42:00Z]
One row per parcel, first scan wins:
Scan[parcelId=PK-1001, depot=LEEDS, status=RECEIVED, scannedAt=2026-10-26T08:00:00Z]
Scan[parcelId=PK-1002, depot=LEEDS, status=RECEIVED, scannedAt=2026-10-26T08:01:00Z]
Scan[parcelId=PK-1003, depot=DERBY, status=RECEIVED, scannedAt=2026-10-26T08:05:00Z]
If you searched for how to remove duplicates from a list in Java and keep the order, the one-line answer is new ArrayList<>(new LinkedHashSet<>(list)). It's correct. It's also the answer that quietly fails when your elements don't have equals(), when "duplicate" means "same parcel id" rather than "same everything", or when you pass it an immutable list. This article builds a small parcel-tracker helper that handles all of those, with the options side by side and a failing case for each.
It's for you if you know List, Set and streams at a basic level. There's no library involved, and nothing to install beyond a JDK.
Versions I used
| Tool | Version I used | Check it with |
|---|---|---|
| JDK | OpenJDK 21.0.12.1 (Ubuntu build) | java -version |
| javac | 21.0.12.1 | javac -version |
| Build tool | none, plain javac |
Everything compiles on Java 16 or later, because Stream.toList() and records both arrived in 16. On Java 11 swap .toList() for .collect(Collectors.toList()) and write the Scan record as an ordinary class with equals() and hashCode() (see how to implement equals() and hashCode()).
I compiled and ran all of it from the command line. The IDE notes below come from the vendors' documentation; I didn't click through them. Maven Central wasn't reachable from my build machine, so the tests are 21 plain checks in a main method rather than JUnit. They fail loudly, which is all I need from them. The build took about half an hour, and most of that went on one for loop that refused to fail the way every tutorial says it will.
Set up the parcel-tracker project
parcels/
└── src/main/java/com/example/parcels/
├── Scan.java
├── LegacyScan.java
├── ScanDeduper.java
├── Main.java
├── Checks.java
├── Traps.java
└── Bench.java
From the command line. Create the folders with mkdir -p parcels/src/main/java/com/example/parcels && cd parcels, add the files from the next section, then compile with:
javac -Xlint:all --release 21 -d out $(find src -name '*.java')
On Windows PowerShell there's no find; list the seven files by name instead.
In IntelliJ IDEA. New Project, choose the Java generator, set the JDK to 21, and tick "Add sample code" off. Right-click src/main/java and create the package com.example.parcels. Run Main with the green arrow in the gutter next to main.
In Eclipse. File → New → Java Project, pick a JRE 21 execution environment, and make sure "Create module-info.java file" is unticked. Create the package, paste the files in, and use Run As → Java Application.
How to remove duplicates from a list in Java, one option at a time
Step 1: Model a scan as a record
Goal: an element type that already knows when two scans are the same.
src/main/java/com/example/parcels/Scan.java
package com.example.parcels;
import java.time.Instant;
/** One scan event from a depot scanner. Records give us equals() and hashCode() for free. */
public record Scan(String parcelId, String depot, String status, Instant scannedAt) {
public static Scan of(String parcelId, String depot, String status, String isoTime) {
return new Scan(parcelId, depot, status, Instant.parse(isoTime));
}
}
A record generates equals() and hashCode() from every component. That matters because every Set-based approach below leans on those two methods. If you haven't met records yet, how to use Java records covers them.
Checkpoint: javac prints nothing. No news is the good news here.
Step 2: Write the dedupe helpers
Goal: one class holding every approach, so you can compare them on the same data.
src/main/java/com/example/parcels/ScanDeduper.java
package com.example.parcels;
import java.util.ArrayList;
import java.util.Collection;
import java.util.HashSet;
import java.util.LinkedHashMap;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.function.BinaryOperator;
import java.util.function.Function;
import java.util.stream.Collectors;
public final class ScanDeduper {
private ScanDeduper() {}
/** Whole-object duplicates, first occurrence wins, order kept. Returns a new mutable list. */
public static <T> List<T> withLinkedHashSet(Collection<T> input) {
return new ArrayList<>(new LinkedHashSet<>(input));
}
/** Same result through a stream. Returns an unmodifiable list. */
public static <T> List<T> withStream(Collection<T> input) {
return input.stream().distinct().toList();
}
/** Same result with a loop, so you can see what the set does for you. */
public static <T> List<T> withLoop(Collection<T> input) {
Set<T> seen = new HashSet<>();
List<T> result = new ArrayList<>();
for (T item : input) {
if (seen.add(item)) { // add() returns false when the item was already there
result.add(item);
}
}
return result;
}
/** Removes duplicates from the list you pass in. The list must be modifiable. */
public static <T> void inPlace(List<T> list) {
Set<T> seen = new HashSet<>();
list.removeIf(item -> !seen.add(item));
}
/** Duplicates by a key (for example the parcel id), first occurrence wins. */
public static <T, K> List<T> byKeyKeepFirst(Collection<T> input, Function<? super T, ? extends K> key) {
Set<K> seen = new HashSet<>();
return input.stream().filter(item -> seen.add(key.apply(item))).toList();
}
/** Duplicates by a key, the last occurrence wins, positions follow first appearance. */
public static <T, K> List<T> byKeyKeepLast(Collection<T> input, Function<? super T, ? extends K> key) {
BinaryOperator<T> lastWins = (first, second) -> second;
Map<K, T> byKey = input.stream().collect(
Collectors.toMap(key, Function.identity(), lastWins, LinkedHashMap::new));
return List.copyOf(byKey.values());
}
/** Case-insensitive, whitespace-tolerant de-duplication of plain strings. */
public static List<String> normalizedStrings(Collection<String> input) {
Set<String> seen = new HashSet<>();
return input.stream()
.filter(s -> seen.add(s.strip().toLowerCase(java.util.Locale.ROOT)))
.toList();
}
/** The O(n squared) version people write first. Kept here only so we can time it. */
public static <T> List<T> naiveContains(Collection<T> input) {
List<T> result = new ArrayList<>();
for (T item : input) {
if (!result.contains(item)) {
result.add(item);
}
}
return result;
}
}
Four things in that file are worth a second look.
withLinkedHashSet is the one to reach for first. A LinkedHashSet drops repeats like any Set, but it remembers insertion order, so the survivors come out in the order they first appeared. A plain HashSet doesn't promise any order, and a TreeSet sorts, which isn't the same as keeping order. I check both of those below.
withLoop shows what the set is doing. Set.add returns false when the element was already present, and that one boolean is the entire algorithm. The same trick sits inside inPlace and byKeyKeepFirst, which is how you dedupe by a field without writing a class around it.
byKeyKeepLast uses Collectors.toMap with a merge function and LinkedHashMap::new. Both extras are needed: without the merge function, a repeated key throws (I force that in a moment), and without LinkedHashMap::new you get a HashMap and lose the order. There's a longer treatment of toMap in how to collect stream results.
naiveContains is there so we can time it. It's what most of us write first.
Step 3: Run it on a messy feed
Goal: see exact-duplicate removal and by-parcel dedupe produce different answers from the same input.
src/main/java/com/example/parcels/Main.java
package com.example.parcels;
import java.util.ArrayList;
import java.util.List;
public class Main {
public static void main(String[] args) {
List<Scan> scans = new ArrayList<>(List.of(
Scan.of("PK-1001", "LEEDS", "RECEIVED", "2026-10-26T08:00:00Z"),
Scan.of("PK-1002", "LEEDS", "RECEIVED", "2026-10-26T08:01:00Z"),
Scan.of("PK-1001", "LEEDS", "RECEIVED", "2026-10-26T08:00:00Z"), // exact duplicate
Scan.of("PK-1003", "DERBY", "RECEIVED", "2026-10-26T08:05:00Z"),
Scan.of("PK-1002", "LEEDS", "RECEIVED", "2026-10-26T08:01:00Z"), // exact duplicate
Scan.of("PK-1001", "DERBY", "IN_TRANSIT", "2026-10-26T11:30:00Z"),
Scan.of("PK-1001", "DERBY", "DELIVERED", "2026-10-26T15:42:00Z")));
System.out.println("Raw feed: " + scans.size() + " scans");
System.out.println();
System.out.println("Exact duplicates removed (LinkedHashSet):");
ScanDeduper.withLinkedHashSet(scans).forEach(s -> System.out.println(" " + s));
System.out.println();
System.out.println("One row per parcel, first scan wins:");
ScanDeduper.byKeyKeepFirst(scans, Scan::parcelId).forEach(s -> System.out.println(" " + s));
System.out.println();
System.out.println("One row per parcel, latest scan wins:");
ScanDeduper.byKeyKeepLast(scans, Scan::parcelId).forEach(s -> System.out.println(" " + s));
}
}
Checkpoint: compile, then run java -cp out com.example.parcels.Main. The output is the block at the top of this article, plus a third section:
One row per parcel, latest scan wins:
Scan[parcelId=PK-1001, depot=DERBY, status=DELIVERED, scannedAt=2026-10-26T15:42:00Z]
Scan[parcelId=PK-1002, depot=LEEDS, status=RECEIVED, scannedAt=2026-10-26T08:01:00Z]
Scan[parcelId=PK-1003, depot=DERBY, status=RECEIVED, scannedAt=2026-10-26T08:05:00Z]
Look at the first block again. Parcel PK-1001 still appears three times, because its three scans differ in depot, status and time. They aren't duplicates, they're a history. "Remove duplicates" is meaningless until you decide what counts as a duplicate, and that decision is the real work. Whole-object equality gives you the first block. A key gives you the other two.
One caveat on "latest scan wins": my method keeps the last one in the list, not the one with the newest timestamp. If your feed can arrive out of order, sort by scannedAt first, or use a merge function that compares the timestamps.
Try it end to end on a bigger list
Goal: confirm that the set-based versions scale and the naive one doesn't.
src/main/java/com/example/parcels/Bench.java
package com.example.parcels;
import java.util.ArrayList;
import java.util.List;
import java.util.Random;
public class Bench {
public static void main(String[] args) {
Random rnd = new Random(42);
for (int n : new int[] {10_000, 50_000, 200_000}) {
List<Integer> data = new ArrayList<>(n);
for (int i = 0; i < n; i++) data.add(rnd.nextInt(n / 2)); // about half are repeats
for (int round = 0; round < 3; round++) { // warm up
ScanDeduper.withLinkedHashSet(data);
}
long t0 = System.nanoTime();
int a = ScanDeduper.withLinkedHashSet(data).size();
long t1 = System.nanoTime();
int b = n <= 50_000 ? ScanDeduper.naiveContains(data).size() : -1;
long t2 = System.nanoTime();
System.out.printf("n=%d unique=%d linkedHashSet=%d ms naive=%s%n", n, a, (t1 - t0) / 1_000_000,
b < 0 ? "skipped" : ((t2 - t1) / 1_000_000) + " ms");
}
}
}
java -cp out com.example.parcels.Bench
n=10000 unique=4350 linkedHashSet=1 ms naive=30 ms
n=50000 unique=21553 linkedHashSet=4 ms naive=470 ms
n=200000 unique=86371 linkedHashSet=23 ms naive=skipped
This isn't a proper benchmark. It's one JVM, a three-round warm-up and a wall clock, so treat the numbers as a shape rather than a measurement. The shape is the point: five times the data made the naive loop about fifteen times slower, because List.contains scans the whole result list for every element. The hash-based version grew roughly in line with the input. If you want trustworthy numbers, use JMH.
Prove it with 21 checks
src/main/java/com/example/parcels/LegacyScan.java
package com.example.parcels;
/** A plain class with no equals() or hashCode(). Every instance is "different" to a HashSet. */
public class LegacyScan {
private final String parcelId;
private final String status;
public LegacyScan(String parcelId, String status) {
this.parcelId = parcelId;
this.status = status;
}
public String parcelId() { return parcelId; }
public String status() { return status; }
@Override
public String toString() { return parcelId + "/" + status; }
}
src/main/java/com/example/parcels/Checks.java
package com.example.parcels;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Set;
import java.util.TreeSet;
public class Checks {
static int passed = 0, failed = 0;
static void check(String name, boolean ok) {
if (ok) { passed++; System.out.println("PASS " + name); }
else { failed++; System.out.println("FAIL " + name); }
}
public static void main(String[] args) {
List<String> ids = List.of("b", "a", "b", "c", "a");
check("LinkedHashSet keeps first-seen order", ScanDeduper.withLinkedHashSet(ids).equals(List.of("b", "a", "c")));
check("stream distinct keeps first-seen order", ScanDeduper.withStream(ids).equals(List.of("b", "a", "c")));
check("loop keeps first-seen order", ScanDeduper.withLoop(ids).equals(List.of("b", "a", "c")));
check("naive contains gives same answer", ScanDeduper.naiveContains(ids).equals(List.of("b", "a", "c")));
check("TreeSet sorts instead of keeping order", new ArrayList<>(new TreeSet<>(ids)).equals(List.of("a", "b", "c")));
List<String> mutable = new ArrayList<>(ids);
ScanDeduper.inPlace(mutable);
check("inPlace edits the list you passed", mutable.equals(List.of("b", "a", "c")));
List<String> empty = new ArrayList<>();
ScanDeduper.inPlace(empty);
check("empty list stays empty", empty.isEmpty());
check("input of withLinkedHashSet is not touched", ids.size() == 5);
check("withLinkedHashSet result is mutable", mutableOk(ScanDeduper.withLinkedHashSet(ids)));
check("withStream result is unmodifiable", !mutableOk(ScanDeduper.withStream(ids)));
boolean threw = false;
try { ScanDeduper.inPlace(List.of("x", "x")); } catch (UnsupportedOperationException e) { threw = true; }
check("inPlace on List.of throws UnsupportedOperationException", threw);
List<String> withNull = Arrays.asList("a", null, "b", null);
check("LinkedHashSet accepts one null", ScanDeduper.withLinkedHashSet(withNull).equals(Arrays.asList("a", null, "b")));
List<Scan> scans = List.of(
Scan.of("P1", "LEEDS", "RECEIVED", "2026-10-26T08:00:00Z"),
Scan.of("P1", "LEEDS", "RECEIVED", "2026-10-26T08:00:00Z"),
Scan.of("P1", "DERBY", "DELIVERED", "2026-10-26T15:00:00Z"));
check("record: exact duplicates collapse", ScanDeduper.withLinkedHashSet(scans).size() == 2);
check("byKeyKeepFirst gives one row per parcel", ScanDeduper.byKeyKeepFirst(scans, Scan::parcelId).size() == 1);
check("byKeyKeepFirst keeps the RECEIVED scan", ScanDeduper.byKeyKeepFirst(scans, Scan::parcelId).get(0).status().equals("RECEIVED"));
check("byKeyKeepLast keeps the DELIVERED scan", ScanDeduper.byKeyKeepLast(scans, Scan::parcelId).get(0).status().equals("DELIVERED"));
List<LegacyScan> legacy = List.of(new LegacyScan("P1", "RECEIVED"), new LegacyScan("P1", "RECEIVED"));
check("class without equals(): duplicates survive", ScanDeduper.withLinkedHashSet(legacy).size() == 2);
check("class without equals(): key-based dedupe still works", ScanDeduper.byKeyKeepFirst(legacy, l -> l.parcelId() + l.status()).size() == 1);
List<String> emails = List.of("Ann@Example.com", " ann@example.com", "bob@example.com");
check("plain distinct misses case and spaces", ScanDeduper.withStream(emails).size() == 3);
check("normalizedStrings merges them", ScanDeduper.normalizedStrings(emails).equals(List.of("Ann@Example.com", "bob@example.com")));
check("Set.copyOf rejects nothing but promises no order", Set.copyOf(ids).size() == 3);
System.out.println();
System.out.println(passed + " passed, " + failed + " failed");
if (failed > 0) System.exit(1);
}
static boolean mutableOk(List<String> list) {
try { list.add("zzz"); return true; } catch (UnsupportedOperationException e) { return false; }
}
}
java -cp out com.example.parcels.Checks
PASS LinkedHashSet keeps first-seen order
PASS stream distinct keeps first-seen order
PASS loop keeps first-seen order
PASS naive contains gives same answer
PASS TreeSet sorts instead of keeping order
PASS inPlace edits the list you passed
PASS empty list stays empty
PASS input of withLinkedHashSet is not touched
PASS withLinkedHashSet result is mutable
PASS withStream result is unmodifiable
PASS inPlace on List.of throws UnsupportedOperationException
PASS LinkedHashSet accepts one null
PASS record: exact duplicates collapse
PASS byKeyKeepFirst gives one row per parcel
PASS byKeyKeepFirst keeps the RECEIVED scan
PASS byKeyKeepLast keeps the DELIVERED scan
PASS class without equals(): duplicates survive
PASS class without equals(): key-based dedupe still works
PASS plain distinct misses case and spaces
PASS normalizedStrings merges them
PASS Set.copyOf rejects nothing but promises no order
21 passed, 0 failed
Two of those checks surprised me when I wrote them, in a good way. "class without equals(): duplicates survive" is the single most common reason a LinkedHashSet appears to do nothing: LegacyScan inherits identity-based equals() from Object, so two scans with the same content are two different objects to the set. And the last check is named badly. I left it because it's what I ran: Set.copyOf does de-duplicate, but its documentation promises nothing about iteration order, so don't use it when order matters.
What the set is actually asking
A HashSet decides whether it has seen an element by computing hashCode() to pick a bucket and then calling equals() on whatever's in that bucket. If your type doesn't override both consistently, the set can't recognise a repeat. LinkedHashSet is a HashSet that also threads a linked list through its entries, which is where the insertion order comes from. That's the whole mechanism. No sorting, no comparison between neighbours.
This also explains the cost. Each add is a constant-time lookup on average, so the pass over the list is linear. The naive version asks a List the same question, and a List can only answer by walking itself.
Stream's distinct() does the same job. For an ordered stream, the documentation says it keeps the first occurrence of each element, so it's stable. I'd choose between the two on style: distinct() when the code is already a stream pipeline, the LinkedHashSet when it isn't. The result type differs too, and that bites in the next section. For more on equals() and hashCode(), there's a whole article.
Errors I forced, and the fixes
src/main/java/com/example/parcels/Traps.java
package com.example.parcels;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.function.Function;
import java.util.stream.Collectors;
public class Traps {
public static void main(String[] args) {
trap("remove inside for-each", () -> {
List<String> l = new ArrayList<>(List.of("a", "b", "a", "c"));
Set<String> seen = new HashSet<>();
for (String s : l) { if (!seen.add(s)) l.remove(s); }
});
trap("removeIf on List.of", () -> {
Set<String> seen = new HashSet<>();
List.of("a", "a").removeIf(s -> !seen.add(s));
});
trap("toMap without merge function", () -> {
List<Scan> scans = List.of(
Scan.of("P1", "A", "X", "2026-10-26T08:00:00Z"),
Scan.of("P1", "B", "Y", "2026-10-26T09:00:00Z"));
scans.stream().collect(Collectors.toMap(Scan::parcelId, Function.identity()));
});
trap("distinct() on a stream, then add", () -> {
List<String> l = List.of("a", "a").stream().distinct().toList();
l.add("z");
});
trap("List.copyOf with a null", () -> {
List<String> l = new ArrayList<>();
l.add("a"); l.add(null);
List.copyOf(l);
});
}
static void trap(String name, Runnable r) {
try { r.run(); System.out.println(name + ": no exception"); }
catch (Exception e) { System.out.println(name + " -> " + e.getClass().getName() + (e.getMessage() == null ? "" : ": " + e.getMessage())); }
}
}
remove inside for-each: no exception
removeIf on List.of -> java.lang.UnsupportedOperationException
toMap without merge function -> java.lang.IllegalStateException: Duplicate key P1 (attempted merging values Scan[parcelId=P1, depot=A, status=X, scannedAt=2026-10-26T08:00:00Z] and Scan[parcelId=P1, depot=B, status=Y, scannedAt=2026-10-26T09:00:00Z])
distinct() on a stream, then add -> java.lang.UnsupportedOperationException
List.copyOf with a null -> java.lang.NullPointerException
Going through them:
removeIf on List.of: UnsupportedOperationException. List.of, List.copyOf and Stream.toList() all return lists you can't modify. That's why my inPlace method says the list must be modifiable. Fix: copy into new ArrayList<>(...) first, or use a method that returns a new list.
distinct() then add: UnsupportedOperationException. Same cause. Stream.toList() is unmodifiable. If the caller needs to add to the result, use Collectors.toCollection(ArrayList::new). My check "withStream result is unmodifiable" pins this behaviour down.
toMap without a merge function: IllegalStateException: Duplicate key P1. It's thrown at the first repeated key. The message even includes both values, which is handy. Add a merge function. There's a separate article on this error already: Duplicate key in Collectors.toMap.
List.copyOf with a null element: NullPointerException. LinkedHashSet accepts one null, as my check shows, but the immutable-list factories don't accept any. If nulls can appear, filter them first.
Removing inside a for-each loop. This one fooled me. Every tutorial says it throws ConcurrentModificationException. My first attempt, on [a, b, a, c], printed no exception at all:
[a, b, a, c] -> no exception, result [b, a, c]
[a, a, b, c] -> java.util.ConcurrentModificationException
Two lessons in three lines. The first input slips through because the removed element is the second-to-last one, so the iterator's hasNext() sees cursor == size and quietly stops before it would have noticed the change. Same bug, no symptom, so a test on one lucky input proves nothing. The second lesson is in the result: l.remove(s) removes the first equal element, so it deleted the earlier "a" and kept the later one. The loop "worked" and kept the wrong duplicate. Use removeIf, an iterator's remove(), or build a new list.
Before this goes near production
- Decide what a duplicate is before you write code. Whole-object, by key, case-insensitive, by normalised value. My
normalizedStringshandles case and stray whitespace for strings; the check "plain distinct misses case and spaces" shows what you'd get without it. For emails, also think about whetherann+news@example.comis the same person. That's a business rule, not a Java one. - Mutable elements break sets. If you change a field that feeds
hashCode()after the element is in aHashSet, the set can't find it again. Use records or other immutable classes for anything you dedupe. - Big lists: memory is the cost, because the set holds a second reference per unique element. For a few hundred thousand items, ignore it. For hundreds of millions, dedupe in the database, with a unique constraint or
SELECT DISTINCT, not in Java. - Sorted input is a different problem. If the list is already sorted, adjacent-compare in one pass and skip the set. If you want the result sorted anyway, a
TreeSetdoes both jobs, usingcompareToinstead ofequals; see Comparable and Comparator for how that can disagree withequals. - Concurrent access needs its own plan. None of these helpers are thread-safe.
- Related next steps: a
LinkedHashMapis the tool when you also want to keep a value per key, and I'll cover it in how to use LinkedHashMap.
The short version
- Whole-object duplicates with order kept:
new ArrayList<>(new LinkedHashSet<>(list)), orstream().distinct().toList(). - Duplicates by a field:
seen.add(key)inside afilter, ortoMapwith a merge function andLinkedHashMap::new. - The elements need
equals()andhashCode(). A record gives you both. - Don't remove from a list inside a for-each loop. Use
removeIfon a modifiable list. List.of,List.copyOfandStream.toList()give you lists you can't change.
Questions people ask about removing duplicates in Java
How do I remove duplicates from an ArrayList without changing the order?
Put it through a LinkedHashSet and back into a new ArrayList, or call stream().distinct(). Both keep the first occurrence of each element in its original position. A HashSet won't promise that, and a TreeSet will sort.
Why doesn't distinct() or HashSet remove my duplicates?
Almost always because the element class doesn't override equals() and hashCode(). Two objects with identical fields are still two objects. Make it a record, or implement both methods.
How do I remove duplicates by one field, like an id?
Keep a Set of the keys you've seen and filter with seen.add(key), or collect with Collectors.toMap(key, identity, mergeFunction, LinkedHashMap::new) and take values(). The first keeps the first match; the second lets you choose which one survives.
Can I remove duplicates in place?
Yes, if the list is modifiable: list.removeIf(x -> !seen.add(x)). It fails with UnsupportedOperationException on List.of and Stream.toList() results.