Remove Duplicate Lines
Delete repeated lines from a list while keeping the original order.
- Runs in your browser
- Nothing is uploaded
- Free, no sign-up
Not case sensitive: Apple and apple count as the same line, and the first spelling is kept. Order is preserved. Everything stays on your device.
About this tool
Paste a list, one item per line, and get it back with the repeats removed and the original order intact. You can keep the first or last copy of each line, keep only the lines that appear once, or do the opposite and list just the ones that repeat.
How to use it
- Paste your list or open a text file.
- Choose what to keep, and set case sensitivity, trimming and blank-line handling.
- Copy the result or download it, and check the counts underneath.
Four things people mean by “remove duplicates”
The phrase covers different jobs, so the tool offers each of them.
| Option | Input | Result | Use it to |
|---|---|---|---|
| Keep the first of each | b a b c a |
b a c |
Clean a list, keeping the earliest entry |
| Keep the last of each | b a b c a |
b c a |
Keep the most recent entry in a log |
| Keep only lines that appear once | b a b c a d |
c d |
Find what is missing from the other copies |
| Keep only repeated lines | b a b c a d |
b a |
List the duplicates themselves |
Why “duplicates” sometimes survive
Two lines are only duplicates if they match character for character. The usual culprits:
- Trailing spaces.
appleandapplelook identical and are different. Leave Trim spaces around each line on to treat them as equal. - Capital letters.
Appleandappleare different unless case sensitivity is off, which is the default here. - Invisible characters. A zero-width space or a byte-order mark makes a line differ while looking the same. Tabs and non-breaking spaces are covered by trimming, but zero-width characters are not.
- Different line endings. Windows and Unix line breaks are both understood, and the output keeps the style of your input.
Order, size and what comes next
The original order is always kept, so you can deduplicate without losing the structure of a list. Lines are compared with a hash set, which keeps it fast on lists of hundreds of thousands of lines.
If you want the result alphabetised as well, use Sort lines, which can remove duplicates in the same step. To change the text itself rather than remove it, use Find and replace.
Frequently asked questions
Is the original order preserved?
Yes. Lines stay in the order they came in. The first occurrence keeps its place, or, in keep-last mode, the last occurrence does.
Is there a limit on the number of lines?
No fixed limit. Lists of a few hundred thousand lines are fine; the limit is your browser's memory.
Can it deduplicate a column of a CSV?
It works on whole lines. Copy the column on its own, deduplicate it, and paste it back, or use the JSON–CSV converter to reshape the file first.
Is my list uploaded?
No. The work happens in your browser, so lists of email addresses or customer IDs never leave your device.
Related tools
Sort Lines Alphabetically
Sort a list of lines A to Z, naturally, by number or by length.
Find and Replace Text
Replace text in bulk with a plain search or a regular expression.
Word & Character Counter
Count words, characters, sentences and reading time live.
Text Case Converter
Convert text between camelCase, snake_case, kebab-case and more.