Remove Duplicate Lines
Strip repeated lines from a list, with a count of what went.
Off treats Apple and apple as the same.
What does Remove Duplicate Lines do?
Remove duplicate lines from any list — emails, URLs, keywords, IDs — with options for case sensitivity, whitespace trimming, and whether to keep the first or last occurrence. It tells you exactly how many duplicates it removed.
How to Use Remove Duplicate Lines
- 1Paste your listOne entry per line.
- 2Set the optionsCase sensitivity and trimming affect what counts as a duplicate.
- 3Copy the resultThe count of removed lines is shown above it.
Key Features
- Case-sensitive or case-insensitive matching
- Optional whitespace trimming before comparison
- Keep the first or the last occurrence of each duplicate
- Count of lines removed and lines remaining
- Optional sort and blank-line removal in the same pass
- Invert mode: show only the lines that were duplicated
Frequently Asked Questions
Should I use case-sensitive matching?
For email addresses and URLs, no — Example@site.com and example@site.com are the same address. For passwords, codes and identifiers, yes, because case is meaningful.
Why are lines that look identical not being removed?
Almost always trailing whitespace or a different line ending. Switch on trimming, which normalises both before comparing.
Can I see which lines were duplicated?
Yes. Invert mode lists only the entries that appeared more than once, which is what you want when auditing rather than cleaning.
Does the original order survive?
Yes, unless you switch on sorting. Keeping the first occurrence preserves the original sequence, which matters when the list is chronological.
Explore Related Workflows & Utilities
More tools in Text Tools and related problem-solving tools
Direct Workflow Extensions
Sort a list alphabetically, numerically, by length or randomly.
Strip formatting, HTML, smart quotes and invisible characters.
Compare two texts and highlight exactly what changed.
Collapse double spaces, strip trailing whitespace and tidy text.
Search and replace across text, with regular expression support.
Number every line of text, with control over format and start.
Complementary Utilities across Categories
Topical Deep Dives & Guides
Popular Related Searches for Remove Duplicate Lines
Everything you paste stays in this page. Text tools run as JavaScript in your browser, so nothing is uploaded, logged or stored.
Related Tools
Sort Lines
Sort a list alphabetically, numerically, by length or randomly.
Text Cleaner
Strip formatting, HTML, smart quotes and invisible characters.
Text Diff Checker
Compare two texts and highlight exactly what changed.
Remove Extra Spaces
Collapse double spaces, strip trailing whitespace and tidy text.