Image tools

Remove Duplicate Lines

Paste a list and get every line back exactly once — with a count of what was removed and a list of which lines repeated.

Runs in your browser. Your image is never uploaded.

How to use Remove Duplicate Lines — Free Online Dedupe Tool

  1. 1Paste your list or text into the left-hand box.
  2. 2Set whether to trim lines and whether case should matter.
  3. 3Copy the result, or download it as a text file.

About Remove Duplicate Lines

Duplicate lines creep into every list that was assembled by hand or merged from two sources — email exports, keyword lists, SKU codes, log lines, URLs pulled from a crawl. This tool keeps the first appearance of each line and drops the rest, leaving the order untouched so you can still see the shape of the original list. That matters more than it sounds: a spreadsheet's "remove duplicates" often sorts your data as a side effect, and a chronological list that has been silently alphabetised is hard to trust afterwards.

The two options that decide the outcome are trimming and case. Exact string comparison is the honest default — "apple " really is not the same text as "apple" — but it is also the reason people paste a list twice and swear the tool is broken. Turn on trimming when your lines came out of a table or a PDF where stray spaces are likely, and turn on case-insensitive matching for things where capitalisation carries no meaning, such as email addresses or tags. Leave both off when the exact characters matter, as they do for passwords, hashes or code.

The duplicate report is the other half of the job. Knowing that 412 lines were removed is useful; knowing that one particular keyword appeared nine times, or that two SKUs were entered twice, is what actually lets you fix the source. Every repeated line is listed with its count, sorted by how often it appeared, and that list is copyable on its own. Nothing you paste leaves your device — the whole tool is JavaScript running locally, so you can work on a customer list or an internal export without it touching a server.

Blank lines are handled deliberately rather than silently: by default they are stripped, which is what you want for a list pasted out of a document, and ticking "Keep blank lines" preserves them when the spacing is part of the content.

Frequently asked questions

Does it reorder my list?
No. The first time a line appears it stays exactly where it was, and only the later copies are deleted. If you also want the result alphabetical, run it through the sort tool afterwards — deduping and sorting are kept separate on purpose so you can see what each one changed.
Are "Apple" and "apple" treated as duplicates?
Not by default — the comparison is exact, so those are two different lines. Tick "Ignore upper / lower case" and they collapse into one; the version kept is whichever appeared first in your list, so "Apple" survives if it came first.
Why does my list still have duplicates?
Almost always invisible whitespace. A single trailing space or a tab makes two otherwise identical lines different strings. Tick "Trim spaces around each line" and they match — the output is then written without that leading and trailing whitespace.
What happens to blank lines?
They are dropped unless you tick "Keep blank lines", in which case they are preserved and deduplicated like any other line. Blank lines never appear in the duplicate report, because a row saying "empty line × 94" tells you nothing useful.
Can I see which lines were the duplicates?
Yes, and that is often the reason to use this tool. Under the result, every line that appeared more than once is listed with its count, most-repeated first. The on-screen list stops at 200 rows to keep the page usable; "Copy list" always gives you the complete set.
Is there a limit on how long my list can be?
There is no fixed cap, because the work happens in your browser rather than on a server. Lists of a few hundred thousand lines deduplicate instantly on a laptop; a very large paste on an older phone is limited by that device's memory and may pause while it runs.