Paste a list and the repeats disappear. The default treats each line as one item, trims the spaces around it and drops blank lines, which is what you want for a column of URLs, emails or product codes copied out of a spreadsheet.
A worked example with the default options plus ignore case:
apple / banana / Apple / banana / (blank) / cherry
Output:
apple / banana / cherry
Result: 3 items kept, 2 duplicates removed
"Apple" disappeared because case was ignored and the first spelling — lowercase "apple" — is the one kept. With ignore case off, the same list keeps both spellings and only the repeated "banana" goes.
Four ways to define an item
| Mode | An item is | Output joins with |
|---|---|---|
| Lines | Everything between two line breaks | A new line |
| Words | Everything between two spaces | A single space |
| Sentences | Text up to the next full stop, question or exclamation mark | A single space |
| Comma values | Text between commas or line breaks | A comma and a space |
Comma values is the mode people miss. It splits on commas and on newlines at the same time, so a list pasted from a single cell and a list pasted from a column are handled by the same click. The output is rebuilt as apple, banana, cherry, ready to paste back into a spreadsheet cell.
Words mode is the lossy one. It rebuilds the text with single spaces, so line breaks, indentation and the original spacing are all gone. Use it to deduplicate a vocabulary list, not to clean a paragraph.
The four options, and the order they run
- Trim spaces (on by default) removes whitespace from both ends of every item. Without it,
appleandappleare different items. - Remove empty (on by default) drops items that are empty after trimming, which is what blank lines between paragraphs become.
- Ignore case (off by default) lowercases each item before comparing, so
Appleandapplecollide. The spelling that survives is the first one in the list. - Sort A–Z (off by default) reorders what is left, using the browser's locale-aware comparison.
Order matters, and the tool applies them in a fixed sequence: trim, then drop empties, then case-fold for comparison, then deduplicate, then sort. Sorting last is why the kept copy of a duplicate is always the first occurrence in your original order — turning sort on does not change which spelling wins, only where it ends up.
One counting quirk worth knowing. The "original items" figure is the kept items plus the duplicates removed, so items dropped as empty are not counted at all. In the worked example above, six lines went in and the stat reads 5, because the blank line vanished before the tally.
Sentences mode is the weakest one
Sentence splitting depends on punctuation, and this site's splitter masks abbreviations by replacing their full stop before splitting. The mask is broad: Sat, Sun, Mon, Wed, No, Fig and St are all on the abbreviation list, so any sentence ending in the word "sat" or "sun" is glued to the next one.
Type the cat sat. the cat sat. the dog ran. and the tool reports no duplicates, because it sees one long sentence rather than three. The fault is in the splitting, not the deduplication. If your text has sentences ending in those words, use lines mode with one sentence per line instead.
Sentences mode also joins its output with spaces, so paragraph breaks are lost. That is fine for a deduplicated list of one-line sentences, and wrong for prose you intend to keep.
What counts as a duplicate
Matching is exact after the options are applied. There is no fuzzy matching, no similarity score and no normalisation of punctuation or Unicode. New York and new york are duplicates only with ignore case on. New York and New York with a non-breaking space are never duplicates, because the trim option removes ordinary whitespace at the ends of an item, not a space in the middle.
Trailing commas and other punctuation stay attached in words mode, so apple, and apple are two different items. That is often the reason a "dedupe" appears to do nothing on a list copied from prose.
Limits and habits
- No counts or reports. You get items kept, duplicates removed and the character count of the result. There is no table of which item appeared how many times.
- No fuzzy or near-duplicate removal. For that, compare the two lines with the similarity checker first.
- No undo. Applying the result replaces your input, so copy the original if the order matters.
- Text stays in the tab. Nothing is uploaded and a reload clears the box.
For a mailing list, the safe routine is: trim and remove empty on, ignore case on, sort on, then read the first and last few lines. Sorting brings variants of the same entry next to each other, which is how you spot the ones exact matching cannot merge — Acme Ltd next to Acme Limited.