# Remove Duplicate Lines

Remove duplicate lines keeping the first of each, or show only the lines that repeat. Free, and nothing you paste is uploaded.

Canonical page: https://filetity.com/remove-duplicate-lines

## Capabilities

- Free: Yes, with no watermark and no page limit.
- Account required: No. There is no account system.
- Processed on device: Yes. The file is read by your browser and never uploaded. There is no server to send it to.
- Formats: Any lines in. Either the unique lines, keeping the first of each, or only the lines that appear more than once.
- Limits: Lines are compared whole. Two lines differing anywhere at all are different lines, unless case or edge whitespace is being ignored.
- Mobile: Yes, on any modern mobile browser. Large files are limited by the memory the phone gives the browser.
- Offline: Yes, once the page has loaded. The tool keeps working with no network connection.

## Questions

### Which copy of a duplicate is kept?

The first. That preserves whatever order the list was in, which matters when it is ranked, chronological or simply something a person arranged. Keeping the last would silently reorder the result, which is rarely what anyone wants.

### Are Bob and bob the same line?

Not by default, and yes if you turn on ignore case. Names and email addresses are the usual reason to want it. When it is on, the first spelling is kept rather than a lowercased version, so the list keeps the capitalisation somebody typed.

### How do I find the duplicates instead of removing them?

Switch to showing only repeats, which lists each line that appears more than once, once. That is the option you want when the duplicates are the problem you are investigating rather than noise you are clearing away.

### What about lines that differ only by a trailing space?

They are different lines unless trimming is on, which it is by default. An invisible trailing space is the most common reason a deduplicated list still looks like it has duplicates in it, and it is impossible to see in a text box.

### Does it change the order of what is left?

No. The lines that survive come back in exactly the order they first appeared. If you want them sorted as well, run the result through the line sorter, which keeps the two operations separate and predictable.

## Related tools

- [Sort Lines](https://filetity.com/sort-lines): Sort lines alphabetically, naturally, by length or at random.
- [Compare Text](https://filetity.com/compare-text): Find what changed between two texts, down to the words inside a line.