Data Deduplicator and Cleaner
100% PrivateData & JSON ToolsRemove duplicate lines or CSV rows from pasted data and clean what remains.
Runs entirely in your browser. Your input is not uploaded to Ovatly or anywhere else.
- 1Paste your lines, or CSV with a header row.
- 2Choose the data shape: one value per line, or CSV rows.
- 3Set trimming, case sensitivity, blank-line handling and which occurrence to keep.
- 4Press Clean data and review the change log above the result.
Input lines: Ana, ben, ANA, (blank), Ben With trim on, case-insensitive matching, blank removal and keep-first, the output is: Ana ben Change log: 5 input lines, 2 duplicates removed, 1 blank line removed, 2 lines kept.
- CSV mode deduplicates whole rows (all cells compared together) and always keeps the header row. Line mode deduplicates one value per line.
- Case-insensitive matching compares lowercased copies; the kept row or line keeps its original spelling.
- Similarity is exact after the chosen normalizations. Near-duplicates with different punctuation or extra words are not merged.
When duplicates exist, the final occurrence survives in its original position instead of the first one. The set of unique values is the same either way.
About this tool
The Data Deduplicator and Cleaner removes duplicate lines or duplicate CSV rows from pasted data, with control over trimming, letter case and blank lines, and prints a short change log so you can see exactly what was removed and why.
Related tools
Part of Ovatly's free tools in Data & JSON Tools.