Remove duplicate lines from text files.
Utility that removes duplicate lines from the textual file.
Uses very little memory.
When you have constraints like:
Large \n delimited wordlist -- one word per line. Size of files range from 2GB to as large as 10GB.
Need to remove any duplicate lines.
The process may sort the list during the course of removing the duplicates but not required.
There is enough space on the partition to hold the new unique wordlist outputted.
Then this is the solution
Operating systems:
Windows 98/Me/NT/2000/XP/2003/Vista/Server 2008/7/8/10
Date:
25.09.2015
Related Software
