Eliminating duplicate (identical) lines

Ed Bernasconi

Hell!
Is there any way that will find and eliminate duplicate lines?

Example:
Machine 1
Machine 2
Machine 2
Machine 2
Machine 3
Machine 4
Machine 4
Machine 5

Once cleaned up, should give:
Machine 1
Machine 2
Machine 3
Machine 4
Machine 5

Thank you for any suggestion!
Ed

guy038

Hello, Ed,

The suppression of all the duplicate lines, in a pre-sorted file, can be easily obtained with a Search/Replacement, in Regular expression mode !

Open your file, containing the sorted list of items
Open the Replace dialog ( CTRL + H )
In the Search what: field, type (?-s)(^.+\R)\1+
In the Replace with: field, type \1
Check the Regular expression radio button
Click on the Replace All button

Et voilà !!

Notes :

The (?s) in-line modifier ensures you that the special regex dot character will match standard characters, only, even if you, previously, checked the . matches newline option !
Then, the part ^.+\R matches all the characters ( .+ ) of any non-empty line, between the beginning of line (^ ) and its End of Line character(s) ( \R ), included
So, the part (^.+\R), enclosed by round brackets, simply stores any complete line contents, as group 1
Finally the part \1+ tries to match any positive amount of subsequent identical lines, following the previous line
And if a overall match can be found, all that block of identical lines, is just replaced by the group 1 ( \1 ), that is to say, ONE copy of that block

Best Regards,

guy038