Remove Duplicate Lines

hasan123

Hi,
Is this possible to remove duplicate lines in a text file/list with Notepad++ ?

To change this:

aaaaaaaaaaaa
bbbbbbbbbbbb
aaaaaaaaaaaa
aaaaaaaaaaaa
cccccccccccc
dddddddddddd
eeeeeeeeeeee
ffffffffffff
cccccccccccc
gggggggggggg
hhhhhhhhhhhh
gggggggggggg
gggggggggggg
cccccccccccc

To

aaaaaaaaaaaa
bbbbbbbbbbbb
cccccccccccc
dddddddddddd
eeeeeeeeeeee
ffffffffffff
gggggggggggg
hhhhhhhhhhhh

Alan Kilborn

@hasan123

Maybe. Key question: Can your lines be sorted without losing meaning?

hasan123

@Alan-Kilborn

Sorting is not important for my case.

Alan Kilborn

@hasan123

Then your best option is to sort the lines using one of the Sort commands, then use the Remove Consecutive Duplicate Lines command. All are on the Edit > Line Operations menu.

hasan123

@Alan-Kilborn

Yes, Thanks for the info.

rinku singh

Use Remove Duplicate Lines plugin

https://youtu.be/OP_VqD7lCGw

Prahlad-Makwana4145

Hello, @hasan123

Based on your problem please follow steps to remove duplicate lines with notepad++.

Note :- Notepad++ is a feature rich text editor but does not have an integrated feature to remove duplicate lines, however, the option can be added in via the external plugin system Notepad++ uses. Although some are already included the TextFX plugin is not installed by default and needs to be added manually.

Step 1:- Go to the Plugins menu.
Step 2:- In plugins menu select Plugin Manager > Show Plugin Manager.
Step 3:- Find the TextFX Characters plugin from the list.
Step 4:- Check the box, click Install and restart the program.
Step 5:- After installation, select all the text (Ctrl+A), go to TextFX > TextFX Tools and select the option +Sort outputs only UNIQUE (at column) lines.
Step 6:- Go back to the same TextFX Tools menu and either click on Sort lines case sensitive (at column) or Sort lines case insensitive (at column). This will remove the duplicate lines.

I hope above information will be useful for you.
Thank you.

Alan Kilborn

@Prahlad-Makwana4145

I would not advise people to use the TextFX plugin as it is old and support is questionable.