How to compare 2 text files and delete duplicates

JAK

would compare + perform this

Thanks

PeterJones

ComparePlus is great for doing a “diff”, where it shows the difference. But it’s not really for quickly/easily deleting extras

It depends on exactly what you have, but there are some simple ways to delete things from one file that are found in another:

If you have
file1:

apple
banana
carrot
daikon
eggplant
fig
grape

file2:

apple
carrot
jalapeno

If you copy all the contents of file2 and paste them before a --- at the beginning of file1, like:

apple
carrot
jalapeno
---
apple
banana
carrot
daikon
eggplant
fig
grape

Then File > Line Operations > Remove Duplicate Linees will remove the second (or more) occurrence of any line. So the second apple and carrot lines, leaving

apple
carrot
jalapeno
---
banana
daikon
eggplant
fig
grape

then delete everything before and including the --- line, and your file will now have every line that was in file2 removed from file1. (It will also delete duplicates inside file1, so if file1 had started with an extra fig after the grape, only the first fig would remain.)

If that doesn’t do what you want, you will have to give more details about your rules.

Also, if you want an easy way to delete everything before and including the --- line, use FIND WHAT = (?s)\A.*^---\R, REPLACE = empty, Wrap Around = Checkmarked, Search Mode = Regular Expression, Replace All

pnedev

Hi @JAK ,

You can use ComparePlus but from the menu choose Find Unique Lines command. It will mark all unique lines in a file (that are not found in the other file).
Then use Diff Visual Filters... to hide all diffs (this will hide all unique lines). Don’t worry if there is still one visible diff line on top - first document line cannot be hidden but it is not a problem for the next operation.
Then select the file from which you’d like to remove duplicating lines (set the focus/caret in that file) and execute Delete all/selected visible lines from the ComparePlus plugin menu.
This will do what you are trying to accomplish.

BR