@PeterJones said:
It’s not ideal, but without some magical regex manipulation to make it less backtracky (hints in the complexity section), you’re not likely to find a regex that doesn’t become invalid for you.
@blake-g.-barrington , @PeterJones:
I can’t say for certain — much of Boost.regex (the regular expression engine Notepad++ uses) is still a black box to me — but I think it’s unlikely that any regex can do what is requested without getting the complexity error on large files. As best I can tell, Boost.regex is a bit “trigger happy” with that warning. It doesn’t really know whether the expression is trapped in a loop that isn’t progressing, it just knows that it seems to be scanning the same text over and over again. For any regex to solve this problem, it has to scan the same text over and over: for every line, it needs to check every following line up to the end of the file, unless it finds a match sooner. Given that and what I have observed about Boost.regex, I would not recommend spending a lot of time trying to improve the regular expression.
As @PeterJones noted, if you don’t need to preserve the order of lines in the files, the solution is simple: sort the file first.
If you do need to preserve order, the next question is whether this is a one-time thing you need to do to solve a specific problem, or a recurring task you’ll need to perform against various files in the future.
If it’s a recurring task, ditch the regular expressions and write a script. Python is probably the best language for scripting in Notepad++, but I’m not very familiar with Python, so I’ll leave it to someone else to explain that approach if that is what you need.
If this is a one time thing, then I would suggest:
Insert line numbers at the beginning of each line.
Make a zero-width rectangular selection following the line numbers and sort.
Adapt your regular expression to find adjacent lines that contain duplicate text following the line number and make a replace expression that adds a marker character following the line number: a character that isn’t used otherwise in the file.
Sort on the line numbers to put things back in order.
Mark lines that contain the marker character.
Use Replace All to delete the marker character.
