Select/mark all lines which contain a certain pattern?

Scott Sumner

Darn it,
Sco{1}t{2}

Perry Sticca

Rather than start a new thread, I’ll ask a very similar question in this one:
I’d like to mark, sort and then delete all lines that end with:

;0

(That’s a semicolon and a zero. A semicolon and a zero might occur elsewhere in the line, but that’s OK - I only care about if they occur at the end of each line.)

I’d like to first sort them to the beginning or end of the file, so I can take a look at them, but then I’d like to delete them. What’s the best way to accomplish this?

guy038

Hello @perry-sticca and All,

The more simple way, what I’m thinking of, would be :

Move back to the very beginning of your file ( Ctrl + Home )
Choose the menu choice Search > Mark…
Tick the option Bookmark line and, may be, the Purge for each search option
Select the Regular expression search mode
Type the regex ;0$, in the Find what line
Click on the Mark All button

=> All the specific lines, ending with the string ;0 are highlighted and bookmarked !

Choose the menu choice Search > Bookmark > Cut Bookmarked Lines
Move to the very end of your file ( Ctrl + End )
You may insert a short line separator
Paste, with Ctrl + V, all the lines ending by the string ;0
After having a look at them…, move back to your line separator
Hit the Ctrl + Shift + End shortcut, to select all these specific lines
Delete them by hitting the Suppr key

Et voilà !

Best Regards,

guy038

gerdb42

@Perry-Sticca

If you want to delete the lines anyway, why mark and sort them first? To delete them right away, search for (?-s)^.*?;0(?:\R|$) and replace with <empty>.

Scott Sumner

@gerdb42

Sometimes you just wanna be sure you are doing the right thing…related maybe…

Scott Sumner

A couple of notes on @guy038’s response:

I might change the regex to (?-s)^.+?;0$ as that will redmark the entire line for matches (easier for the visual inspection that @perry-sticca wants to do)
When I use the Mark feature 99.5%(?) of the time I want it to affect the whole file without having to move the caret to the top first; thus I suggest ticking the Wrap around checkbox rather than the action “Move back to the very beginning of your file (Ctrl + Home)”. Either way works, though.

@guy038, what is the Suppr key? Presume you mean the more-commonly labeled Delete key…

----- New spin on a solution:-----

Maybe more aligned with @perry-sticca 's original desire would be to do this Replace-All (not a Mark) operation that puts a “high-valued” character at the start of the wanted-to-be-matched lines:

Find what zone: (?-s)^(?=.*?;0$)
Replace with zone: \x7f
Wrap around checkbox: ticked
Search mode radio-button: Regular expression

Post replacement, do a Edit (menu) -> Line Operations -> Sort Lines ______ to get the desired lines grouped together at one end of the file.

From there, once confirmed by visual inspection that the lines should really be removed, it is a simple matter to manually do so. Of course, if any are to be retained after this inspection it would be smart to remove the leading \x7f character from each!

guy038

Hi, scott-sumner,

Firstly, you’re right about the key name : the keys Insert and Delete, of an American / English keyboard, are named Inser and Suppr, on a French keyboard !

Secondly, I didn’t think about sorting, Nice idea too ! However, there’s a small difference : the block of lines, ending with ;0, moved at end of file, keeps its initial order with the bookmarks solution, whereas it’s sorted with the \x7F character and sort solution !

Thirdly, concerning my preference, to move back, first, at beginning of file, before a regex S/R, rather than ticking the Wrap around option, it’s just because, I think that, in particular cases, it could cause different results, according to the initial location of the caret, inside the file.

So, I tried to find out a regex, which will demonstrate this fact ! After a while, I came across this one :

SEARCH (?-s)^abc\R(?s).*abc\z

REPLACE EMPTY

and I tested it, against the example text, below, in a NEW tab :

Line 01
abc
Line 02
Line 03
abc
Line 04
--><--
Line 05
abc
Line 06
Line 07
abc

IMPORTANT : The last line abc, of this text, does NOT contain any line break and physically ends the new tab contents !

Then :

Put the caret between the two arrows, in the middle of the text.
Check the Wrap around option
Click on the Find Next button

=> The following block of text is selected :

abc
Line 06
Line 07
abc

Now, click on the Replace button ( Not the Replace All one ! )

=> This block is, of course, deleted and it remains :

Line 01
abc
Line 02
Line 03
abc
Line 04
--><--
Line 05

You get the message Replace: 1 occurrence was replaced. The next occurrence not found. This is logical as, when the process wraps around and moves back to beginning of text, to go on searching, it cannot find, again, a string abc at the very end of the new tab ( part abc\z )

Now, let’s redo the same steps and use, instead, the Replace All button :

Get the original text, by an undo action ( Ctrl + Y )
Put, again, the caret between the two arrows, in the middle of the text.
Verify that the Wrap around option is still checked
Click on the Replace All action

=> Surprisingly, this time, almost all text is deleted and it remains the unique line :

Line 01

In other words, it looks like the process moved back, first, at beginning of file, selected the block between the first string abc and the last string abc ( the last line ) and, finally, deleted it !!

Logically, if process had begun at caret position and had moved back to beginning of file, it should had deleted, only, the following block of text, as the ending string abc does not exist anymore, due to the first replacement !!??

abc
Line 06
Line 07
abc

Consequently, it seems that, when the Regular expression search mode and the wrap around option are, both, ticked, a click on the Replace All button performs the regex S/R from the very beginning of the file till its very end, EVEN IF the initial caret position was anywhere inside the file contents !

If this assertion is exact, we would better to always tick the Wrap around option, before a full regex S/R:-) Could you investigate, a bit, about it, may be from source code ? Many thanks, Scott, by advance !

Cheers,

guy038

Scott Sumner

@guy038 :

I didn’t think about sorting

Normally I would not think about sorting either but @perry-sticca had said, “I’d like to mark, sort and then delete all lines…”, so sorting in proposed solutions was fair game. :-)

we would better to always tick the Wrap around option, before a full regex S/R.
Could you investigate, a bit, about it, may be from source code ?

Yes, once I sort out what is being asked. Not unclear, just a lot of info in your post… :-)

Okay, so let’s go…let’s define RA to mean Replace-All for the rest of this posting, so I save typing! Also S/R is Search+Replace!

…concerning my preference, to move back…beginning…before a regex S/R, rather than ticking…Wrap around…it could cause different results, according to the initial location of the caret

Actually these RA related things (move to beginning versus ticking Wrap around) are equivalent as we will see. And it works the same way whether or not it is a regex S/R operation, but you will never see effects like what your example shows with a literal S/R operation…at least I don’t think so.

…we would better to always tick the Wrap around option, before a full regex S/R

I agree if by “full” you mean a desired replacement on the entire file contents in a RA. But it is “better” only because it is easier to tick the checkbox than to move the caret (and maybe lose your concentration on where you were editing in the file).

So here are the relevant parts of the source code for the RA operation called from the button press in the Replace tab window (note: unimportant parts of the code to this analysis have been replaced with ...snip...):

INT_PTR CALLBACK FindReplaceDlg::run_dlgProc(UINT message, WPARAM wParam, LPARAM lParam)
{
    switch (message)
    {
        //...snip...
        case WM_COMMAND :
        {
            //...snip...
            switch (LOWORD(wParam))
            {
                //...snip...
                case IDREPLACEALL :
                {
                    //...snip...
                    if (_currentStatus == REPLACE_DLG)
                    {
                        //...snip...
                        int nbReplaced = processAll(ProcessReplaceAll, &_options);

// prototype:   int processAll(ProcessOperation op, const FindOption *opt, bool isEntire = false, const FindersInfo *pFindersInfo = nullptr, int colourStyleID = -1);
int FindReplaceDlg::processAll(ProcessOperation op, const FindOption *opt, bool isEntire,         const FindersInfo *pFindersInfo,           int colourStyleID)
{
    //...snip...

    CharacterRange cr = (*_ppEditView)->getSelection();
    int docLength = int((*_ppEditView)->execute(SCI_GETLENGTH));

    // Default :
    //        direction : down
    //        begin at : 0
    //        end at : end of doc
    int startPosition = 0;
    int endPosition = docLength;

    //...snip...

    //first try limiting scope by direction
    if (direction == DIR_UP)
    {
        startPosition = 0;
        endPosition = cr.cpMax;
    }
    else
    {
        startPosition = cr.cpMin;
        endPosition = docLength;
    }

    //then adjust scope if the full document needs to be changed
    if (pOptions->_isWrapAround || isEntire) //entire document needs to be scanned
    {
        startPosition = 0;
        endPosition = docLength;
    }

    //then readjust scope if the selection override is active and allowed
    if ((pOptions->_isInSelection) && (((op == ProcessReplaceAll) && (!isEntire))))
    {
        startPosition = cr.cpMin;
        endPosition = cr.cpMax;
    }

    //...snip...

    findReplaceInfo._startRange = startPosition;
    findReplaceInfo._endRange = endPosition;
    return processRange(op, findReplaceInfo, pFindersInfo, pOptions, colourStyleID);
}

We are interested in two things:

the Wrap around checkbox setting, in the code as the _isWrapAround variable
the scope of the search, in the startPosition and endPosition variables

Note that isEntire in this case is always false from the function prototype and the way processAll is called with the RA button press.

So the very key thing is this part: if (pOptions->_isWrapAround .... From here we see that this option being true, i.e., the checkbox being ticked, will cause the start and end of the search to be the top and bottom of the document, respectively. Thus here is the equivalence to moving the caret back to the top of the document and doing a (downward) RA. If the checkbox is not ticked, we have for a downward S/R:

startPosition = cr.cpMin;
endPosition = docLength;

And the “opposite” for an upwards search:

startPosition = 0;
endPosition = cr.cpMax;

In these cases cr.cpMin and cr.cpMax mean one of the following:

cr.cpMin == cr.cpMax == current caret position if no selection is active
cr.cpMin is the lowest position and cr.cpMax is the highest position of the main (most recently created) selection if one or more selections are active (note: In selection not ticked)
cr.cpMin is the lowest position and cr.cpMax is the highest position of the column block if a column block is active (note: In selection not ticked)

So I think at this point we have the answers that were sought: RA with Wrap around operates on the entire file from top to bottom regardless of caret position when the operation is initiated.

But let’s see what an interactive S/R does…

And now here are the relevant parts of the source code for the Find Next operation, which is key to the interactive step-by-step Replace (not RA) operation:

INT_PTR CALLBACK FindReplaceDlg::run_dlgProc(UINT message, WPARAM wParam, LPARAM lParam)
{
    switch (message)
    {
        //...snip...
        case WM_COMMAND :
        {
            //...snip...
            switch (LOWORD(wParam))
            {
                //...snip...
                case IDOK : // Find Next : only for FIND_DLG and REPLACE_DLG
                {
                    // ...snip...
                    FindStatus findStatus = FSFound;
                    processFindNext(_options._str2Search.c_str(), _env, &findStatus);

bool FindReplaceDlg::processFindNext(const TCHAR *txt2find, const FindOption *options, FindStatus *oFindStatus, /* ...snip... */)
{
    // ...snip...

    int docLength = int((*_ppEditView)->execute(SCI_GETLENGTH));
    CharacterRange cr = (*_ppEditView)->getSelection();

    //The search "zone" is relative to the selection, so search happens 'outside'
    int startPosition = cr.cpMax;
    int endPosition = docLength;

    if (pOptions->_whichDirection == DIR_UP)
    {
        //When searching upwards, start is the lower part, end the upper, for backwards search
        startPosition = cr.cpMax - 1;
        endPosition = 0;
    }

    // ...snip...

    int posFind;
    posFind = (*_ppEditView)->searchInTarget(pText, stringSizeFind, startPosition, endPosition);
    if (posFind == -1) //no match found in target, check if a new target should be used
    {
        if (pOptions->_isWrapAround)
        {
            //when wrapping, use the rest of the document (entire document is usable)
            if (pOptions->_whichDirection == DIR_DOWN)
            {
                startPosition = 0;
                endPosition = docLength;
                if (oFindStatus)
                    *oFindStatus = FSEndReached;
            }
            else
            {
                startPosition = docLength;
                endPosition = 0;
                if (oFindStatus)
                    *oFindStatus = FSTopReached;
            }

            //new target, search again
            posFind = (*_ppEditView)->searchInTarget(pText, stringSizeFind, startPosition, endPosition);
        }

        if (posFind == -1)
        {
            if (oFindStatus)
                *oFindStatus = FSNotFound;
            //failed, or failed twice with wrap

This time, because the search/replace is interactive, it is different because it needs to be–the user expects to move from the caret position in his desired direction (if literal search; regex is always downward), so it has to work this way. But again we can see how the _isWrapAround variable affects the search, by doing a true wrap when an end of the file (0 or docLength position) is reached during searching.

Final note: This increased understanding has inspired me to change the look of my Find/Replace/Mark windows, already heavily customized, as follows:

Imgur

guy038

Hi, scott-sumner and All,

Thousand thanks for your investigation :-))

You said :

So I think at this point we have the answers that were sought: RA with Wrap around operates on the entire file from top to bottom regardless of caret position when the operation is initiated.

Yes, that’s what I presumed ! So, the easiest method, to perform a neat regex S/R, seems to be :

Paste all the text, which is to be modified, in a new tab
Check the wrap around option and the Regular expression search mode
Add, at beginning of your search regex, the appropriate i and s modifiers, in order to invalidate the previous user choices, for the Match case and . matches newline options
Click on the Replace All button

Et voilà :-D

BTW, Scott, your Find/Replace/Mark dialog seems strongly annotated. No doubt that, after some practice, this window could be simplified a bit !!

Cheers,

guy038

Scott Sumner

@guy038

Your described use-case seems to be for someone that works on regexes for fun (you?!) and does a lot of testing on temporary data. My use, and I suspect most users, is to so S/R in-place on real data. So I wouldn’t necessarily follow the steps you outlined in your most-recent post. I’d just be careful that I knew what I was doing to my data (the main relevant difference for this thread being: Do I want to do a RA from start-to-caret, from caret-to-end, in-selection, or on the entire document).

P.S. You think my annotations could be simplified? Well, sure, probably, especially for the newest thing in yellow… Don’t I get any points for my little regex reminders text, e.g. (?-i), (?s) ? I used to like my \< and \> text but I think that is out-dated and I should probably change to \b given the uncertainty cast by this information. :(

guy038

Hello, @scott-sumner, and All

You said :

Do I want to do a RA from start-to-caret, from caret-to-end, in-selection, or on the entire document ?

You’re perfectly right. It’s the main point ! So, do you agree, with the following table ?

•-------------•-------------------------------------------------------------•--------------------------------------------------•
|  PREVIOUS   |            OPTIONS of the "Replace** dialog                 |        RANGE of the Search / Replacement         |
|             |-----------------•------------------------•------------------•                                                  |
|  Selection  |  "Wrap around"  |  "Backward direction"  |  "In selection"  |       if click on the "Replace All" button       |
•-------------•-----------------•------------------------•------------------•--------------------------------------------------•
|     NO      |       OFF       |           OFF          |        OFF       |  From  CARET location      to  END of file       |
|             |                 |                        |                  |                                                  |
|     YES     |       OFF       |           OFF          |        OFF       |  From  START of selection  to  END of file       |
•-------------•-----------------•------------------------•------------------•--------------------------------------------------•
|     NO      |       OFF       |           ON           |        OFF       |  From  START of file       to  CARET location    |
|             |                 |                        |                  |                                                  |
|     YES     |       OFF       |           ON           |        OFF       |  From  START of file       to  END of selection  |
•-------------•-----------------•------------------------•------------------•--------------------------------------------------•
|     YES     |       -/-       |           -/-          |        ON        |  From  START of selection  to  END of selection  |
•-------------•-----------------•------------------------•------------------•--------------------------------------------------•
|     -/-     |       ON        |           -/-          |        OFF       |  From  START of file       to  END of file       |
•-------------•-----------------•------------------------•------------------•--------------------------------------------------•

Concerning, my previous post, as a matter of fact, I rather spoke of a method to tell people about regexes, in my future posts ! And I was not claiming, at any moment, for people to follow these steps strictly, of course :-D

Cheers,

guy038

Steve Rothman

@Claudia-Frank This is great!!! I’ve been a casual user of notepad++ for many years and never knew about this. You saved me a ton of time and more importantly a ton of mistakes. Much appreciated!!

Scott Sumner

@guy038

So, do you agree, with the following table ?

Response delayed by 10 months: Yes, I do. :-)

guy038

Hi, @sasumner, @peterjones and All,

Here is a new version of my previous table, which recapitulates the range of the Replace All, Count and Mark All action , depending of the three options Wrap around, Backward direction and In selection

Many thanks, again to @sasumner which improved the Count and Mark option, which share the same behaviour than the Replace All option ;-))

•-------------•-------------------------------------------------------------•--------------------------------------------------•
|  PREVIOUS   |    OPTIONS in the "Find", "Replace" and "Mark" dialogs      |   RANGE of SEARCH/REPLACE , COUNT or MARK when   |
|             •-----------------•------------------------•------------------•                                                  |
|  Selection  |  "Wrap around"  |  "Backward direction"  |  "In selection"  |   click on "Replace All", "Count" or "Mark All"  |
•-------------•-----------------•------------------------•------------------•--------------------------------------------------•
|     NO      |       OFF       |           OFF          |        OFF       |  From  CARET location      to  END of file       |
|             |                 |                        |                  |                                                  |
|     YES     |       OFF       |           OFF          |        OFF       |  From  START of selection  to  END of file       |
•-------------•-----------------•------------------------•------------------•--------------------------------------------------•
|     NO      |       OFF       |           ON           |        OFF       |  From  START of file       to  CARET location    |
|             |                 |                        |                  |                                                  |
|     YES     |       OFF       |           ON           |        OFF       |  From  START of file       to  END of selection  |
•-------------•-----------------•------------------------•------------------•--------------------------------------------------•
|     YES     |       -/-       |           -/-          |        ON        |  From  START of selection  to  END of selection  |
•-------------•-----------------•------------------------•------------------•--------------------------------------------------•
|     -/-     |       ON        |           -/-          |        OFF       |  From  START of file       to  END of file       |
•-------------•-----------------•------------------------•------------------•--------------------------------------------------•

Best Regards,

guy038