Select/mark all lines which contain a certain pattern?

Scott Sumner · Nov 7, 2017, 1:57 PM

I didn’t think about sorting

Normally I would not think about sorting either but @perry-sticca had said, “I’d like to mark, sort and then delete all lines…”, so sorting in proposed solutions was fair game. :-)

we would better to always tick the Wrap around option, before a full regex S/R.
Could you investigate, a bit, about it, may be from source code ?

Yes, once I sort out what is being asked. Not unclear, just a lot of info in your post… :-)

Okay, so let’s go…let’s define RA to mean Replace-All for the rest of this posting, so I save typing! Also S/R is Search+Replace!

…concerning my preference, to move back…beginning…before a regex S/R, rather than ticking…Wrap around…it could cause different results, according to the initial location of the caret

Actually these RA related things (move to beginning versus ticking Wrap around) are equivalent as we will see. And it works the same way whether or not it is a regex S/R operation, but you will never see effects like what your example shows with a literal S/R operation…at least I don’t think so.

…we would better to always tick the Wrap around option, before a full regex S/R

I agree if by “full” you mean a desired replacement on the entire file contents in a RA. But it is “better” only because it is easier to tick the checkbox than to move the caret (and maybe lose your concentration on where you were editing in the file).

So here are the relevant parts of the source code for the RA operation called from the button press in the Replace tab window (note: unimportant parts of the code to this analysis have been replaced with ...snip...):

INT_PTR CALLBACK FindReplaceDlg::run_dlgProc(UINT message, WPARAM wParam, LPARAM lParam)
{
    switch (message)
    {
        //...snip...
        case WM_COMMAND :
        {
            //...snip...
            switch (LOWORD(wParam))
            {
                //...snip...
                case IDREPLACEALL :
                {
                    //...snip...
                    if (_currentStatus == REPLACE_DLG)
                    {
                        //...snip...
                        int nbReplaced = processAll(ProcessReplaceAll, &_options);

// prototype:   int processAll(ProcessOperation op, const FindOption *opt, bool isEntire = false, const FindersInfo *pFindersInfo = nullptr, int colourStyleID = -1);
int FindReplaceDlg::processAll(ProcessOperation op, const FindOption *opt, bool isEntire,         const FindersInfo *pFindersInfo,           int colourStyleID)
{
    //...snip...

    CharacterRange cr = (*_ppEditView)->getSelection();
    int docLength = int((*_ppEditView)->execute(SCI_GETLENGTH));

    // Default :
    //        direction : down
    //        begin at : 0
    //        end at : end of doc
    int startPosition = 0;
    int endPosition = docLength;

    //...snip...

    //first try limiting scope by direction
    if (direction == DIR_UP)
    {
        startPosition = 0;
        endPosition = cr.cpMax;
    }
    else
    {
        startPosition = cr.cpMin;
        endPosition = docLength;
    }

    //then adjust scope if the full document needs to be changed
    if (pOptions->_isWrapAround || isEntire) //entire document needs to be scanned
    {
        startPosition = 0;
        endPosition = docLength;
    }

    //then readjust scope if the selection override is active and allowed
    if ((pOptions->_isInSelection) && (((op == ProcessReplaceAll) && (!isEntire))))
    {
        startPosition = cr.cpMin;
        endPosition = cr.cpMax;
    }

    //...snip...

    findReplaceInfo._startRange = startPosition;
    findReplaceInfo._endRange = endPosition;
    return processRange(op, findReplaceInfo, pFindersInfo, pOptions, colourStyleID);
}

We are interested in two things:

the Wrap around checkbox setting, in the code as the _isWrapAround variable
the scope of the search, in the startPosition and endPosition variables

Note that isEntire in this case is always false from the function prototype and the way processAll is called with the RA button press.

So the very key thing is this part: if (pOptions->_isWrapAround .... From here we see that this option being true, i.e., the checkbox being ticked, will cause the start and end of the search to be the top and bottom of the document, respectively. Thus here is the equivalence to moving the caret back to the top of the document and doing a (downward) RA. If the checkbox is not ticked, we have for a downward S/R:

startPosition = cr.cpMin;
endPosition = docLength;

And the “opposite” for an upwards search:

startPosition = 0;
endPosition = cr.cpMax;

In these cases cr.cpMin and cr.cpMax mean one of the following:

cr.cpMin == cr.cpMax == current caret position if no selection is active
cr.cpMin is the lowest position and cr.cpMax is the highest position of the main (most recently created) selection if one or more selections are active (note: In selection not ticked)
cr.cpMin is the lowest position and cr.cpMax is the highest position of the column block if a column block is active (note: In selection not ticked)

So I think at this point we have the answers that were sought: RA with Wrap around operates on the entire file from top to bottom regardless of caret position when the operation is initiated.

But let’s see what an interactive S/R does…

And now here are the relevant parts of the source code for the Find Next operation, which is key to the interactive step-by-step Replace (not RA) operation:

INT_PTR CALLBACK FindReplaceDlg::run_dlgProc(UINT message, WPARAM wParam, LPARAM lParam)
{
    switch (message)
    {
        //...snip...
        case WM_COMMAND :
        {
            //...snip...
            switch (LOWORD(wParam))
            {
                //...snip...
                case IDOK : // Find Next : only for FIND_DLG and REPLACE_DLG
                {
                    // ...snip...
                    FindStatus findStatus = FSFound;
                    processFindNext(_options._str2Search.c_str(), _env, &findStatus);

bool FindReplaceDlg::processFindNext(const TCHAR *txt2find, const FindOption *options, FindStatus *oFindStatus, /* ...snip... */)
{
    // ...snip...

    int docLength = int((*_ppEditView)->execute(SCI_GETLENGTH));
    CharacterRange cr = (*_ppEditView)->getSelection();

    //The search "zone" is relative to the selection, so search happens 'outside'
    int startPosition = cr.cpMax;
    int endPosition = docLength;

    if (pOptions->_whichDirection == DIR_UP)
    {
        //When searching upwards, start is the lower part, end the upper, for backwards search
        startPosition = cr.cpMax - 1;
        endPosition = 0;
    }

    // ...snip...

    int posFind;
    posFind = (*_ppEditView)->searchInTarget(pText, stringSizeFind, startPosition, endPosition);
    if (posFind == -1) //no match found in target, check if a new target should be used
    {
        if (pOptions->_isWrapAround)
        {
            //when wrapping, use the rest of the document (entire document is usable)
            if (pOptions->_whichDirection == DIR_DOWN)
            {
                startPosition = 0;
                endPosition = docLength;
                if (oFindStatus)
                    *oFindStatus = FSEndReached;
            }
            else
            {
                startPosition = docLength;
                endPosition = 0;
                if (oFindStatus)
                    *oFindStatus = FSTopReached;
            }

            //new target, search again
            posFind = (*_ppEditView)->searchInTarget(pText, stringSizeFind, startPosition, endPosition);
        }

        if (posFind == -1)
        {
            if (oFindStatus)
                *oFindStatus = FSNotFound;
            //failed, or failed twice with wrap

This time, because the search/replace is interactive, it is different because it needs to be–the user expects to move from the caret position in his desired direction (if literal search; regex is always downward), so it has to work this way. But again we can see how the _isWrapAround variable affects the search, by doing a true wrap when an end of the file (0 or docLength position) is reached during searching.

Final note: This increased understanding has inspired me to change the look of my Find/Replace/Mark windows, already heavily customized, as follows:

guy038 · Nov 8, 2017, 1:10 PM

Hi, scott-sumner and All,

Thousand thanks for your investigation :-))

You said :

So I think at this point we have the answers that were sought: RA with Wrap around operates on the entire file from top to bottom regardless of caret position when the operation is initiated.

Yes, that’s what I presumed ! So, the easiest method, to perform a neat regex S/R, seems to be :

Paste all the text, which is to be modified, in a new tab
Check the wrap around option and the Regular expression search mode
Add, at beginning of your search regex, the appropriate i and s modifiers, in order to invalidate the previous user choices, for the Match case and . matches newline options
Click on the Replace All button

Et voilà :-D

BTW, Scott, your Find/Replace/Mark dialog seems strongly annotated. No doubt that, after some practice, this window could be simplified a bit !!

Cheers,

guy038

Scott Sumner · Nov 8, 2017, 1:11 PM

@guy038

Your described use-case seems to be for someone that works on regexes for fun (you?!) and does a lot of testing on temporary data. My use, and I suspect most users, is to so S/R in-place on real data. So I wouldn’t necessarily follow the steps you outlined in your most-recent post. I’d just be careful that I knew what I was doing to my data (the main relevant difference for this thread being: Do I want to do a RA from start-to-caret, from caret-to-end, in-selection, or on the entire document).

P.S. You think my annotations could be simplified? Well, sure, probably, especially for the newest thing in yellow… Don’t I get any points for my little regex reminders text, e.g. (?-i), (?s) ? I used to like my \< and \> text but I think that is out-dated and I should probably change to \b given the uncertainty cast by this information. :(

guy038 · Nov 10, 2017, 8:50 PM

Hello, @scott-sumner, and All

You said :

Do I want to do a RA from start-to-caret, from caret-to-end, in-selection, or on the entire document ?

You’re perfectly right. It’s the main point ! So, do you agree, with the following table ?

•-------------•-------------------------------------------------------------•--------------------------------------------------•
|  PREVIOUS   |            OPTIONS of the "Replace** dialog                 |        RANGE of the Search / Replacement         |
|             |-----------------•------------------------•------------------•                                                  |
|  Selection  |  "Wrap around"  |  "Backward direction"  |  "In selection"  |       if click on the "Replace All" button       |
•-------------•-----------------•------------------------•------------------•--------------------------------------------------•
|     NO      |       OFF       |           OFF          |        OFF       |  From  CARET location      to  END of file       |
|             |                 |                        |                  |                                                  |
|     YES     |       OFF       |           OFF          |        OFF       |  From  START of selection  to  END of file       |
•-------------•-----------------•------------------------•------------------•--------------------------------------------------•
|     NO      |       OFF       |           ON           |        OFF       |  From  START of file       to  CARET location    |
|             |                 |                        |                  |                                                  |
|     YES     |       OFF       |           ON           |        OFF       |  From  START of file       to  END of selection  |
•-------------•-----------------•------------------------•------------------•--------------------------------------------------•
|     YES     |       -/-       |           -/-          |        ON        |  From  START of selection  to  END of selection  |
•-------------•-----------------•------------------------•------------------•--------------------------------------------------•
|     -/-     |       ON        |           -/-          |        OFF       |  From  START of file       to  END of file       |
•-------------•-----------------•------------------------•------------------•--------------------------------------------------•

Concerning, my previous post, as a matter of fact, I rather spoke of a method to tell people about regexes, in my future posts ! And I was not claiming, at any moment, for people to follow these steps strictly, of course :-D

Cheers,

guy038

Steve Rothman · Nov 21, 2017, 9:25 PM

@Claudia-Frank This is great!!! I’ve been a casual user of notepad++ for many years and never knew about this. You saved me a ton of time and more importantly a ton of mistakes. Much appreciated!!

Scott Sumner · Sep 6, 2018, 12:33 PM

@guy038

So, do you agree, with the following table ?

Response delayed by 10 months: Yes, I do. :-)

guy038 · Mar 22, 2021, 7:52 PM

Hi, @sasumner, @peterjones and All,

Here is a new version of my previous table, which recapitulates the range of the Replace All, Count and Mark All action , depending of the three options Wrap around, Backward direction and In selection

Many thanks, again to @sasumner which improved the Count and Mark option, which share the same behaviour than the Replace All option ;-))

•-------------•-------------------------------------------------------------•--------------------------------------------------•
|  PREVIOUS   |    OPTIONS in the "Find", "Replace" and "Mark" dialogs      |   RANGE of SEARCH/REPLACE , COUNT or MARK when   |
|             •-----------------•------------------------•------------------•                                                  |
|  Selection  |  "Wrap around"  |  "Backward direction"  |  "In selection"  |   click on "Replace All", "Count" or "Mark All"  |
•-------------•-----------------•------------------------•------------------•--------------------------------------------------•
|     NO      |       OFF       |           OFF          |        OFF       |  From  CARET location      to  END of file       |
|             |                 |                        |                  |                                                  |
|     YES     |       OFF       |           OFF          |        OFF       |  From  START of selection  to  END of file       |
•-------------•-----------------•------------------------•------------------•--------------------------------------------------•
|     NO      |       OFF       |           ON           |        OFF       |  From  START of file       to  CARET location    |
|             |                 |                        |                  |                                                  |
|     YES     |       OFF       |           ON           |        OFF       |  From  START of file       to  END of selection  |
•-------------•-----------------•------------------------•------------------•--------------------------------------------------•
|     YES     |       -/-       |           -/-          |        ON        |  From  START of selection  to  END of selection  |
•-------------•-----------------•------------------------•------------------•--------------------------------------------------•
|     -/-     |       ON        |           -/-          |        OFF       |  From  START of file       to  END of file       |
•-------------•-----------------•------------------------•------------------•--------------------------------------------------•

Best Regards,

guy038