Wildcard in replace field

daxliniere

Hi all!

I need to convert punctuation in lines like below from:

,Stop and sleep.,04:00:08
,Grab a cab.,05:21:12

to

,Stop and sleep.,04:00.08
,Grab a cab.,05:21.12

It’s simply changing the second last character of the line from : to .

I got a RegEx search to work (:.*:), but RegEx replace was beyond me and I haven’t been able to find anything on a forum that matched my need.

I tried replace :.: with :.. (plus a few variants with quotation marks), but nothing worked for me.
What is the correct syntax for this, please?

Thank you in advance!
Dax.

Ekopalypse

@daxliniere

there are a few characters which do have special meaning in regex.
If you need to search or replace them literally you need to escape them by using a leading \ in front of the “special char”.
Means, \. would use the dot instead of any char.

Alan Kilborn

Hmm, I might go a bit beyond what the OP did and search for (\d\d:\d\d):(?=\d\d) and replace with ${1}.

Alan Kilborn

@Alan-Kilborn said in Wildcard in replace field:

and replace with ${1}.

Note that my “period” at the end is actually part of the replace expression, and isn’t a “sentence ender”.

Maybe I should have said it this way:

…and replace with ${1}..

:-)

PeterJones

aside to @Ekopalypse :

remember we’re on boost 1.70 now, not 1.55. The updated boost docs are here. (edit: they don’t change the list of which characters are special, but providing the modern link is probably better in the long run)

aside to @Alan-Kilborn ,

Note that my “period” at the end is actually part of the replace expression, and isn’t a “sentence ender”.

in such circumstances, the notation of

FIND = (\d\d:\d\d):(?=\d\d)
REPLACE = ${1}.
SEARCH MODE = regular expression

is less likely to confuse or cause ambiguity. Or you could use an optional output group or escape-sequence, such as

REPLACE = ${1}(.)
or
REPLACE = ${1}\x2E

both of which are less ambiguous as to the what goes in the replacement field, at the expense of being less understandable about what will actually output. :-)

Ekopalypse

aside to all :-D

ignore the groups and do :(?=\d\d$)
and replace with \.

Alan Kilborn

@Ekopalypse said in Wildcard in replace field:

ignore the groups and do :(?=\d\d$)

Well, it depends upon what you want to “grab onto” mentally and decide what is really important. Your expression chose to grab onto the end-of-line as important.
Mine tried to grab onto the more general concept of a 00:00:00 timestamp occurring in data somewhere as being important.
Which is better? Dunno.
Perhaps whatever gives the OP the best “walk away” feeling; sadly we’ll never know the answer to that. :-)

Alan Kilborn

@PeterJones

Wow, Peter picks on my notation. :-(
Just kidding, I know you’re really not picking on it. :-)

But… sometimes I present like this:

Open the Replace dialog by pressing Ctrl+h and then set up the following search parameters:
Find what box: (\d\d:\d\d):(?=\d\d)
Replace with box: ${1}.
Search mode radiobutton: Regular expression
Wrap around checkbox: ticked
Option checkboxes not mentioned are typically not important to the operation, but should in general be unticked.
Then press the Replace All button.

And nobody has ever said “Hey, that style looks nice.” :-(
No no no no no, not looking for any misplaced praise. :-)
But, the time I don’t present like that, I get told a method for “better presenting”. :-(
JUST KIDDING!

guy038

Hello @peterjones, @ekopalypse, @alan-kilborn, @daxliniere, and All,

@peterjones, you said :

remember we’re on boost 1.70 now, not 1.55. The updated boost docs are here.

Where did you see that the N++ regex engine migrates from Boost v1.55.0 to Boost v1.70.0 ?!

I’ve tried to see any mention of this, in change.log files of recent N++ versions, without any result !

Anyway, I compared the HTML contents of :

http://www.boost.org/doc/libs/1_55_0/libs/regex/doc/html/boost_regex/syntax/perl_syntax.html

and

http://www.boost.org/doc/libs/1_73_0/libs/regex/doc/html/boost_regex/syntax/perl_syntax.html

The main difference concerns the addition of the Backtracking Control Verbs functionality which, due to lack of testing, does not seem to be of prime importance, but I’m probably wrong !

Here are the list of changes / additions between our Boost version v1.55.0 and the last version v1.7.3 of the regex library :

----------------------------------------------------------------------------------------------------------------------------------------------------------------

Right BEFORE "Wilcard" paragraph, ADDITION of the line :


Other characters are special only in certain situations - for example ] is special only after an opening [.

----------------------------------------------------------------------------------------------------------------------------------------------------------------

Right BEFORE the sentence "It is an error to use a repeat operator....", ADDITION of the line : 


Note that the "{" and "}" characters will treated as ordinary literals when used in a context that is not a repeat: this matches Perl 5.x behavior. For example in the expressions "ab{1", "ab1}" and "a{b}c" the curly brackets are all treated as literals and no error will be raised.

-----------------------------------------------------------------------------------------------------------------------------------------------------------------

Right AFTER the FIRST sentence, in "Backreferences" paragraph, CHANGE of the REGEX :


^(a*)[^a]*\1$    ( instead of ^(a*).*\1$ )

-----------------------------------------------------------------------------------------------------------------------------------------------------------------

Right BEFORE the "Operator precedence" paragraph, ADDITION of the "Backtracking Control Verbs" paragraph :


Backtracking Control Verbs

This library has partial support for Perl's backtracking control verbs, in particular (*MARK) is not supported. There may also be detail differences in behaviour between this library and Perl, not least because Perl's behaviour is rather under-documented and often somewhat random in how it behaves in practice. The verbs supported are:

    (*PRUNE) Has no effect unless backtracked onto, in which case all the backtracking information prior to this point is discarded.
    (*SKIP) Behaves the same as (*PRUNE) except that it is assumed that no match can possibly occur prior to the current point in the string being searched. This can be used to optimize searches by skipping over chunks of text that have already been determined can not form a match.
    (*THEN) Has no effect unless backtracked onto, in which case all subsequent alternatives in a group of alternations are discarded.
    (*COMMIT) Has no effect unless backtracked onto, in which case all subsequent matching/searching attempts are abandoned.
    (*FAIL) Causes the match to fail unconditionally at this point, can be used to force the engine to backtrack.
    (*ACCEPT) Causes the pattern to be considered matched at the current point. Any half-open sub-expressions are closed at the current point.

-----------------------------------------------------------------------------------------------------------------------------------------------------------------

Inside the "Operator precedence" paragraph, CHANGE of the REGEX :


2. Escaped characters \    ( instead of [^] )

-----------------------------------------------------------------------------------------------------------------------------------------------------------------

Best Regards,

guy038

Alan Kilborn

@guy038 said in Wildcard in replace field:

Where did you see that the N++ regex engine migrates from Boost v1.55.0 to Boost v1.70.0 ?!

Well, the build instructions for Notepad++ mention Boost 1.70…

But I suppose that isn’t definitive.

It would be nice if maybe the Debug Info for Notepad++ included both the Scintilla and Boost version numbers it uses?

PeterJones

@Alan-Kilborn said in Wildcard in replace field:

@guy038 said in Wildcard in replace field:

Where did you see that the N++ regex engine migrates from Boost v1.55.0 to Boost v1.70.0 ?!

Well, the build instructions for Notepad++ mention Boost 1.70…

It’s also mentioned in here:
https://github.com/notepad-plus-plus/notepad-plus-plus/blob/master/scintilla/boostregex/BoostRegExSearch.cxx#L1-L10

Ah, okay, @xylographe made https://github.com/notepad-plus-plus/npp-usermanual/issues/47, which came as a result of BUILD.md.

It would be nice if maybe the Debug Info for Notepad++ included both the Scintilla and Boost version numbers it uses?

That would be nice. But unfortunately, none of us can be the one to put in the issue, otherwise it is likely to be ignored.

Alan Kilborn

@PeterJones said in Wildcard in replace field:

That would be nice.

It would probably also be something a dev would have to remember to do by hand when it changes, instead of being automated, so it might be a poor idea anyway (I don’t trust the hoomans to remember to update these types of things).

guy038

Hi, @alan-kilborn, @peterjones, @ekopalypse, and All,

Alan and Peter, thank you so much for these clarifications ;-)

Although the last Boost regex 1.7.3 version is not used and that Notepad++ is build with Boost v1.7.0 since N++ v7.8.0, the differences between the search regex documentation of Boost versions v1.5.5 and v1.7.0, mentioned in my previous post, are identical and are still valid ;-))

So, the good news is that our new regex library does support the Perl’s backtracking control verbs. Woooow !! I’m quite eager to investigate this new side of regular expressions:-)) I will keep you informed of the interest of these new functions and will try to provide you with relevant examples for a better understanding…

As a result, I have also updated the FAQ contents !

Now, while browsing the Boost C++ libraries website, I came across a thorny problem : when we speak of Boost v1.70.0, I think that it’s an abuse of language ! Because the strings 1.55.0 and 1.70.0 refer to a documentation version and NOT to a library version :-( Indeed :

Formerly, we were using the Boost-Regex library from the Boost-1.54.0 version of the Boost C++ libraries and we referred to its Boost 1.55.0 library documentation
Since Notepad v7.7, we’re using the Boost-Regex-5.1.3 library from the Boost-1.64.0 version of the Boost C++ libraries and we refer to its Boost 1.70.0 library documentation

Refer to all these links, below. It’s a bit tricky !? Compare the different links, two by two

https://www.boost.org/doc/libs/1_55_0/

https://www.boost.org/doc/libs/1_70_0/

https://www.boost.org/doc/libs/1_55_0/libs/regex/doc/html/index.html

https://www.boost.org/doc/libs/1_70_0/libs/regex/doc/html/index.html

https://www.boost.org/doc/libs/1_55_0/libs/regex/doc/html/boost_regex/background_information/history.html

https://www.boost.org/doc/libs/1_70_0/libs/regex/doc/html/boost_regex/background/history.html

Could you confirm my guesses ? TIA !

Cheers,

guy038

P.S :

The last version uses the Boost-Regex-5.1.4 library from the Boost-1.72.0 version of the Boost C++ libraries and refers to the Boost 1.73.0 library documentation

See :

https://www.boost.org/doc/libs/1_73_0/

https://www.boost.org/doc/libs/1_73_0/libs/regex/doc/html/index.html

https://www.boost.org/doc/libs/1_73_0/libs/regex/doc/html/boost_regex/background/history.html

daxliniere

A big thank you to everyone who replied to my question offering help. @Alan-Kilborn’s suggestion seems to work perfectly.

All the best to this great community!
-Dax.

daxliniere

Hey all!
I have no idea why, but for some reason, this solution has stopped working…
Seriosuly, I have no idea what’s going on, but (\d\d:\d\d):(?=\d\d) no longer finds a string of text that looks like 0:10:02 using RegEx.

Is it possible a recent update to NPP might have broken this or changed syntax??
I’m so confused.

Thanks in advance!
-Dax.

astrosofista

Hi @daxliniere

The quoted regex matches strings with two leading numbers, as “00:12:12”.
In order to match strings with only one leading number, just remove a “\d” from the expression, as (\d:\d\d):(?=\d\d).

Have fun!

daxliniere

@astrosofista Thank you so very much!! It works exactly as before.
I have no clue what could have changed, but this has solved the problem.

I really appreciate your time on this. I was in the middle of a job and got stuck on this. Was comtemplating manual correction of ~400 lines!

Have a great weekend. :)
-Dax.

daxliniere

AHHH!!! I understand it now!

The data I had been receiving previously had trailing zeros so every line had xx:xx:xx formatting. It seems whoever was typing this started dropping the trailing zeros.
Is there a more robust form of (\d:\d\d):(?=\d\d) needed?

PeterJones

@daxliniere ,

You say “trailing”, but the regex shown (as modified by @astrosofista) has removed the requirement for leading zeros. To make it match both, I’d do (\d{1,2}:\d\d):(?=\d\d), which will match whether there is one digit or two.

But if you give an example of “with” or “without” “trailing zeros”, we can be more confident of what you’re really asking for.

Thomas 2020

@PeterJones
instead:
(\d{1,2}:\d\d):(?=\d\d)

maybe that’s better?
(:\d{1,2}):

the shorter the pattern, the better