Problems with Math FAQ's Columns++ entry

Coises

@PeterJones said in Pythonscript plugin / console not working:

I have now updated the Math Replacement FAQ to include a section about the Columns++ Plugin, because that is definitely a viable alternative to scripting plugin

I think there is a bit of a problem with your example, though. The example replaces the number after “num_1=” with a counter, starting from 1 and incrementing by 1, and the number after “num_2=” with a counter starting from 100 and incrementing by 2. If you wanted — as your description reads, to me — to double the existing value after “num_1=” and add 98, you would need:

Find what: num_1=(\d+), num_2=\d+
Replace with: num_1=$1, num_2=(?=98+2*reg(1))

PeterJones

@Coises ,

(I moved to a separate discussion, because it had nothing to do with that other discussion)

I think there is a bit of a problem with your example, though

No, there’s a bit of a problem with my description of the example. I apparently didn’t understand what match was doing in the original regex. :-) (update: I’ve “fixed” it (or at least hopefully made it less wrong) until I can get answers to some of the below)

I had understood (?=match) to somehow be the contents of the match… but with your use of the word “counter” in the description, I am now understanding it to be the “match index”, in other words, “a counter of successful matches”. Is this correct?

in this new example, what’s the difference between $1 and reg(1) ? Is the first the “string” version and the second the “numeric” version?

From the Columns++ page, I couldn’t find any description of those syntaxes. If you have it documented somewhere, please share a link. If not, could you give a quick table of a few simple tokens that I could add to the FAQ to get people started with math? Something like:

regex	desc
`(?=match)`	Essentially, the match number (a counter of successful matches)
`$1`	The string contents of match #1 (like normal regex replacement)
`reg(1)`	The numerical value of match #1 ???
…	… add more if there are more critical tokens

PeterJones

@PeterJones said in Math FAQ: columns++:

could you give a quick table of a few simple tokens that I could add to the FAQ to get people started with math

… and/or provide some examples to solve the same tasks (or similar tasks) to those that we solved with PythonScript in the FAQ: for any that’s possible from “add_1”, “round to integer or to fixed-digits-after-decimal”, “auto-increment”, “loop through N values”, “search for N >= 25”, or others like what is shown in the FAQ, or examples that will show the power of Columns++ math regex. :-)

Coises

@PeterJones

Documentation is in the Formulas section of the help. Scroll past the first table (which is for the Calculate command), to the paragraph:

When formulas are used in regular expression replacements for the Search dialog, the formula given is evaluated once for each match. Formulas in regular expression replacements are specified as:
(?=formula) or (?=format:formula)

The next two tables describe the variables and functions in search (match, line, this, reg, sub and last) and the optional format string (used to control things like leading zeros and decimal places).

The remainder of the Formulas topic describes features common to formulas in either context (Calculate or Search).

The replacement string in regular expression mode is identical to any other regular expression replacement string except that the sequence (?=…) has the special meaning of enclosing a formula substitution.

(Strictly speaking, the match is found, then formula substitutions are computed, then the replacement string is constructed using the substituted values, and finally the replacement is performed. In odd cases this might matter, since the formula substitution could form part of a special regular expression replacement sequence; for example, ${(?=line%3+1)} would be replaced with the nth capture group, where n is one greater than the remainder of dividing the line number by three. That detail is not yet in the documentation.)

I realize this is all rather daunting, and I will attempt to give some less overwhelming explanation in another comment. The rather annoying learning curve to get started with this is something I dislike, but could not find a way to avoid.

Coises

@PeterJones said in Math FAQ: columns++:

I had understood (?=match) to somehow be the contents of the match… but with your use of the word “counter” in the description, I am now understanding it to be the “match index”, in other words, “a counter of successful matches”. Is this correct?

in this new example, what’s the difference between $1 and reg(1) ? Is the first the “string” version and the second the “numeric” version?

$1 is an ordinary capture group reference, the same as in any regular expression replacement.

(?=…) is a formula replacement. It is replaced by whatever is computed by the enclosed formula as applied to each match.

match is a variable which means “counting from one and including the current match, the number of times the regular expression has matched.” (this is the numeric value of the entire match — only useful if the entire match is, in fact, a number.)

reg(n) is a function which returns the numeric value of the nth capture group.

In a sense you are correct that $1 is a string, while reg(1) must be a number. The essential difference, though, is that $1 is part of a regular expression replacement string, while reg(1) is part of a formula, which has entirely different syntax and rules than regular expression replacement strings, and can only be used within a (?=…) group.

PeterJones

@Coises ,

Do you think this is reasonable for a Quick Start?

All Formula Replacements go inside of a (?=...) wrapper: the math / formula goes in place of the ... . Any of the expressions below can go inside a Formula Replacement wrapper, as well as any standard math using + - * / for the addition, subtraction, multiplication, and division operators.

match ⇒ This variable is the number of times the regular expression has matched (essentially, a counter for matches). So a replacement of (?=match) will replace with 1 on the first match, 2 on the second match, 3 on the third, etc.

reg(ℕ) ⇒ This grabs the numerical value of the ℕth capture group (similar to $ℕ or ${ℕ} in a normal regex).

this ⇒ This variable is the numeric value of the entire match — only useful if the entire match is, in fact, a number.

For all the details, see the Columns++ Documentation > Formulas – it includes other functions and operators for getting and manipulating data from your file, as well as other math functions and operators

Coises

@PeterJones said in Problems with Math FAQ's Columns++ entry:

some examples to solve the same tasks (or similar tasks) to those that we solved with PythonScript in the FAQ

Here are some examples of math using Columns++ Search in indicated region.

When you want to do math, you need to select the numbers as capture groups in your search. For example, if you have text like:

time=3, distance=36
time=7, distance=280
time=4, distance=24

and you want to end up with:

speed=12, time=3, distance=36
speed=40, time=7, distance=280
speed=6, time=4, distance=24

you could use:

Find what : time=(\d+), distance=(\d+)

with:

Replace with : speed=(?=reg(2)/reg(1)), $&

Select the entire document, or the part of the document you want to search. (Rectangular selections are supported.) Select Search… from the Columns++ plugin menu, enter the find and replace strings, and be sure Regular expression is checked. Then use Replace or Replace All as desired.

Add a value to each match (or subtract, or multiply, or divide, or other f(x)):

Use the method above; make sure the find string isolates the number you want to use as a capture group, replace the formula (reg(2)/reg(1)) with whatever computation you require, and adjust the rest of the replacement string as needed.

Beware of using a capture group like $1 immediately followed by a (?=…) substitution; at present, this does not work as expected. Instead, write the capture group as ${1} to avoid unintended results.

Round off values to the nearest XXX

For the most common cases, you can use the format string in a formula substitution to control this:

(?=1:reg(1)) — replaces with the numeric value of the first capture group rounded to an integer
(?=4:reg(1)) — … rounded to an integer and padded: 0003, 0234, 5612, 98765
(?=1.2:reg(1) — … two decimal places: 4.37, 16.20, 0.07
(?=0.2:reg(1) — … two decimal places: 4.37, 16.20, .07
(?=1.-4:reg(1) — … up to four decimal places: 2, 3.7, 9.1423, 0.432
(?=1.0-4:reg(1) — … up to four decimal places: 2., 3.7, 9.1423, 0.432

For more complex cases, use the round function with appropriate math:

(?=round(reg(1)/25)*25) — round to the nearest multiple of 25
(?=ceil(reg(1)/10)*10) — round up to a multiple of 10

Add hard line numbers to the current file

Find what : ^
Replace with : (?=6:line)

to add line numbers at the left, zero-padded to six digits; use:

Find what : ^
Replace with : L(?=3:line*5)-

to add line numbers L005-, L010-, L015-, etc.

Replace with a simple counter (i.e., renumbering)

To change:

zone_235 zone_249 zone_193 zone_151
zone_207 zone_172 zone_5 zone_221
zone_2 zone_270 zone_186 zone_228

to:

zone_1 zone_2 zone_3 zone_4
zone_5 zone_6 zone_7 zone_8
zone_9 zone_10 zone_11 zone_12

use:

Find what : zone_(\d+)
Replace with : zone_(?=match)

To get:

zone_60 zone_65 zone_70 zone_75
zone_80 zone_85 zone_90 zone_95
zone_100 zone_105 zone_110 zone_115

use:

Replace with : zone_(?=55+5*match)

Coises

@PeterJones said in Problems with Math FAQ's Columns++ entry:

Do you think this is reasonable for a Quick Start?

Yes, it is.

Coises

I said in Problems with Math FAQ's Columns++ entry:

Beware of using a capture group like $1 immediately followed by a (?=…) substitution; at present, this does not work as expected. Instead, write the capture group as ${1} to avoid unintended results.

This is fixed in version 0.7.3-alpha, so there is no further need for this warning.