[New Plugin] MultiReplace

Thomas Knoefel

@Coises said in [New Plugin] MultiReplace:

I didn’t aim for simplicity, though. I gave up on that early (while I was still implementing column calculations) and decided instead to find a free and open source library of some sort that would avoid having to write my own expression parser. I chose ExprTk ; so far, it is working well.

I’m aiming for something more basic as it is mainly a Replacement Tool which is not deeply rooted in complex calculations but can handle basic counting operations. I’m trying to avoid using external Libraries right now (as long as possible)

This is my proposed structure:

Calculation Block

$calc(VARIABLE_NAME = EXPRESSION)

Manages mathematical operations with a focus on simplicity.
$calc(NUM = $CNT + 5)
2. Conditional Block

$if(CONDITION, THEN_OUTPUT, ELSE_OUTPUT)

Allows for the inclusion of calculated variables within conditions or outputs.
$if($NUM > 10, “Value $NUM is greater”, “Value $NUM is smaller”)
3. String Block

$str(STRING_EXPRESSION)

Efficiently merges calculated values with strings for the final replacement string. Can be used instead of an if Conditional Block. So either if or str to pass a replacement string.
$str(“Total: $NUM key”)

One potential challenge might be the extraction of regex groups by numbers, like $1, $2, etc., before sending them to the replacement string. Scintilla doesn’t support this with its Regex searches. But actually, this isn’t my primary concern.

@guy038, thank you for highlighting these facts. They’ve provided valuable insights. The NUM example, in particular, inspired the concept of new Command Blocks.

I hope that Command Blocks give the syntax more transparency in reading and even more robustness for parsing.

guy038

Hello, @coises, @thomas-knoefel,

Oh…, I’m terribly sorry, @coises : I forgot your valuable Columns++ plugin which already provides such a solution to my request. A special mention, too, for your excellent documentation support ;-))

To be honest with you both, I’ve been following, for a while, the Notepad++'s development and its plug-ins, in a rather theorical way, on two machines :

An old Win XP machine, with Notepad++ v7.9.2 32-bit x86
A recent Win 10 laptop, with Notepad++ v8.5.4 64-bit x64, located on an USB drive

On these two versions, I just add the BetterSelection, ComparePlus and Elastic Tabstops plugins and the main script plugins : LuaScript, NppExec and Python Script

I’m still in the process of cleaning up my old computer, before transferring “clean” data on my new laptop. ( Don’t worry, @alan-kilborn, I’m nearing the end : the 785 and 792 still need to be reviewed in depth, as well as the deletion of multiple copy files… before the definitive migration ! )

In addition, I’m presently involved in managing all my holiday photos, since June. In short, for all these reasons, I have not installed the last N++ version yet, as well as your two plugins Columns+++ and MultiReplace :-((

So, the best thing I can do is to move on, once and for all, to the 21st century, which will allow me to make more accurate suggestions regarding your two excellent plugins !!

Best Regards,

guy038

Coises

@Thomas-Knoefel said in [New Plugin] MultiReplace:

One potential challenge might be the extraction of regex groups by numbers, like $1, $2, etc., before sending them to the replacement string. Scintilla doesn’t support this with its Regex searches. But actually, this isn’t my primary concern.

Unless I misunderstand what you mean, SCI_GETTAG does this.

Thomas Knoefel

@Coises Wow, you’re always full of good ideas! :-) Thanks! I hope I can share some of mine with you too. :-))

Thomas Knoefel

The first functional release is now available for testing and can be downloaded here.

While I’ve made some changes from the original concept, it still closely aligns with that initial vision. This Dynamic Substitution is compatible with all search settings of Search Mode, Scope, and the other options.

By activating the 'Use Variables’ checkbox, variables associated with specified strings can be employed, allowing for conditional and computational operations within the replacement string.

Variables Overview

Variable	Description
CNT	Count of the detected string.
LINE	Line number where the string is found.
APOS	Absolute character position in the document.
LPOS	Relative line position.
COL	Column number where the string was found (CSV-Scope option selected).
CAP1, CAP2, …	Correspond to capture groups $1, $2, … in regex patterns. They can be used for calculations or as strings.
	Note: Both dot and comma can be used as decimal separators in Capture Variables but thousands separators are not supported.

Command Overview

String Composition

.. is employed for concatenation.
E.g., "Detected "..CNT.." times."

set(strOrCalc)

Outputs strings or numbers directly.

Example	Result (assuming LINE = 5, CNT = 3)
`set("replaceString"..CNT)`	“replaceString3”
`set(LINE+5)`	“10”

cond(cond, trueVal, [falseVal])

Implements if-then-else logic, or if-then if falseVal is omitted.

Example	Result (assuming LINE = 5)
`cond(LINE<=5 or LINE>=9, "edge", "center")`	“edge”
`cond(LINE<3, "Modify this line")`	(Original text remains unchanged)
`cond(LINE<10, cond(LINE<5, cond(LINE>2, "3-4", "0-2"), "5-9"), "10+")`	“5-9” (Nested condition)

fmtN(num, maxDecimals, fixedDecimals)

Formats numbers based on precision (maxDecimals) and whether the number of decimals is fixed (fixedDecimals being true or false).

Note: The fmtN command can exclusively be used within the set and cond commands.

Example	Result
`set(fmtN(5.73652, 2, true))`	“5.74”
`set(fmtN(5.0, 2, true))`	“5.00”
`set(fmtN(5.73652, 4, false))`	“5.7365”
`set(fmtN(5.0, 4, false))`	“5”

Operators

Type	Operators
Arithmetic	`+`, `-`, `*`, `/`, `^`, `%`
Relational	`==`, `~=`, `<`, `>`, `<=`, `>=`
Logical	`and`, `or`, `not`

Engine Overview

MultiReplace uses the Lua engine, allowing for Lua math operations and string methods. Refer to Lua String Manipulation and Lua Mathematical Functions you can use for more information.

Thomas Knoefel

I have incorporated two additional variables which I find beneficial:

LCNT: Count of the detected string within the line.
MATCH: Contains the text of the detected string, in contrast to CAP variables.

The complete documentation can be found here.

Should anyone have ideas for additional variables that are missing and should be included into this option, feel free to share.

Below are some examples demonstrating how to utilize the “Use Variables” option.

Find in:	Replace with:	Description/Expected Output	Regex	Scope CSV
`;`	`cond(LCNT==5,";Column5;")`	Adds a 5th Column for each line into a `;` delimited file.	No	No
`(\d+)`	`set("$1€ The VAT is: ".. (CAP1 * 0.15).."€ Total with VAT: ".. (CAP1 + (CAP1 * 0.15)).."€")`	Finds a number and calculates the VAT at 15%, then displays the original amount, the VAT, and the total amount. E.g., `50` becomes `50€ The VAT is: 7.5€ Total with VAT: 57.5€`	Yes	No
`---`	`cond(COL==1 and LINE<3, "0-2", cond(COL==2 and LINE>2 and LINE<5, "3-4", cond(COL==3 and LINE>=5 and LINE<10, "5-9", cond(COL==4 and LINE>=10, "10+"))))`	Replaces `---` with a specific range based on the `COL` and `LINE` values. E.g., `3-4` in column 2 of lines 3-4, and `5-9` in column 3 of lines 5-9 assuming `---` is found in all lines and columns.	No	Yes
`(\d+)\.(\d+)\.(\d+)`	`cond(CAP1 > 0 and CAP2 == 0 and CAP3 == 0, MATCH, cond(CAP2 > 0 and CAP3 == 0, " " .. MATCH, " " .. MATCH))`	E.g., `1.0.0` is aligned left, `1.2.0` is prefixed with a space, shifting right, indicating a second-level version change.	Yes	No
`(\d+)`	`set(CAP1 * 2)`	Doubles the matched number. E.g., `100` becomes `200`.	Yes	No
`;`	`cond(LCNT == 1, string.rep(" ", 20- (LPOS))..",")`	Inserts spaces before the semicolon to align it to the 20th character position if it’s the first occurrence.	No	No
`-`	`cond(LINE == math.floor(10.5 + 6.25 * math.sin((2 * math.pi * LPOS) / 50)), "*", " ")`	“For the fun part ;-)” Draws a sine wave across a canvas of ‘-’ characters spanning at least 20 lines and 80 characters per line.	No	No

Thomas Knoefel

After intensive testing, Release 2.0.07 is complete and is shipped with the new Notepad++ release 8.5.8.

New Functionalities:

CSV Syntax Highlighting: Provides visual formatting to delimited files, aiding in the readability and structure recognition of the data.
CSV Column Scope: Allows specific column targeting for search and replace in any delimited files.
Rectangular and Multi-selections Support: Extends search and replace to specified rectangular or multi-selected areas.
Conditional and Computational Functionality via “Use Variables” Option: Enables dynamic replacement strings using variables for conditional and computational operations, facilitated by simple set and cond commands.

Vitalii Dovgan

@Thomas-Knoefel
I’ve noticed that the latest archives (e.g. MultiReplace-v2.0.0.7-x64.zip and MultiReplace-v2.0.0.7-Win32.zip) contain pretty old README.md that does not mention all the amazing new features.

And it could be a good idea to add some kind of interactive help. Maybe in a form of popup hints or maybe as a clickable [?] button that allows to “apply” the ‘?’ sign to an UI element by showing the corresponding help text when applied.

Thomas Knoefel

@Vitalii-Dovgan said in [New Plugin] MultiReplace:

I’ve noticed that the latest archives (e.g. MultiReplace-v2.0.0.7-x64.zip and MultiReplace-v2.0.0.7-Win32.zip) contain pretty old README.md that does not mention all the amazing new features.

Ah, now that you mention it, I realize I updated the README after the release, which was an oversight on my part. Thanks for catching that. There’s a new release coming with the next N++ version, which will introduce “Replace in all open documents”. I’ll make sure to get the order right.then.

@Vitalii-Dovgan said in [New Plugin] MultiReplace:

And it could be a good idea to add some kind of interactive help. Maybe in a form of popup hints or maybe as a clickable [?] button that allows to “apply” the ‘?’ sign to an UI element by showing the corresponding help text when applied.

Thanks for the suggestion. My initial idea on that is to add a Question Mark Button next to ‘Use Variables’. This would launch a small help window with different sections like Commands, Options, Operators, Math Functions, and String Functions. Each section would be foldable and offer a description of each list entry. Clicking on a specific entry would insert its function at the cursor position in the Replace String Section.

Thomas Knoefel

The next MultiReplace release 2.1.0.8 will be shipped with the next Notepad++ version 8.5.9

These are the release notes:

Added feature: “Replace All across all open documents”.
Enhancement: Added tooltips to column headers in lists.
Optimization: Removed “Normal” column due to redundant information.
Bug fix: Resolved issue with “Extended” in combination with “Use Variables.”

Thomas Knoefel

I wish you all a Happy New Year.!

The next MultiReplace 2.2.0.9 version will be shipped with Notepad++ 8.6.1 or can be dowloaded here.

Release Notes for 2.2.0.9:

“Replace first match only” checkbox: I’ve noticed that this option will be occasionally used, especially when replacing strings with the same find pattern in a list, but with different replace strings. Although achievable by the ‘Use Variables’ option using cond(CNT==1,‘Replace String’).
Help Button for 'Use Variables’: A new help button, is now included on the panel.
A big thanks from here to @Coises Coises, the creator of the Columns++ Plugin, providing the template for the HTML help page.
Enhancement in ‘Use Variables’ with init({}): This feature is closing the gap for self-defined variables. These Variables are integrated into the entire Replace-All process and across all list run replacements.

The init command introduces a way to initialize newly defined variables, ensuring the value is set just once. With this, the newly introduced if-Then feature becomes significantly more useful, especially when combined with other available variables.

1st example:
It is summing digits of Column2 and Column4 and Replaces the 4th found digit in the line:
Find: (\d+)
Replace: init({COL2=0,COL4=0}); cond(LCNT==4, COL2+COL4); if COL==2 then COL2=CAP1 end; if COL==4 then COL4=CAP1 end;

Output:
1,20,text,2,0 -> 1,20,text,2,22.0

2nd example:
Removes duplicate lines, keeping the first occurrence of each line. Like discussed in this Post.
Find: ^(.*)$
Replace: init({MATCH_PREV=1}); cond(MATCH == MATCH_PREV, ''); MATCH_PREV=MATCH;

Thomas Knoefel

I want to announce the new MultiReplace release 2.3.0.10, bringing new CSV features in Notepad 8.6.3.

Here’s what’s new:

Sorting Lines in CSV by Columns: Ascend or descend, combining columns in any prioritized order.
Deleting Multiple Columns: Simultaneously remove various columns, cleaning up any obsolete delimiters.
Clipboard Column Copying: Instantly copy columns, complete with original delimiters, into the clipboard.

Header Line Sorting Control:

The header line is excluded from sorting by default, as shown in the demo. To alter the number of excluded lines, change HeaderLines=1 to your desired count in [userdir]\AppData\Roaming\Notepad++\plugins\config\MultiReplace.ini.

Regex Assistance for Numeric Sorting:

To prepare numbers for lexicographical sorting and later remove the added leading zeros, you may use the following regex patterns:

Purpose	Find Pattern	Replace With
Align Numbers with Leading Zeros (Decimal)	`\b(\d*)\.(\d{2})`	`set(string.rep("0",9-string.len(string.format("%.2f", CAP1)))..string.format("%.2f", CAP1))`
Align Numbers with Leading Zeros (Non-decimal)	`\b(\d+)`	`set(string.rep("0",9-string.len(CAP1))..CAP1)`
Remove Leading Zeros (Decimal)	`\b0+(\d*\.\d+)`	`$1`
Remove Leading Zeros (Non-decimal)	`\b0+(\d*)`	`$1`

Required Settings:

For “Align Numbers with Leading Zeros” operations:
- Enable ‘Regex’
- Enable ‘Use Variables’
For “Remove Leading Zeros” operations:
- Enable ‘Regex’

Thomas Knoefel

These are the latest updates for MultiReplace! Here’s a quick rundown of the recent version releases and their new features and fixes.

MultiReplace 3.0.2.14 (Latest)

Implemented updates:

Updated the edit field in the list to automatically remove line breaks when pasting, enabling support for complex “Use Variables” statements.
Added correct escaping of special characters in “Use Variables” when used with regex.
Optimized nil checks in conditional statement.
Escaped special characters in languages.ini

MultiReplace 3.0.1.13

Fixed issue where:

Quotes were removed when pasting.
Entries were not inserted at the correct position in the list.
Special characters were removed from CAPs.

MultiReplace 3.0.0.12

Enhanced Statistics Columns: Added additional columns to display statistics for found and replaced values, providing more detailed insights.
Improved Sorting Functionality: Enhanced the ability to revert to the original sorting order of the CSV file, even after rows are modified, deleted, or added.
New Context Menu: Introduced a new context menu for the list, offering a comprehensive overview of all available functions.
Direct List Editing Features: Users can now perform copy, paste, and edit operations directly within the list.
Dark Mode Support for Help Page: The help page for the ‘Use Variables’ option now supports Dark Mode.

Thomas Knoefel

@Thomas-Knoefel

Version 4.0.0.19 has been released and will be included in the next Notepad++ update. Below are the key changes and improvements:

Implemented updates:

DPI Font Support: Includes automatic DPI scaling for adjusted UI elements. A custom scaling factor ranging from 0.5 to 2.0 can also be set via the ScaleFactor variable in file C:\Users\<Username>\AppData\Roaming\plugins\config\MultiReplace.ini.
‘Use Variables’ Feature: Now allows initial setting of variables before processing the complete list by adding a list entry with an empty Find string next to the initial Replace statement.
Collapsible List Window: The list window can now be expanded or collapsed for better handling and space management.
Find and Single Replace: Now support selection scope, including rectangular and multi-selection.
File Handling:
- New ‘Save’ and ‘New File’ buttons for managing list files.
- List files can now be loaded via drag and drop functionality.
Minor Fixes: Various UI adjustments and bug fixes.

Here’s a visual take on the changes:

Enjoy!