[New Plugin] MultiReplace

rdipardo

@Thomas-Knoefel said in [New Plugin] MultiReplace:

@rdipardo said in [New Plugin] MultiReplace:

because Scintilla maps the ANSI code page identifiers to the same values as the Win32 API.

Does it mean that UTF8 would directly match with ANSI in scintilla?

Based on how SCI_GETCODEPAGE works in practice, the alternative encoding to Unicode should be thought of as the “system default” rather than “ANSI”.

For most of N++'s history, the “ANSI” code page was indeed single-byte (or, in the case of the legacy CJK encodings, double-byte). But the addition of a UTF-8 OEM code page in Windows version 1903 makes “ANSI” a less useful identifier, even a potentially deceptive one. The system default is no longer directly opposed to Unicode as it once was.

So, yes, there may be times when “UTF8 would directly match with ANSI,” but only if 65001 is the value of the ACP key in the system’s registry. Check on this first:

reg query HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Nls\CodePage /s /f "CP"

Anybody an idea how to convert a widestr into UTF8 for SCI_SEARCHINTARGET to find these Characters in ANSI?

When you see const char * in the prototype of a Scintilla API (as you will for SCI_SEARCHINTARGET), it means the expected input is a byte string (i.e. “ANSI”). The conversion you want is probably from wchar_t* to char*. A debugger can show you what the encoded text looks like after conversion.

Thomas Knoefel

Thanks, I was still a little bit too much focused on UTF8 with the preperation of ANSI. But this part is working now all directional.

@rdipardo said in [New Plugin] MultiReplace:

Except for the Double-byte Character Sets, which are (still!) the typical OEM encoding on PCs in East Asian countries. Scintilla has a dedicated API for those.

I’m trying to test DBCS on my non-Asian Windows system. Is this even possible somehow? When I test all encodings in Notepad++, SCI_GETCODEPAGE returns 0 for ANSI, and all the others give me 65001. Is there no chance of obtaining one of these encodings?
codePage == 932 || codePage == 936 || codePage == 949 || codePage == 950 || codePage == 136
I tried the BIG5 and Shift_JIS encodings, both of which are DBCS, but I obtained the same result. Even saving and reopening makes no difference. I have the feeling that i’m looking in the wrong place.

Vitalii Dovgan

@Thomas-Knoefel said in [New Plugin] MultiReplace:

I’m trying to test DBCS on my non-Asian Windows system. Is this even possible somehow?

Yes, go to the “Language & region” system settings, and by clicking the “Administrative language settings” a “Region” dialog is shown. This “Region” dialog has “Administrative” tab where there is a button “Change system locale” for non-Unicode programs.
( This is for Windows 11, it was much faster to find in Windows 7 :) )

Vitalii Dovgan

And regarding your other question about conversion between a custom multi-byte encoding (either ANSI or DBCS) and UTF-8, this actually is achieved by double conversion:

First, call MultiByteToWideChar to convert the input multi-byte string (e.g. ANSI/DBCS) to WCHAR string
Second, call WideCharToMultiByte to convert the WCHAR string from the step 1 into a resulting multi-byte string (e.g. UTF-8).

To convert from UTF-8 to ANSI/DBCS, just specify CP_UTF8 in the step 1 and then the desired ANSI/DBCS codepage in the step 2.

Michael Vincent

@Thomas-Knoefel

I opened a few issues and added some pull requests to your repo.

If you are willing to accept pull requests, I have a few more to add once those are merged.

Cheers.

Thomas Knoefel

@Michael-Vincent Thanks, I’ve seen it, and I’m going to commit them. However, the latest updates for codepage handling have not been committed yet. I still need to set up a VMware for Chinese Language settings in order to test DBCS. Once that is finished, I’ll upload the final updates.

Thomas Knoefel

@Thomas-Knoefel These are the facts i fgured out. In Notepad++, when you ask about SCI_GETCODEPAGE, it’s always 0 for ANSI and 65001 for UTF8 you won’t encounter any other codepage. Asian codepages, like DBCS, only matter when you’re reading or writing files. So, these codepages won’t mess things up unless you’re working with files saved in these codepages. As for the Save and Load File feature of the plugin, which is designed for an internal store, it will always save in UTF8 format when handling CSV files.
I think this fact will simplify the handling of codepages.

Thomas Knoefel

@Michael-Vincent said in [New Plugin] MultiReplace:

I have a few more to add once those are merged.

Thank you for your input! All requests are welcome. I can just learn from it.

Thomas Knoefel

I have finished RC-2 version with fixed ANSI support and 32 Bit code compatibility. You can find it on Github.

Vitalii Dovgan

Thank you! I like it!
What probably may add more abilities to the plugin is: 1) to have a button that swaps the text between the Find What and Replace With fields; 2) to have checkboxes in the list to specify which Find-Replace pairs to activate and which to deactivate.

Thomas Knoefel

@Vitalii-Dovgan I will probably add both options before final release. Thanks for your input!

Thomas Knoefel

@Vitalii-Dovgan said in [New Plugin] MultiReplace:

What probably may add more abilities to the plugin is: 1) to have a button that swaps the text between the Find What and Replace With fields; 2) to have checkboxes in the list to specify which Find-Replace pairs to activate and which to deactivate.

RC-3 has been finished!

This update includes the following new features:

Set individual or multiple list entries active or inactive by clicking the select button.
The Space key enables you to select or unselect multiple highlighted entries at once. You can also select or unselect all by clicking at the Header.
The selection status will be saved in CSV
Swap Button for switching Find and Replace text.

The Readme and Screenshot has been updated. Feel free for testing it! You can find it on Github.

Vitalii Dovgan

@Thomas-Knoefel
Thank you!
For some reason I had an impression that the Find-Replace list, as well as the histories of the “Find what” and “Replace with” are automatically saved when Notepad++ exits… Could this be added? Probably the plugin may store this information under the PLUGINSCONFIGDIR folder as e.g. “NppMultiReplace.csv” and “NppMultiReplace.ini”
Well, I realize this also implies some settings where one could enable/disable saving of the Find-Replace list and the “Find what” and “Replace with” histories :)

Vitalii Dovgan

As a side note, I noticed that the LocationNavigate plugin crashes on huge number of replacements, either when a lot of replacements have been done or when the replacement are undoed. (For example, I reproduced it while replacing “CMD” with “CMD” inside of “NppExecEngine.cpp” - 1149 occurrences).
It seems to be a problem of the LocationNavigate plugin itself, but it was worth mentioning anyway.
As maybe an idea for the future version of the MultiReplace plugin, the replacements performance could be dramatically increased if the whole text was read from Scintilla into a memory buffer first, then all the replacements were done with the text in the memory buffer, and finally the resulting text was put into Scintilla. That would probably fix the problem with the LocationNavigate plugin as well.

Thomas Knoefel

@Vitalii-Dovgan said in [New Plugin] MultiReplace:

For some reason I had an impression that the Find-Replace list, as well as the histories of the “Find what” and “Replace with” are automatically saved when Notepad++ exits… Could this be added?

I had this already in mind when I started implementation but forgot about it. Yes, I will implement this.

Thomas Knoefel

@Vitalii-Dovgan said in [New Plugin] MultiReplace:

As maybe an idea for the future version of the MultiReplace plugin, the replacements performance could be dramatically increased

I conducted performance tests yesterday and observed a noticeable difference in performance. There is still room for improvement, but I can confirm that there were no crashes. Specifically, the performance of CopyToClipboard is lacking. However, I will prioritize addressing this in the next release.

I’m considering another feature that could be quite helpful and compensate for the absence of Find capabilities. Next to the Mark button, it might be possible to add up and down keys (similar to those found in NPP Find). After marking, it would be useful to navigate to the marked positions, especially in large files. From a technical standpoint, I’m working on a concept, but I’m not yet convinced about the final result. It would be easy to store all marked positions and sort them. However, if any changes are made to the file, it cannot be guaranteed that these positions are still correct or that the marked text still exists. This presents a limitation. After marking all strings, the plugin should detect any changes. In such cases, the up and down navigation should be disabled. Only by marking all strings again would these buttons be enabled. I haven’t researched Scintilla yet, particularly how to detect changes after a specific moment, and perhaps this would need to be monitored through WM_TIMER. However, I’m still unsure if this limitation is acceptable. There might be other ways to solve it that I haven’t explored yet

Thomas Knoefel

@Vitalii-Dovgan said in [New Plugin] MultiReplace:

For some reason I had an impression that the Find-Replace list, as well as the histories of the “Find what” and “Replace with” are automatically saved when Notepad++ exits…

The INI support has been integrated. All settings of List, History, and Options are stored in the INI and will be reopened when loading the Plugin.

Due to the addition of some extra features that I am still working on, I will release RC4 a bit later.

Thomas Knoefel

RC-4 has been finished, featuring:

New Find and Replace functionality in 2Buttons Mode.
Sort lists by clicking on the header.

Several optimizations and fixes, including:

Update of INI Unicode support.
Updates of Codepage assignments.
Clipboard copy performance optimization.
and a lot of other adjustments

Feel free to test it! You can find it on Github.

Mark Olson

@Thomas-Knoefel said in [New Plugin] MultiReplace:

After marking all strings, the plugin should detect any changes.

Pretty sure the way to check if a file has been modified is to subscribed to the SCN_MODIFIED notification and check if the text field of the notification object has been populated (since the notification can fire if there are changes other than text changes).

But of course your plugin changes the text itself, and so you need some way to distinguish from your plugin’s changes… Hmm…

Mark Olson

Actually, there’s an even easier way that I just remembered to determine if a SCN_MODIFIED notification entails text changing:
check if (notification.modificationType & 3) != 0

This works because 3 = SC_MOD_INSERTTEXT | SC_MOD_DELETETEXT, and all text modifications trigger a notification with one of those flags set.