@guy038 said:
FYI, during all my tests, the Autodetect character encoding option, in Settings > Preferences… > MISC, was checked and all the files, described in the P.S. section of my previous post, contain a Byte Order Mark ( BOM ).
Autodetect is irrelevant when a byte order mark is present: Notepad++ always honors byte order marks. The only time autodetect matters is for a file with no byte order mark that doesn’t appear to be UTF-8 and doesn’t have an internal indication (like an HTML file with a <meta charset> declaration) of its encoding. With autodetect off, such files will always be read using the system default code page. With autodetect on, Notepad++ runs some heuristic tests to try to guess whether the file uses a different legacy encoding.
When plugins access documents open in Notepad++ they necessarily use the copies Notepad++ has already loaded, so they “inherit” Notepad++’s encoding detection. The loaded copy will always be either in the system default code page (“ANSI”) or UTF-8. Notepad++ converts any document that isn’t one of those. (Any indication in the status bar other than ANSI or UTF-8 means Notepad++ has converted the document to UTF-8, and will convert it back to the indicated encoding when saving it.)
If a plugin accesses files on disk without opening them in Notepad++ — as Search++ does for Search in Files, and as it appears MultiReplace does when searching in files — it’s up to the plugin to sort out the encoding.
I’m not familiar with the code for @thomas-knoefel’s MultiReplace, but based on your tests, I suspect it doesn’t process UTF-16 correctly when searching in files. You could probably construct a minimized example and submit an issue if you were so inclined.
Now, based on my test, have you conducted any similar tests of your own, and have you found that searching with Search++ also takes longer compared to the native search in N++ ?
Comparisons looked fine at first. I was running regular expressions with a lot of backtracking against thousands of files on an SSD. Then things fell apart when I tried a network share.
Most of my testing has been “stress testing” rather than comparisons, but they point to the same problem. I understand the problem. Engineering a solution is taking some time.
Your tests, with the inclusion of re-running the searches, demonstrate what is happening quite well.
In the rerun tests, the files from your thumb drive were already cached by Windows. Those second tests show (roughly) how much time the actual searching took. (Notice that Search++ was faster there, because it uses multiple cores of your computer to search several files simultaneously instead of processing them one at a time.)
The difference between the first and second runs is (roughly) the time it took to read the files from the thumb drive. That is where the current version of Search++ is just plain doing it wrong. I naïvely supposed that Windows was smart enough to “serialize” access to a device, like a USB drive, from different threads in a manner sensible for the device. Alas, it doesn’t work that way. However many threads your computer can run simultaneously, they’re all hitting the USB drive with requests for different files at the same time. Not only is that not any faster than single-threading them, it’s actually slower, because the controller in the drive is dealing with “too many” requests coming in all at once.
The multi-threading I have works pretty well for an internal SSD. It also works if Windows has already cached the entire folder in memory. For everything else… it sucks.
I’m working on solving that problem now. It’s slow going. There are a lot things you can “get by with” when single-threading that just blow up under one condition or another when multi-threading.
