Hello, @coises and All,
You said :
There are limitations, not yet formally documented, to the way Search++ handles encodings for Search in Files:
1. UTF-16 files without a byte order mark are not recognized as such. (They’ll be misread as something else.)
2. UTF-8 files without a byte order mark that contain any invalid UTF-8 sequences will be processed using the system default (ANSI) code page.
3. Pure ASCII files are processed as UTF-8. (This doesn’t matter for Find, but I will have to review it when I implement Replace, since someone might include non-ASCII characters in the replacement text. I think there will have to be a user control to determine whether to promote ASCII to ANSI or to UTF-8.)
4. Files without a byte order mark that are not pure ASCII and contain any invalid UTF-8 characters are processed using the system default code page. No attempt is made to detect whether a different legacy code page is more likely to be correct, or is explicitly declared within the file.
Thanks for this extra information !
FYI, during all my tests, the Autodetect character encoding option, in Settings > Preferences… > MISC, was checked and all the files, described in the P.S. section of my previous post, contain a Byte Order Mark ( BOM ). Here is the list of all UTF-16 files of my D: USB key
•------------•--------------------------------------------------------------•-----------------•---------------------•--------------------•
| | | | Multi-Replace | Search++ / N++ |
| BOM | File | Encoding | Matches 'Fi' | Matches 'Fi' |
•------------•--------------------------------------------------------------•-----------------•---------------------•--------------------•
| FE FF | D:\Plane_0_UCS-2_BE.txt | UTF-16 BE BOM | 2 | 0 |
| FE FF | D:\Planes_0+1_UTF-16.txt | UTF-16 BE BOM | 2 | 0 |
•------------•--------------------------------------------------------------•-----------------•---------------------•--------------------•
| FF FE | D:\862_x64\plugins\NppExec\doc\NppExec\NppExec_HelpAll.txt | UTF-16 LE BOM | 0 | 31 |
| FF FE | D:\Plane_0_UCS-2_LE.txt | UTF-16 LE BOM | 2 | 0 |
| FF FE | D:\862_x64\plugins\Config\npec_cmdhistory.txt | UTF-16 LE BOM | 0 | 0 |
| FF FE | D:\862_x64\plugins\Config\npes_last.txt | UTF-16 LE BOM | 0 | 0 |
•------------•--------------------------------------------------------------•-----------------•---------------------•--------------------•
| | Total | | 3 Files / 6 hits | 1 File / 31 hits |
•------------•--------------------------------------------------------------•-----------------•---------------------•--------------------•
Notes :
The correct results are reported in the last column
My USB key also contains 49 UTF-8-BOM files ( EF BB BF ) correctly detected by all the search engines !
Now, based on my test, have you conducted any similar tests of your own, and have you found that searching with Search++ also takes longer compared to the native search in N++ ?
Of course, I can run more tests myself, but I imagine your tests would be more valuable and might, perhaps, give you some ideas on how to increase the overall search speed with Search++
I’m really curious to hear your own observations !
Best Regards,
guy038