Search++: A work in progress

Coises

@guy038 said:

(I’ll answer various parts of your post in separate comments.)

The more simple , the best ! Don’t analyze files first : just put a warning, before the search, that any non encodable character will be changed into an ? symbol, within the Search++ results panel !

As above, just a warning, before the replacement, that any non-encodable character will be replaced with a ? symbol !

First, I must apologize: my original observation, to which you responded, was not as well thought out, nor as clear, as it should have been. The problem differs significantly depending on whether the search is against the current document, multiple open documents, or files on disk. This response is ridiculously long, largely because I’m “thinking out loud” what I should have thought to myself before posting the item about characters inconsistent with file encodings in the first place.

Don’t analyze files first

The thing is, I don’t know what characters cannot be encoded until I know the encoding of the document or file.

For searching an open document, it’s possible to get the encoding from Notepad++ using NPPM_GETBUFFERENCODING, but there is a problem: as best I can tell, that message doesn’t tell you when an entry from the Encoding | Character Sets menu is in effect (either because the user selected it, or because Settings | Preferences… | MISC. | Autodetect character encoding is enabled). The buffer will be in UTF-8, which means all Unicode characters would appear to be valid, but anything not in the selected character set would be lost when the file was saved.

(Somewhat mitigating this is that Notepad++ itself allows characters that can’t be encoded to be entered or pasted when a character set is selected. You can demonstrate this by opening a new document, selecting Western European | Windows-1252, pasting Abcdefgαβγδxyz. into it, then saving and reopening. The alpha and delta are changed to a and d, the beta is changed to a German sharp s, and the gamma is replaced by a question mark. Yet no sign of this appears when you paste or when you save.)

For files on disk, there is no way to determine the encoding without reading the file. Reading every file first to determine the encoding before doing a find or replace could double the time it took to do the operation; yet with Replace, doing some but not all replacements in a folder could leave the user with a very confusing situation.

The more I think about it, I believe the only way to deal with this — and other things that can go wrong — will be to have Replace in Files by default create new files, effectively keeping the original files as backups that can be restored if the user wants to “undo” the entire operation. The downside is that doing so will take more space on the target device and “pollute” the device, at least temporarily, with unwanted files; for large files with only a few small changes on random access devices, it could make the process much slower as well. So it would have to be an option.

At present, I have nothing equivalent to character sets or Autodetect character encoding in Search in Files; anything that doesn’t have a byte order mark and isn’t valid UTF-8 is assumed to use the system default code page (aka “ANSI”). That could cause trouble if people have mixed legacy encodings in files they try to search, since unlike when loading a document in Notepad++, there would be no visual indication that anything is wrong. At the very least, there is the problem of whether pure ASCII files should be assumed to be ANSI or UTF-8 if a replacement uses a non-ASCII character that is in the system default code page.

just put a warning, before the search, that any non encodable character will be changed into an ? symbol, within the Search++ results panel !

The results list is no problem; it’s always in UTF-8. For the find string, the reason for showing a warning would be to tell the user the requested search can never succeed. For single document searches, it could be helpful to show an error bubble if the find string can never match because it includes characters that cannot be represented in the current document encoding (like searching for a Greek letter gamma in an ANSI document when the system code page is Windows-1252). For multiple document searches, the completion message (e.g., “Found 12 matches in 7 of 9 open documents.” or “No matches found in 5 open documents.”) could include that some (or all) documents were skipped because the find string includes characters inconsistent with the documents’ encoding. Something similar could be added to Find in Files.

The difficult part of this is that for regular expression searches, it’s not simple (algorithmically) to tell which characters must match for the whole expression to match, and which are only required by some alternatives. It might not even be possible in the general case; I would have to investigate that.

But I don’t think it’s all that important for the find string, and it could be limited to warning about Plain text searches that can never succeed, if it is worth doing at all.

For replacing, though, it bothers me more, particularly the problem that in Replace in Files it will be dreadfully inefficient to attempt to determine the encoding of all files before starting any replacements — which suggests that notifying the user when a problem actually happens, not trying to predict it ahead of time, would be the way to go — yet, being left with a folder where some files have been processed and some could not be processed could easily be problematic.

In the end, I should not have raised this point at this time. It doesn’t have much impact in the find case (just identifying why a search has no results), while the worst case, Replace in Files, can’t really be evaluated until I actually design and implement Replace in Files. I apologize, again, for the diversion.

Coises

@guy038 said:

You said :

Keyboard navigation in the Search++ dialog (the main one that can be docked, not the Search in Files dialog) is clunky and not comprehensive.

There’s a setting to focus the document after stepwise Find and Replace, but if you check that then there’s no way to do the next Find or Replace without switching back to the Search++ dialog first. The only way I can think of to manage that would be to add two more menu items, which you could assign to free keyboard shortcuts… as if anyone has any free yet memorable keyboard shortcuts. Doing repeated Find actions while being able to edit immediately in the document after a Find seems like a common task. It should be natural and fluid, but it isn’t.

I didn’t quite understand this specific point. Personally, I do not see any major difference between whether or not to enable the Focus the document after option named : stepwise Find and Replace commands ? Could you clarify this for me ?

The difference with that option is whether Find or Replace (but not Find All or Replace All) leave keyboard focus in the Search++ dialog or put it in the document.

Putting focus in the document means you can start typing, deleting or navigating, if you choose, immediately upon finding or replacing. You don’t need to click in the document or type Ctrl+N first. However, it also means that if you want to Find or Replace again, you have to use the mouse to click the button or use whatever key combination you’ve assigned to Plugins | Search++ | Search… to return focus to the dialog. You can’t just press Alt+F and/or Alt+R repeatedly, because those only apply when the Search++ dialog has focus.

What I’m saying I think is missing is something similar to F3 for Notepad++ search: the ability to leave focus in the document, instead of bouncing back and forth between the document and the dialog, and still do repeated finds (and, as I intend it, also replacements) with a keyboard combination. If you use the floating dialog rather than the docking one, being able to close the dialog and still continue searching/replacing step by step is valuable.

The earlier point was just about the general lack of key combinations for functions on the drop-down menus. Windows has the split button (button with a drop-down) as a native UI element, but while you can access the drop-down with the down arrow on the keyboard once focus is already on the button, the only way to move focus to the button without activating its main function is with the tab key; you can’t use an Alt+ combination, because that does the action on the key (and does not move the focus). So if focus were in the Find box and you wanted to perform a “Show” using only the keyboard, you would have to type Alt+M twice (moving focus to the Match case box and toggling it, then toggling it back), then Shift+Tab (tabbing backward to the Find All button without activating it), then pressing the down arrow key to open the menu and H to select Show.

That’s way too hard to do a simple thing. I tend to be sensitive to keyboard stuff because, a long time ago, I was on a beta team for some software, and also on the team was someone who was disabled in such a way that he could not use a mouse. It made me very aware that there should always be a straightforward way to do anything by keyboard only.