Search++: A work in progress

Coises

@guy038 said in Search++: A work in progress:

Now, for any command that cancels all previous selection(s), like the Find in Selection option, the Replace and Find in Selection option and the Find or Replace in Selection option, they seem to act on the whole document, anyway !

Not quite (unless there’s a bug). The first match (meaning first in time) will be the first match (meaning top/left) in the selection for a forward find, or the last match in the selection for a backward find.

After that, the selection is lost. However:

Strictly speaking, to handle properly this case, you should keep a map of the beginning and end of EACH selection, in current document ! Probably not easy with huge documents and a nightmare because of all possible types of selection :-((

That came up in the discussion @Alan-Kilborn and I had when I was working out search in Columns++. Later in that discussion, Alan mentioned the idea of marking text, which became the basis for how search in Columns++ works.

In Search++, there is an option is in the Settings dialog: Convert selections to marked text before beginning a stepwise search. This applies only when you use a stepwise command without a scope specified (default scope); so, if that option is checked, Find when there is a selection (but no marked text) will convert the selection to marked text and search within the marked text, but Find in Selection won’t.

BTW, why the choice between the Replace and Find... options and the Find or Replace... options is not placed in the Settings dialog, like within Notepad++ ? This would simplify some menus !

My thought was that users might change their mind about whether they want to see what the replacement did before moving on; for example, with a complicated regex replacement, perhaps at first wanting to see each replacement in context, then if they seem to be working as expected, start skipping forward to the next candidate for replacement.

guy038

Hello, @coises,

For the last time this season, I went skiing with two friends, just three hours, because the snow conditions started to deteriorate in the afternoon. Our previous outing, last week, was great because it was much colder that day !

Let’s back on topic ;-))

You said :

Not quite (unless there’s a bug). The first match (meaning first in time) will be the first match (meaning top/left) in the selection for a forward find, or the last match in the selection for a backward find.

After that, the selection is lost. However:

So, I agree with you that the very first match is correct and expected, but all the subsequent ones are not !

But, luckily, you add :

In Search++, there is an option is in the Settings dialog: Convert selections to marked text before beginning a stepwise search. This applies only when you use a stepwise command without a scope specified (default scope); so, if that option is checked, Find when there is a selection (but no marked text) will convert the selection to marked text and search within the marked text, but Find in Selection won’t.

Ah… very instructive, indeed. So, we have a mean to look within selections ONLY, if we check the Convert selections to marked text before beginning a stepwise search, in the Settings dialog. It will automatically cancel the selection(s) and replace them with some equivalent Marked text regions. I tested that feature : nice !

Regarding the choice between the two options Replace and Find and Find or Replace :

You said :

My thought was that users might change their mind about whether they want to see what the replacement did before moving on; for example, with a complicated regex replacement, perhaps at first wanting to see each replacement in context, then if they seem to be working as expected, start skipping forward to the next candidate for replacement.

I do understand your point of view ! Now, I’m thinking about an other solution :

In settings, you would create a new option :

How many times you want to [ immediately ] check the result of replacements, with the possible values :

Never
Always
A digit from 1 to 9

So :

In case of the Never answer, the usual Find And Replace would be used
In case of the Always answer, it would be like when the Replace : Don't move to the following occurrence option is checked, in Settings > Preferences > Searching of native N++
In case of a digit answer, between 1 to 9, the first replacements would act as the Find or Replace options of Search++ and all the next replacements would act as the Replace and Find options of Search++

And, to my mind, a default value of 3 would be appropriate !

Best Regards,

guy038

Coises

Search++ version 0.5.2 addresses a number of issues:

Fix display of Settings: Mark style drop-down in dark mode. (@Snabel42: this should fix the problem you described here.)
Add a dialog to resist accidentally clicking the Tools menu options to remove marks from documents in view or from all open documents. (per @guy038, last point here)
Rearrange Settings dialog and add four new settings:
- Fill after Search… menu command or shortcut even if search dialog is already visible.
- Allow default Select command to Select in Selection.
- When eligible selection and marks are present, default search is within selection.
- Allow default Mark command to Mark in Marked Text.
Make commands that explicitly specify in Selection or in Marked Text show a failure message (rather than fall back to in Whole Document) if there is no selection or marked text.
Remove Before and After commands without an explicit scope from command button menus.
Apply the action Convert selections to marked text when beginning a default stepwise search only when the search finds a match.
Change names of stepwise replace commands to Replace (then find) and Replace (then wait).
Fix adding to selection not working when the selection is a rectangular selection. Add note to help regarding unexpected results when adding to existing selections.
Update and clarify documentation, particularly the Search commands and Settings sections.

Thanks to everyone who has contributed to this discussion so far! Search++ still has a lot of rough spots and omissions, but it’s getting better.

Vitalii Dovgan

@Coises
Thank you for the update!
My next suggestion may ruin one of the initial purposes of Search++, but maybe you’ll find it useful anyway? Here it is: the “icudt78.dll” is a big DLL with a size of 32 MB, and when I think about a portable version of Notepad++, I usually consider a portable and small version of Notepad++. Correspondingly, what do you think about an attempt to load the “icudt78.dll” at runtime and if it is not available then just disable the “ICU” button and allow the rest of the plugin to operate normally?

Coises

@Vitalii-Dovgan said in Search++: A work in progress:

My next suggestion may ruin one of the initial purposes of Search++, but maybe you’ll find it useful anyway? Here it is: the “icudt78.dll” is a big DLL with a size of 32 MB, and when I think about a portable version of Notepad++, I usually consider a portable and small version of Notepad++. Correspondingly, what do you think about an attempt to load the “icudt78.dll” at runtime and if it is not available then just disable the “ICU” button and allow the rest of the plugin to operate normally?

Regex search also depends on ICU, so I can’t really make it optional.

One of the major “under the hood” changes I made when I built Search++ based on the search in Columns++ was to use ICU for all the Unicode properties instead of the rather hacked-together approach I used in Columns++. The underlying engine in both search in Columns++ and Regex search in Search++ is Boost.Regex, with a considerable amount of customization to make it work well with Scintilla and with Unicode.

I’m really hesitant to reverse that, as the ICU properties are more accurate and easier to update when Unicode releases new versions, while the Columns++ approach is tricky and a bit fragile, and still doesn’t get everything right.

However, as you note, it does increase the size: that one dll is over six times the size of the plugin dll. It also appears that the current version of the ICU4C dll files (at least the pre-compiled ones) don’t work on Windows 7 (and presumably Windows 8).

So… I suppose it’s possible that at some point I’ll try to undo the use of ICU and go back to the cobbled-together approach, but this time get it right. I can’t hold out hope that it will happen soon, nor promise that it will happen at all.

Edit to add:

I included the ICU search mostly for testing. ICU’s native search doesn’t integrate well with Scintilla, and it lacks some of the syntax familiar to Notepad++ users from Boost.Regex. Since I haven’t manipulated its search process at all, results from ICU searches can serve as a check on whether Regex is properly implemented (allowing for expected/intentional differences). I might remove it before Search++ comes out of pre-release status; but @guy038 has expressed hope that I will leave it in, so it will probably remain, perhaps hidden in some way.

Snabel42

@Coises said in Search++: A work in progress:

Search++ version 0.5.2 addresses a number of issues:

Fix display of Settings: Mark style drop-down in dark mode. (@Snabel42: this should fix the problem you described here.)

It does indeed, thank you!

guy038

Hi,@coises and All,

I’m exploring all the possibilities when you click on the ICU button and, really, you access a new regex world with a lot of new options, all related to UNICODE !

In addition, you get some new regex syntaxes which are easier to interpret and allow an almost infinite number of ways to define an Unicode range of characters !

For example, if we’re searching ,with the Match case option checked, in my Total_Chars.txt file :

The search [[\p{letter}][\p{number}]] is the UNION of the [\p{letter}] and the [\p{Number}] properties. Thus, it returns 145,672 matches OR 1,924 matches = 147,596 matches. It could simply be replaced by the Boost syntax [\p{letter}]|[\p{number}].
The search of [[:letter:]&&[\p{InArabic}]] is the INTERSECTION of the [:letter:] and the \[\p{InArabic}]] properties. Thus, it returns 153 matches which is the amount of letter characters in the main Arabic block. It could be replaced by the Boost syntax (?=[[:letter:]])\p{InArabic}. It returns 145,672 AND 256 = 153 letters within this main Arabic block.
The search of [[:letter:]--[a-z]] is the ASYMETRIC DIFFERENCE between the [[:letter:]] and [a-z] properties = 145,672 - 26 which returns 145,646 characters. It could be replaced by the Boost syntax (?![a-z])[[:letter:]] which also returns 145,646 chars.

I’m still looking for all the ICU properties returning a valid value, against my Total_Chars.txt file ! Be patient some days as it remains about 600 properties not tested !!

Best Regards,

guy038

BTW, @coises, what do you think of my idea regarding the two options, Replace (then find) and Replace (then wait) ?

Coises

@guy038 said in Search++: A work in progress:

I’m still looking for all the ICU properties returning a valid value, against my Total_Chars.txt file ! Be patient some days as it remains about 600 properties not tested !!

I would expect that they will all work. I have avoided tampering with the ICU search, so it can serve as a reference standard for the Regex search as far as Unicode properties.

There are some Boost.Regex constructs that ICU does not support. The ones I know about right now are backtracking control (like (*SKIP)(*FAIL)); \K; and \l and \u as shorthand for [[:lower:]] and [[:upper:]]; there may be others. I don’t expect that I will attempt to modify the ICU syntax in any way.

Offering ICU as a “first class” search (working on documents that aren’t UTF-8, being able to replace as well as find, being reasonably efficient for large documents, etc.) is probably possible, but it will take a lot of work. I’m not at all certain it is a task I will attempt.

On the other hand, once the framework is in place and I feel comfortable taking Search++ out of pre-release status, I do hope to add more of the ICU properties to regular Regex.

BTW, @coises, what do you think of my idea regarding the two options, Replace (then find) and Replace (then wait) ?

I see your point that the Replace drop-down has become overwhelming with all the options, especially in Plain search.

I’m now planning to add a toggle item (with a check mark) to the Tools menu; something like Jump to next match after Replace. I’m also planning to add keyboard shortcuts for (most or all of) the Tools menu items. That would make it reasonably easy to toggle between Replace (then find) and Replace (then wait) without complicating the menu so badly. (With the shortcut it would be more keyboard-friendly than the present arrangement.) The next version will include something like that, unless I encounter an unexpected obstacle or think of a better idea.

Coises

Search++ version 0.5.3 is available:

Avoid a crash when searching in Marked Text in Open Documents or in Marked Text in Documents in this View and a document has no marked text.
Fix button menus overlapping the button when the button is near the bottom of the screen. This could cause inadvertent activation of the last menu option.
Make the Scintilla controls (Find box, Replace box and Results list) change colors when dark mode changes. Fix some visibility problems for caret and found text indicators in dark mode.
Make Show All on the Tools menu scroll current position or selection into view.
Add toggles Bookmark lines when marking text and Jump to next match after Replace to Tools menu.
Add keyboard shortcuts for most Tools menu items.
Condense the Replace button menu and coordinate with the new Jump to next match after Replace toggle.

Note: The keyboard shortcut assignments on the Tools menu are new and might change in the next version in response to my own and others’ experiences, and/or to accommodate any additional menu entries.

@Lachlanmax: Does the implementation of bookmarks in this version work for you? What changes, if any, would you suggest?

@guy038: Do you think the way I’ve done Replace (with the Jump to next match after Replace toggle and the submenu for opposite jump behavior) works sensibly?

All comments, critiques, suggestions and experiences are most welcome!

guy038

Hello, @coises and All,

Sorry to begin with some bugs :-((

The Bookmarks feature does not seem to work at all, even if I close and re-open N++ with the Bookmark lines when marking text option already checked in the Search++ > Tools dialog !
When focus on Search++, the two shortcuts Ctrl + Shift + Y or Ctrl + Shift + E ( which open a new dialog ) wrongly add the control char EN or ENQ to current text typed in the Find dialog. To this purpose, note that in Settings > Preferences... > Editiing 2 I personally did not check the Prevent control character (C0 only) typing into document option, in order to be able to include some of them in my texts or posts !
- Note also that this bug happens ONLY IF you first use the shortcut. In case that you first open the Tools menu and choose the Copy Marked Text... or Settings... option directly, nothing is added in the Find dialog !

Now, regarding the two ways to do a step-wise replacement :

I do like your new option, in the Tools menu, which TOGGLE the replacement type, on the fly, thanks to the Ctrl + J shortcut ( I noted the J for Junp )
But then, I suppose that the last line of the Replace dialog ( Do not jump to next match or Jump to next match ) and its sub-menu ( to adopt the opposite behavior ) are rather redondant and useless ? What is your feeling about it ?

And, personally, I think that a new setting Force the 'Do not jump to next match' behavior during the X first replacements, with 0 < X < 9, would be interesting !

Indeed :

When that new option would be checked :
- If current behavior is Jump to next match it would force the Do not jump to next match for the X first replacements, then all subsequent replacements would follow the current behavior.
- If current behavior is Do not jump to next match it would not change anything.
When that new option would not be checked :
- If current behavior is Jump to next match, it would follow this behavior
- If current behavior is Do not jump to next match, it would follow this behavior

Best Regards,

guy038

guy038

Hi, @coises,

@coises, very sorry about the supposed bug with the Bookmarks feature ! Finally, I understood that this bug happens ONLY IF you choose the ICU regex engine ! If you’re using the Plain or Regex options, everything woks as expected ;-)) Is there a limitation to use Bookmarks when the true ICU regex engine is active ?

I even noticed that, like with the native Mark dialog, if you have, both, bookmarks and marked text in current document and that the Bookmark lines when marking text option, in Tools, is unchecked, a Remove marks and bookmarks from active document action does not clear the Bookmarks, as expected !

BR

guy038

Coises

@guy038 said in Search++: A work in progress:

When focus on Search++, the two shortcuts Ctrl + Shift + Y or Ctrl + Shift + E ( which open a new dialog ) wrongly add the control char EN or ENQ to current text typed in the Find dialog.

Thank you for catching that! I see it here, too. I will figure out why (no doubt it’s related to the fact that those two combinations open a dialog) and fix it.

But then, I suppose that the last line of the Replace dialog ( Do not jump to next match or Jump to next match ) and its sub-menu ( to adopt the opposite behavior ) are rather redondant and useless ? What is your feeling about it ?

I think it’s not useless because:

If you’re going to choose an option from the drop-down menu and you want opposite behavior, it’s easier to open one menu than two.
If you just click, rather than shift+click, the options on the sub-menu don’t change the Jump to next match after Replace setting, they just override it for that one command. So if you like to make a setting and keep it but only occasionally do it differently, you can do that and not have to remember to change the setting back again.

And, personally, I think that a new setting Force the 'Do not jump to next match' behavior during the X first replacements, with 0 < X < 9, would be interesting !

This sounds to me like it would be confusing to use. Your explanation is clear enough, I don’t mean that it is confusing. But using a counter that’s invisible to the user, the behavior of the same command just changes when it counts to a certain point? I can’t see it. Why would it be likely that people would want the same number of searches without automatic find every time, under all conditions?

If others also say they want this behavior, I won’t refuse to give it a try, but… I can’t say I’m fond of the idea.

I note and admit that convenient keyboard navigation of the button drop-down menu options is sorely needed, and as yet I have no good ideas for how to provide it.

Coises

@guy038 said in Search++: A work in progress:

@coises, very sorry about the supposed bug with the Bookmarks feature ! Finally, I understood that this bug happens ONLY IF you choose the ICU regex engine ! If you’re using the Plain or Regex options, everything woks as expected ;-)) Is there a limitation to use Bookmarks when the true ICU regex engine is active ?

Nothing to be sorry about — thank you for the observation, and for narrowing down to ICU. I see what’s wrong: it’s a coding error. I’ll fix it.

I even noticed that, like with the native Mark dialog, if you have, both, bookmarks and marked text in current document and that the Bookmark lines when marking text option, in Tools, is unchecked, a Remove marks and bookmarks from active document action does not clear the Bookmarks, as expected !

If you look closely, when Bookmark lines when marking text is checked, the menu item should say Remove marks and bookmarks from active document; when Bookmark lines when marking text is not checked, that menu item should say Remove marks from active document.

(I’m not sure why I didn’t make the same change to Remove marks from multiple documents…, since it also removes bookmarks as well if Bookmark lines is checked.)

It could be that this is a confusing way to do it. I intended to make the Bookmark lines setting so that when it’s checked, bookmarks and marks “go together” for Mark and Show commands and for Remove marks; but when it’s not checked, Search++ only affects marks and does nothing with bookmarks.

Except that Add selection to marked text doesn’t also add bookmarks regardless of whether Bookmark lines is checked.

This does need refinement. There is certainly some inconsistency. I do want to preserve the ability to manipulate marks without touching bookmarks at all; but which commands should affect bookmarks, and whether I need separate commands for some kinds of bookmark manipulation, remains an open question.