[New plugin] Columns++

Coises

Columns++ version 0.7-alpha enables formulas in regular expression replacements in the Search in indicated region dialog. For example, if you want to change every occurrence of “IDN-number” by adding 7 to the numeric part, you can use:

Find what: (?<=\bIDN-)\d+\b
Replace with: (?=this+7)

There are also enhancements regarding the use of time formats (days:hours:minutes:seconds) in calculations, some more flexibility for managing search regions, and other improvements. The help has been updated (in particular, see the sections on Search, Calculation, Formulas and Number formats).

There are some known problems (not new to this release) when Elastic tabstops are enabled and you make a large rectangular selection (e.g., many thousands of lines — it also matters how many tabs are on each line, regardless of whether those tabs are included in the selection). The most serious is that when switching from a tab in which the MarkdownPanel plugin shows its side panel to a tab with a large rectangular selection and Elastic tabstops enabled, Notepad++ can freeze for so long that there is no reasonable choice but to force close it. The takeaway for now is that if you choose to enable Elastic tabstops on large files (more than a few thousand lines, particularly if there are also many tabs per line), be sure to keep significant work in all tabs saved.

I haven’t delayed the release for this, because it is not new to this release… it’s just that I only discovered it yesterday. Of course, when I find a way to mitigate it, I’ll update with a new release. Right now, I do not understand what having a side panel open on a different tab should have to do with it, and I don’t know if it’s specific to the MarkdownPanel plugin. (It doesn’t happen with the built-in Document Map panel.) Because I don’t understand it, I can’t predict when I’ll have a fix.

rdipardo

@Coises said in [New plugin] Columns++:

Right now, I do not understand what having a side panel open on a different tab should have to do with it, and I don’t know if it’s specific to the MarkdownPanel plugin.

MarkdownPanel is an “eager” listener of Scintilla notifications when the panel is showing: https://github.com/mohzy83/NppMarkdownPanel/blob/833521f0e744987ebb415b6a9f5af56883cb6022/NppMarkdownPanel/MarkdownPanelController.cs#L80

My guess is that you’re flooding the main thread with SCN_UPDATEUI events, which would keep MarkdownPanel’s event loop very busy, since it doesn’t bit-mask the ScNotification.Updated field to improve the signal-to-noise ratio.

Coises

@rdipardo said in [New plugin] Columns++:

My guess is that you’re flooding the main thread with SCN_UPDATEUI events, which would keep MarkdownPanel’s event loop very busy, since it doesn’t bit-mask the ScNotification.Updated field to improve the signal-to-noise ratio.

Thanks. What threw me is that the updates are occurring in the tab being activated, and the markdown panel is for the tab being switched away from. But it appears that the panel doesn’t close until after my NPPN_BUFFERACTIVATE routine is finished. I have seen that there is no “this document is losing focus” notification for plugins, so now that I think of it, it makes sense that clean-up doesn’t always happen when it logically should. Probably the alphabetically later plugin gets the NPPN_BUFFERACTIVATE message later, so MarkdownPanel doesn’t have a chance to close its panel and stop listening for notifications until after I’ve already set my custom tabs. That gives me something to consider, though; perhaps I can defer the processing until the next SCN_UPDATEUI after the NPPN_BUFFERACTIVATE, if that doesn’t cause jumpiness or missed updates.

I’m currently attempting to project the time to completion for these potentially long-running activities and raise a progress dialog when the projected time remaining exceeds a threshold (e.g., three seconds). At least the dialog will make it possible to cancel the operation, with the understanding that visual layout and/or rectangular selection might be mangled; that’s a lot better than force-closing Notepad++.

Thanks again for the heads-up regarding MarkdownPanel.

Coises

Columns++ version 0.7.1-alpha adds a progress dialog for long-running Elastic tabstops operations. The operation can be cancelled using a link in the dialog or the Escape key. A control to set the minimum estimated time remaining to trigger the progress dialog was added to the Options dialog.

This update is recommended if you use Elastic tabstops, and especially if you use it with large files. Under some circumstances — as discussed above in this thread — a seemingly innocuous action can cause a very long delay when Elastic tabstops is enabled on a large file. This update will cause a dialog to be displayed which will make it possible to cancel the operation (rather than locking up all of Notepad++ and possibly leading the user to force-close it).

Thomas Knoefel

@Coises I installed your plugin for the first time and was able to display the progress bar successfully. It works great. However, I encountered some difficulties with its usage in certain areas. How can I replace a string with the same string combined with the number of its occurrence?
Given that the documentation is quite theoretical, perhaps some simple examples or a hands-on section would help with the initial testing steps.

Coises

@Thomas-Knoefel said in [New plugin] Columns++:

@Coises I installed your plugin for the first time and was able to display the progress bar successfully. It works great.

Thanks for giving it a try!

How can I replace a string with the same string combined with the number of its occurrence?

If you wanted to replace, say, Definition: with Definition-n: where n is 1 for the first replacement and increments by 1 each time, you’d use:

Find what: Definition:
Replace with: Definition-(?=match):

If you had something more complicated, like replacing Define(word, definition) with Define(word, n, definition) where n is 10 for the first replacement and increments by 5 each time, you’d use:

Find what: (Define$\w+,)([^)]*$)
Replace with: \1 (?=match*5+5),\2

You use (?=…) to enclose a formula, which gets substituted into the replacement string. In the formula, you use line for the line number, match for the match number, this for the numeric value of the text the regular expression matched, and reg(n) for the numeric value of the text the nth capture group matched. There are more complicated possibilities, but those are the main ones. The formula is written like a typical computer language mathematical expression: +, -, *, / for arithmetic, parentheses for grouping, and so on.

Given that the documentation is quite theoretical, perhaps some simple examples or a hands-on section would help with the initial testing steps.

That is a good idea. I can’t promise when I’ll get it done, but adding an “Examples” or a “How do I…?” section to the help sounds worthwhile. Thank you for the suggestion.

Coises

Columns++ version 0.7.2-alpha replaces a Windows API that is only available in Windows 10 and later with one that is available from Vista on.

If you’re running a version of Windows older than Windows 10, 0.7.1 won’t load; use 0.7.2. If you’re running Windows 10 or later, there should be no discernible difference.

Thomas Knoefel

@Coises said in [New plugin] Columns++:

If you wanted to replace, say, Definition: with Definition-n: where n is 1 for the first replacement and increments by 1 each time, you’d use:

Find what: Definition:
Replace with: Definition-(?=match):

I tried it in the “Search in indicated region” window. It is replacing the strings for me visually with “Definition-(?=match):”. It’s somehow not resolving the instruction.
Besides, when I click on “Undo”, it performs two undos, so I cannot go back to the previous “Definition:” string list.

Coises

@Thomas-Knoefel said in [New plugin] Columns++:

I tried it in the “Search in indicated region” window. It is replacing the strings for me visually with “Definition-(?=match):”. It’s somehow not resolving the instruction.

Almost certainly this is because I left out an important instruction: Regular expression must be selected in the Search Mode box. Formula replacements are only done in regular expression mode.

Besides, when I click on “Undo”, it performs two undos, so I cannot go back to the previous “Definition:” string list.

I can’t reproduce this, but I might not understand. You’re finding that after doing an action in Search in indicated region, using the Notepad++ Undo command reverts both the action from the dialog and whatever action you did previously?

Does this happen regardless of how you do the Undo command (toolbar, menu, Ctrl+Z) and regardless of whether the dialog is still open?

Thomas Knoefel

@Coises said in [New plugin] Columns++:

I left out an important instruction: Regular expression must be selected in the Search Mode box.

… yeah, with that it works!

@Coises said in [New plugin] Columns++:

You’re finding that after doing an action in Search in indicated region, using the Notepad++ Undo command reverts both the action from the dialog and whatever action you did previously?

That’s correct. It will jump back as if I had hit “Undo” twice. I suggest moving further discussions of these smaller issues to your Plugin’s issue list to avoid cluttering this thread.

PS: I did some additional tests and it played out that’s all fine with your Plugin. It was a misbehavior of N++. A new restart fixed the problem.

Coises

Columns++ version 0.7.3-alpha does not introduce any new features, but it fixes a couple potentially annoying misbehaviors:

Improved exception handling: Exceptions from the Scintilla C++ interface will now be passed to Notepad++ in a way that allows it to report them as such, and uncleared error status codes from outside Columns++ will no longer cause exceptions in Columns++.
An error in handling parentheses in regular expression replacement strings was fixed.
A design flaw that could cause formula substitutions in regular expression replacement strings to be misinterpreted as part of a special sequence (such as a $n capture group reference) was fixed.

Coises

Columns++ version 0.7.4-alpha fixes the Align numeric command and the Numeric aligned option of the Calculate… command to take into account numbers formatted as times, based on the settings in the Time formats dialog.

There are no other changes to the plugin. I updated the help, and added to the readme to mention a couple special cases where Columns++ can be useful even for documents that aren’t arranged in columns: search with numeric formulas in regular expression replacement strings and sorting using multiple columns or regular expression capture groups.

Coises

Columns++ version 0.8-alpha uses its own copy of Boost.Regex, rather than accessing the copy in Notepad++ through the Scintilla interface. The main reason for this is that the Scintilla interface doesn’t support access to capture groups beyond 9; but it also makes the code cleaner and, based on my limited tests, faster. Here’s the changelog, including that and other new features:

References in formulas and sort keys to regular expression capture groups beyond 9 work now.
When matching regular expressions in column selections or search regions, lookbehind assertions do not recognize any text beyond the boundary of the selection within the row or the segment of the search region in which a match is attempted. (In previous versions of Columns++, as in Notepad++, the text potentially examined by lookbehind assertions always extended to the beginning of the document.)
Changed the way Search in indicated region behaves when no region is indicated, nothing is selected and a search is initiated; if Auto set is checked (the default), the search region is set to the entire document. The old behavior (raising a dialog requesting a rectangular selection) may be obtained by unchecking Auto set.
Made some refinements in how find and replace strings in Extended search mode are processed, including fixes for potential bugs and adding \Uxxxxxx (note: capital U) accepting up to six hexadecimal digits to specify any valid Unicode code point.
Added dropdown arrows to the Count and Replace All buttons in Search in indicated region to support additional operations: Select All, Count Before, Count After, Select Before, Select After, Replace Before, Replace After and Clear History.
Regular expressions using \K will work for incremental find and replace in Search in indicated region providing focus does not leave the dialog between finding a match and replacing it.
Added an Align… command to the Columns++ menu to support aligning column text on any character, character string or regular expression.

Help has been updated.

wonkawilly

Hi Sir,
I am getting a weird behavior in Columns++ > Elastic Tab Stops: the end of line chars get kinda of “narrowed down” (don’t know it it is the correct definition of such behavior) or they lose the last char of each couple (basically CRLF -> CL) when I activate Elastic tab stops. Disabling Ets eol chars return normal.
(Columns++ Installed version: 1.0.6)
Is it a feature/wanted behavior?

Look at the screenshots:

Coises

@wonkawilly said in [New plugin] Columns++:

Is it a feature/wanted behavior?

It’s not so much a “feature” as a “known side-effect.”

In the Columns++ | Profile… dialog there is a setting:
Apply monospaced font optimizations (Yes, No or Best estimate)
and beneath that, a check box:
Don’t show expanded mnemonics for non-printing characters when monospaced.

The defaults are Best estimate and checked.

Don’t show expanded mnemonics… only applies when the first setting is Yes, or when it is Best estimate and the best estimate is Yes.

What that setting does is to replace the reversed-type black boxes for non-printing characters, like BEL or NBSP, with a single ! in a black box. The reason is that “monospaced font optimizations” depend on every character being the same width; they speed up processing in larger files if that is true, but things get even slower than they would be without the optimizations if there are characters (like the black blobs with control character mnemonics) that aren’t the same width as everything else.

I don’t know why the CR and LF are converted to C and L instead of being an exclamation point, like all other non-printing characters, but that’s what happens. I know of no way to change that while still controlling the width of other non-printing characters.

Knowing that it’s not abnormal, if this still bothers you, there are two things you could do to avoid it:

If your files rarely or never contain non-printing characters (except end-of-line characters), you can uncheck Don’t show expanded mnemonics for non-printing characters when monospaced and the display of line-ending characters will return to normal.
I’ve recently discovered that monospaced font optimizations are only necessary if you do not have Use DirectWrite checked in the Notepad++ options at Settings | Preferences… | MISC.. Recent versions of Notepad++ default that setting to checked (it was unchecked when I first released Columns++). Starting with version 1.1, Columns++ will always use “No” as the Best estimate when DirectWrite is enabled. In version 1.0.6, if you have DirectWrite enabled, or if you never enable elastic tabstops on files that are large enough to make response sluggish, you could set Apply monospaced font optimizations to No.

The nature of the Plugins Admin in Notepad++ is that the plugins listing is only updated when Notepad++ is updated (and even then, with some lag, depending on how long it takes to finalize a release). In the Columns++ | Help/About… dialog you’ll find links to the “newest” and “latest stable” releases, which you can install directly. Right now, 1.0.6 is latest stable; newest, 1.1.1, incorporates the DirectWrite/Best estimate fix along with some other significant fixes and improvements. I expect to release a 1.1.2 version within a day or two. If all goes well that version will be promoted to stable in time to get it into the Plugins Admin list for the next release of Notepad++.

wonkawilly

@Coises said in [New plugin] Columns++:

1.0.6

I understand it now, thank you so much for the explication.

Pete Run

@Coises Just moved to Notepad++ x64 and reevaluating my plugins.
The Elastic Tabstops part of Columns++ looks very good as a replacemnt/addition to Plugin Elastic Tabstops.

There seems to be 1 thing missing, or I can’t find it:
“Selection Only”

I use the current Elastic tabstops as a monospace font “text aligner” with “tabs to spaces” instead of tabs, so being able to restrict to “Selection Only” makes life simpler, otherwise it means placing in another document and copying back.

Will you be adding “Selection Only”?

Cheers Pete

Coises

@Pete-Run said in [New plugin] Columns++:

There seems to be 1 thing missing, or I can’t find it:
“Selection Only”

I use the current Elastic tabstops as a monospace font “text aligner” with “tabs to spaces” instead of tabs, so being able to restrict to “Selection Only” makes life simpler, otherwise it means placing in another document and copying back.

Will you be adding “Selection Only”?

I don’t expect to add an option to align tabs only in the selection. I designed the Elastic tabstops feature to make it appear to the user that the entire document is always instantly aligned, while deferring some processing until needed to keep it responsive wherever possible. I think it would complicate processing a lot to have an option to work only on a selection; I’d have to be pretty convinced that there was a good reason for it.

I don’t follow your description of your use case. However, admittedly I haven’t used the Elastic Tabstops plugin much since it was rewritten and taken over by a new maintainer, then almost immediately abandoned. I used the old version quite a bit, but it blocked me from updating to 64-bit Notepad++.

Convert tabs to spaces on the Columns++ menu takes the selection into account if there is one. If you mean that you need to align tabs in a sequence of lines without regard to tabs elsewhere in the same document, that will usually happen automatically, unless you’ve chosen the Tabular profile or otherwise set Profile… | Line up elastic tabstops throughout entire document checked. The exception is if there is no intervening line without tabs between sections you want aligned independently; in that case, you could add a blank line just until you convert and then remove it again.

Pete Run

@Coises Many thanks for the prompt and informative response.

I found this here: https://github.com/dail8859/ElasticTabstops/releases

v1.3.1 Latest (this is the version I was running on NPPx32)
Add ability to ignore extensions
Rebuild for Notepad++ v7.7 64-bit compatibility

However, although it works on x32 bit, it gets access violation on x64.

I did notice this for v1.3
Changelog:
Limit updates to only the current view region. This can lead to a significant performance increase for large files such as TSV, or when doing Search/Replace. Note: that this may case columns to change sizes while scrolling.
Fix issue with specifying extensions
Fix tab widths for proportional fonts
Allow elastic tabstops to be calculated for files using spaces as indentation

So “Selection Only” was considered and impletmented as an improvement by the previous developer.

For my purposes I can manage with a bit of copy/paste, so thanks for the plugin

Cheers Pete
P.S. see next post for different suggestion.

Pete Run

@Coises I have posted this separately as it is a related but different issue.

Image 1 shows normal tabbed lines with lines with NO Tabs.
Image 2 shows 1 “overly” elasticated, possibly as expected it treats the end of the no tab line as a tab position.
Image 3 shows “neatly” elasticated with blank lines inserted “around” simple “Text with NoTab lines”.

So my request would be an option to “Elastically Ignore” lines with no tabs.

I could think of many situations where in 1 document Tabulated information is separated by some textual explanation/detail (e.g. the old * reference) that would not easily fit tabularly. Even many small tables.

In this instance I would then convert tabs to spaces, so I have a workaround; just adding extra blank lines, but that would take a long time for many small tables. I suspect a simple replace command would fix that though. I feel an addition to my macro coming on! Please do not think I am a programmer, despite being in IT since 1970, I’m now 75, my programming expertise ends at butchering other peoples “scripts” and tinkering with recorded macros, but great fun.

Thanks again for the plugin
Pete

1-standard tabs.jpg