Columns++ version 1.1

[Coises]

There were quite a few changes in this release, so I made it Columns++ version 1.1:

Fixes and improvements:

• When DirectWrite is enabled in Notepad++ settings, “Best estimate” for monospaced font optimization will always be “no”; monospaced font optimizations don’t appear to have any advantage with DirectWrite, and they can cause problems.

• Improvements to the way tab settings are tracked can sometimes significantly speed up, or completely avoid, the “Setting tabstops” message box that interrupts editing when working with large rectangular selections in large files.

• Fixed an error that caused settings for a document to be lost when it was moved from one view to the other or closed in one view while open in the other.

• Fixed an omission that sometimes caused pasting a rectangular selection or a multiple selection to paste into the wrong columns in lines that were not visible on the screen. This fix only works on Notepad++ versions 8.6 and higher. The work-around for pre-8.6 versions is to make a zero-width rectangular selection deep enough to cover all the lines into which you will paste before pasting.

New features:

• Added the Selection submenu, which contains options to extend a rectangular section to the edges of the document (up, left, right or down), to enclose a selection (which can be a single or a multiple selection) in a rectangular selection, and to extend a selection either left to right or top to bottom. Default keyboard shortcuts are established, which can be changed or removed in the Shortcut Mapper in Notepad++.

• Added the Timestamps command and dialog. This command may not be available on Windows versions older than Windows 10 version 1903/19H1. Columns++ will attempt to detect when it will not work and remove it from the menu, but it is possible that it will appear but be non-functional on some older systems.

Notes:

• Some new features and fixes are probably incompatible with the BetterMultiSelection plugin. I believe that most if not all of that plugin’s features are now incorporated natively in current versions of Notepad++; if there are specific problems you cannot solve without that plugin and you observe conflicts with Columns++, please open an issue in the GitHub repository for Columns++ and I will see if there is anything I can do about it.

[Mark Olson]

@Coises

I really appreciate your continued work on this plugin, and I think a lot of the new features look awesome.

Unfortunately, I just uninstalled the plugin out of frustration, and I’ve been thinking about it for a little while to try to clearly articulate why.

TL;DR

The existence of multiple “settings profiles” seems like a trap. It would probably be a good idea to do some serious rethinking of how your settings profiles interact with each other, or doing away with them altogether.

Longer version

I work a lot with very long (say 100,000+ character) new 1 type files. Sometimes these are tabular, sometimes not. I think that Elastic tabstops is excellent, but I would rather have it off by default, because I don’t want them to be recalculated whenever I do a big find/replace action. If Columns++ had one global settings profile, this would be pretty simple. Unfortunately, the existence of multiple settings profiles, combined with uncertainty about whether my current settings will persist when I open a new tab, means that I now have to do a bunch of complex logic in my head to figure out what my settings will even be when I move from one file to another.

Now would be a good time to give an example of how this confusion would arise, but I hope you’ll forgive me for not feeling like crafting one.

One thing I know from painful experience with my JsonTools plugin is that no combination of default settings will satisfy everyone. When you make it difficult to turn a setting off or even know whether a setting is off, it may become easier to simply uninstall rather than figuring out how to get what you want.

[Coises]

@Mark-Olson said in Columns++ version 1.1:

I work a lot with very long (say 100,000+ character) new 1 type files. Sometimes these are tabular, sometimes not. I think that Elastic tabstops is excellent, but I would rather have it off by default

Thank you for taking the time to explain the difficulties you are experiencing.

First, let me give you a quick fix for the problem quoted above.

I think you mean that you open a new tab (since you say “new 1 type files”) and paste in a lot of data. You’d like new tabs to start with Elastic tabstops disabled, so you don’t get hit with long delays as you work.

Open a new tab. Now open the Columns++ Profiles dialog. In the box nearest the bottom, titled Disable elastic tabstops (applies to all profiles) you’ll see a check box labeled when opening new files. Check that box, click OK, and no matter what else is going on, new tabs will always start with elastic tabstops disabled.

If you meant that you’re opening large files and don’t want elastic tabstops kicking in right away, there are two more settings in that same box that turn off elastic tabstops when you first open a file over a given size or number of lines. It could be that the default values there are too large for you.

I understand that your problem with the design is deeper than this, and I will respond soon, probably with some questions to try to understand where it is failing you. But perhaps the above will get past the most immediate difficulty.

[Coises]

@Mark-Olson:

Did you attempt to read the help section on Elastic tabstops, sub-section Automatically enabling or disabling elastic tabstops? I know it’s dull reading; I couldn’t figure out how to make it exciting… but it would help to know if it was just too obnoxious to read, if you read it and it didn’t make sense, or if you read it, it seemed to make sense, but you couldn’t see how it helped you solve your problem. Or if you understood, as far as you could tell, what to do, but it didn’t work.

Would you open this file:
%appdata%\Notepad++\plugins\config\ColumnsPlusPlus.data
(you can paste that right into File name box in the File | Open… dialog in Notepad++; it should still be there even if you’ve removed the plugin) and scroll down to the bottom? The last section should begin with the word Extensions on a line by itself, followed by a blank line and then a few lines after that. Would you copy that section and paste it in your next reply? It might give me a clue why things are confusing; or, at least, let me know what would be expected behavior, so I can figure out if something unexpected is happening to you.

[Mark Olson]

@Coises

Thanks for your thoughtful and prompt replies! Your suggestions have definitely been helpful, and I do not fully endorse my overly bitter complaints above.

I think the issue I was complaining about above is due to an issue that has nothing to do with the multiple “settings profiles”, and is instead due to an unexpected issue where using the Notepad++ find/replace form may turn on Elastic Tabstops. AFAICT from reading your documentation, this is not an intended feature. I’ve created this issue on GitHub.

EDIT: As far as I can tell, the issue with the find/replace form turning on Elastic Tabstops occurs for Columns++ 1.1 on Notepad++ 8.6.4, but is fixed on Notepad++ 8.6.6. This is still pretty concerning, but given its dependency on the Notepad++ version, it may not be worth your time to try to fix it.

[Coises]

@Mark-Olson said in Columns++ version 1.1:

using the Notepad++ find/replace form may turn on Elastic Tabstops

Yes. Thanks to your help in the GitHub issues page, I’ve found it. I mishandled the new NPPN_GLOBALMODIFIED notification.

However, in fixing that I noticed that I made a deeper error connected with my new “Improvements to the way tab settings are tracked” which causes a lot of misbehavior when a document with elastic tabstops enabled is visible in both views at the same time.

So it might be a couple days before I get a fix out. The “Can’t Replace All without causing elastic tabstops to be laid out“ is a bad error, though, so if I can’t fix the other one soon, I’ll put up a version with just that fix.

Thank you for your help in finding this.

[Coises]

@Mark-Olson:

I believe Columns++ v1.1.1 will fix the problem you described. (Unless there is also a different problem I still haven’t found, there must have been some confusion; it’s Notepad++ versions later than 8.6.4 that show the bug, because it comes from mishandling the NPPN_GLOBALMODIFIED message that was introduced in 8.6.5.)

Unfortunately, the problem wasn’t new to the Columns++ 1.1 release; it’s present in the release now baked into the Plugins list going out with Notepad 8.6.7 that just had auto-update enabled.

Not much I can do about it now, except that I’ll remove the “pre-release” tag from this release and make it “latest” (stable) sooner than I otherwise would to try to get this fix out to those open to installing from GitHub, since it’s a bad error. Still, folks who just install plugins “normally” and don’t keep elastic tabstops on all the time are going to have a bad experience with Columns++ until the next Notepad++ auto-update, by which time many will have given up, as you nearly did. :-(

[Mark Olson]

@Coises said in Columns++ version 1.1:

I believe Columns++ v1.1.1 will fix the problem you described.

AFAICT it is fixed, as I noted in the GitHub issue.

Thanks so much for your help!

[Coises]

@Mark-Olson said in Columns++ version 1.1:

I work a lot with very long (say 100,000+ character) new 1 type files. Sometimes these are tabular, sometimes not. I think that Elastic tabstops is excellent, but I would rather have it off by default, because I don’t want them to be recalculated whenever I do a big find/replace action.

I’ve found another problem that might be relevant if you do sometimes turn on elastic tabstops in large files.

There is a serious fault in my progress dialogs. The idea was that if an operation is going to take long enough that it might seem like a hang, the user should see a dialog and have the opportunity to cancel.

What I discovered today is that the progress dialogs themselves cause the operation to take many times longer — I’m estimating at least five to ten times, maybe worse — than it would without them. So if some small thing — the phase of the moon, I don’t know — causes a progress dialog to be triggered, a lot of time is lost. I have a test file of around 12MB, with over 70K lines and only 28 tab characters. If I turn on elastic tabstops for that file, sometimes the response is instant (I can’t even tell how short; it seems to respond to scrolling, etc. immediately); other times a progress dialog is raised and it takes about eight seconds to complete.

At present I have no idea how I can fix this. The dialogs are needed — in a destructive combination with another plugin (Markdown Panel), I’ve seen hangs of over a minute, making it look like Notepad++ is frozen — but now I see they are also causing damage when they are raised and wouldn’t have been needed. (The obvious solution would be to put the long-running operation in a separate thread, but I presume it’s not safe to access Scintilla from anywhere other than the GUI thread. If anyone knows otherwise, I’d love to hear it.)

What I can suggest that until and unless I can fix this, in the Columns++ | Options… dialog, the default value of 2 for Show elastic tabstops progress dialog when seconds remaining exceeds about is too low. Something like 10 or more might be better, so long as you remember that it can hang for that long (or thereabouts… nothing about timing is all that accurate on Windows) without a dialog when analyzing or setting tabstops.

[Ekopalypse]

@Coises

Maybe using a TIMER that is always started but is ignored when the task is finished? Here I do the opposite, starting a dialog for a certain time.

[Coises]

@Ekopalypse said in Columns++ version 1.1:

Maybe using a TIMER that is always started but is ignored when the task is finished?

Thank you for the hint!

I do use SetTimer… and now I see where I went wrong. (The remainder of this is off-topic, as it’s about Windows programming in general, but since it stems from this discussion, perhaps it would be interesting to someone.)

In order to update the display and process possible user input (cancel), the dialog procedure for a progress dialog must return to the modal message loop. If the work must be done in the same thread, that poses a problem: since it’s a modal loop, the work must be done in the dialog procedure itself; but after returning to the message loop, how does the dialog procedure get called again? The only message that will allow WM_PAINT messages to process first (so the display can be updated) is WM_TIMER, so that’s what I use to resume the work to be done after I’ve given the message loop a chance to process user input and update the display.

However, SetTimer has a minimum interval of 10ms. I forgot about that. I work in 100-line chunks, and I tested with small files and with large, “difficult” files, but I neglected to test with large but “easy” files — ones in which the time to process 100 lines was far less than 10ms. Yielding to the message loop every 100 lines is a terrible idea for those cases, because more time is spent waiting for the timer to trigger than working.

A related problem is that I determine when the dialog is needed by checking the time it took to do a 100-line chunk and multiplying by 1/100th of the number of lines remaining in the file. When the time to do a 100-line chunk is very small, small errors and distortions in measuring the time taken to process 100 lines can easily lead to the noise overwhelming the signal. It’s not an accurate way to estimate when the file is “large but easy.”

So I know what to fix now, I just have to get it done.

[Coises]

I said in Columns++ version 1.1:

There is a serious fault in my progress dialogs.
[…]
What I discovered today is that the progress dialogs themselves cause the operation to take many times longer

This is fixed in Columns++ v1.1.2.

I’m still seeing misbehavior when the same file is visible in both views, elastic tabstops is enabled for the file, there is a rectangular selection in one view, and you make edits in the other view (e.g., that enlarge a column to the left of the selection). It might be that this is like the old joke — “Doctor, it hurts when I do this!” / “Well… don’t do that!” — but I’m not sure what is happening. It looks, superficially, as if perhaps the selection remains correct but the display is wrong… then, after sufficient scrolling about, it rights itself. Very strange.

So I guess the bottom line is, another update to the 1.1.* sequence that is not yet marked as stable, and quite possibly not the last before things settle down.