[c#] Adding a custom styler or lexer in C# for scintilla/notepad++

Ekopalypse

@Ekopalypse said in [c#] Adding a custom styler or lexer in C# for scintilla/notepad++:

@Bas-de-Reuver

I tried the edifact lexer with a 20MB file and the first load
took about 7 seconds, but then each change was made
in 4-7 ms according to watch.ElapsedMilliseconds.
A good test to see if a lexer is fast enough is to press a key and hold it.
If you don’t see any stuttering while rendering, the lexer is fast enough.

According to the property, it looks like it is used immediately,
but I have found that it can take a while, depending on where
you are in the current file, since it starts changing from position 0 (IntPtr.Zero).
A possible solution would be to start with the first visible line
to speed things up.

As for the (re)styling, I haven’t found a good general approach.
I suppose this is something each lexer has to take care of itself.

Bas de Reuver

The CSV Lint plug-in uses the custom lexer and it’s working quite nicely, but a colleague pointed something out.

The default color styles for the plug-in uses background colors, so that you can clearly see the data columns. Btw it’s not always vertically aligned as columns, but that’s not the point, the plug-in is used to quickly eyeball the data and the colors help to spot any weird values.

Anyway currently the selected line, or cursor line, is always a blue/purple background color, and this blue background “erases” the column background colors. So when the user selects a line to edit values, the column colors on that line are not visible anymore and it’s harder to see which value is which column. You then have to move the line up or down to see the correct background colors again.

It would be nicer if the selected line has the same background colors except a little darker (or lighter or inverse colors or whatever) shade of the background colors. See mockup below, is it possible to do something like this in the lexer code?

I suspect it’s currently some sort of hard coded Notepad++ cursor background, because all the default lexers (XML, C++, JavaScript etc.) mostly use white background colors. And when they do use a different background color it is erased by blue in the same way.

Alan Kilborn

@Bas-de-Reuver

A suggestion that isn’t answering what you asked…have you considered turning this option off?:

Bas de Reuver

@Alan-Kilborn Thanks for the suggestion 👍 I’ve tried that, although when you turn it off you do miss it, you do notice that having the selected line highlight is a very convenient option.

But it works in keeping the background colors, so I’ll recomment that for now.

Ekopalypse

@Bas-de-Reuver

Yes, in general this can be changed by the lexer by using SCI_SETSELBACK api call.
In addition one can change this using the style configurator to match their themes.

Bas de Reuver

@Ekopalypse Thanks, but if I understand correctly, the SCI_SETSELBACK() affects the color of any selected text (default=grey) not of the current line highlighting (default=blue).

Ekopalypse

@Bas-de-Reuver

oops - did I link the wrong function, sorry. But there is one for the current selected line.
Let me check.

Ekopalypse

@Bas-de-Reuver - https://www.scintilla.org/ScintillaDoc.html#SCI_SETCARETLINEBACK

Bas de Reuver

I’ve tried both SCI_SETCARETLINEBACK and SCI_SETCARETLINEBACKALPHA with all kinds of parameter values, but as far as I can tell they don’t seem to do anything.

        Win32.SendMessage(PluginBase.nppData._nppHandle, SciMsg.SCI_SETCARETLINEBACK, new Colour(255, 0, 0).Value, (IntPtr)0);  
        Win32.SendMessage(PluginBase.nppData._nppHandle, SciMsg.SCI_SETCARETLINEBACKALPHA, (IntPtr)0, (IntPtr)0);

The documentation also mentions SC_ELEMENT_CARET_LINE_BACK but that one is not available in the Scintilla_ifaces file.
Any idea how to make the caret line tansparant so that you can also see the background colors of a selected line?

Bas de Reuver

Another question about the colors. For a custom lexer you also have to distribute a color settings xml file.
It’s easiest to generate it at first time start-up, see this post.

This works great, but different users will have different preferences, in particular when running in dark mode. I know uesrs can change this in the Settings > Style Configurator, but I would like to provide a few color presets that look pretty decent, see screenshot below and the LexerStyles xml. I know I could generate all 4 with 3 of them commented out or something like that, but not all users know how to find the XML file let alone edit it.

So is it possible to provide different XMLs for the different Notepad++ style themes? Or alternatively, can I somehow detect if Notepad++ is in dark mode? Then the plugin can just generate the approporiate dark mode colors to the xml.

Bas de Reuver

A custom lexer can automatically apply the syntax highlighting to files with a certain extension. This setting is visible in the menu under Settings > Style Configuration and then the textbox “Default ext.”.

However, the extension textbox is readonly, user cannot change this value.

So why is the Default ext. textbox read-only? When users want to disable the always automatically applying colors to .csv files (so make ext. empty), they have to go to the settings XML file and change it there.

PeterJones

@bas-de-reuver said in [c#] Adding a custom styler or lexer in C# for scintilla/notepad++:

So why is the Default ext. textbox read-only? When users want to disable the always automatically applying colors to .csv files (so make ext. empty), they have to go to the settings XML file and change it there.

This is just a guess (how can we be expected to know the “why”?): Maybe just to make it harder to accidentally change the default extension? (To avoid a billion “why don’t .cpp files get highlighted with the C++ language highlighting?” to find out that the user had intentionally or accidentally edited the default extension list.)

Or maybe because the developer’s intention was that default extensions would always be handled, and the user-extension was thus enough of a feature for adding new file types, but hadn’t considered wanting to remove a filetype by a “normal” user.

But maybe there is a technical reason – a guess for a possible technical reason: maybe there’s some internal variables that are only setup during the initial launch of Notepad++, which include looking at those default extensions for each lexer, and the developers didn’t want to re-work the logic to allow those to change every time that the dialog is edited.

Also, on a lexer-plugin like the CSVLint, which seems to be using a separate XML file compared to the builtin langs.xml, there might be the added technical hurdle of knowing whether to edit langs.xml or some lexer-plugin-specific file.

It probably wouldn’t hurt to make it a feature request to change that from read-only to user-editable (with the explanation that it’s hard to delete extensions)… but don’t be surprised if it never gets implemented, because it may not be as simple as “just make the box not read-only”. If you do, please link us to that feature request.

Michael Vincent

@bas-de-reuver said in [c#] Adding a custom styler or lexer in C# for scintilla/notepad++:

So why is the Default ext. textbox read-only? When users want to disable the always automatically applying colors to .csv files (so make ext. empty), they have to go to the settings XML file and change it there.

Agree with @PeterJones , not sure why, but there is a “workaround”. I set ext="" in the ‘CSVLint.xml’ file and the in my Style Configurator window, the “Default ext.” is blank, so I add csv in the “User ext.” text box. Now I can enable/disable auto-lexing by just editing the editable (non-read-only) “User ext.” text box.

I believe that is the desired behavior, if not the desired implementation to achieve it.

Cheers.

Bas de Reuver

@peterjones said in [c#] Adding a custom styler or lexer in C# for scintilla/notepad++:

Maybe just to make it harder to accidentally change the default extension? (To avoid a billion “why don’t .cpp files get highlighted with the C++ language highlighting?” to find out that the user had intentionally or accidentally edited the default extension list.)

Sounds like a good explanation. The actual problem is that the lexer is too slow on large csv files, which seems to freeze Notepad++ when opening like >10MB files. But the Lex() function implements the syntax highlighting and Notepad calls this function, including the start/end parameters.

So if I understand correctly Notepad++ takes care of efficiently calling this syntax highlighting function (see also this January 2016 update). afaik the custom lexer doesn’t need to implement extra functions or settings to ensure that the rendering is only done to the visible parts, is that correct?

If not, how can I ensure that the custom lexer doesn’t apply syntax highlighting to the entire file at once from beginning to end, but instead only renders colors for the visible area? Ideally the user scrolls through the file, and the colors are incrementally applied to the visible areas, is that possible?

Michael Vincent

@bas-de-reuver said in [c#] Adding a custom styler or lexer in C# for scintilla/notepad++:

Ideally the user scrolls through the file, and the colors are incrementally applied to the visible areas, is that possible?

Yes, but maybe not by default? Not sure how the Lex() thing works, but your plugin can see the “active” lines and act on them. See:

SCI_GETFIRSTVISIBLELINE
SCI_LINESONSCREEN

Cheers.

Ekopalypse

@bas-de-reuver

Afaik, calculating the area of the document to color is an internal Scintilla thing, and I suppose it makes a difference if you open a document where the cursor is at the very top or rather at the end of the document.
The tip @michael-vincent gave to use getfirstvisibleline and linesonscreen might help if

the lexer is line-based
all lines are visible
and wrapped lines are taken into account
I’m not sure what happens when you scroll up when you have only a bottom part lexed. Does Scintilla think that everything has already been edited?

Ekopalypse

The ILoader interface could help here.
First load only the “visible” part and then more and more. But I suppose without Npp taking this into account as well, it might be difficult to implement this in a safe way.

Bas de Reuver

@ekopalypse Thanks the SetIdleStyling seems to work as intended. When I set the following at the beginning, then large files will open faster

    var editor = new ScintillaGateway(PluginBase.GetCurrentScintilla());

    editor.SetIdleStyling(IdleStyling.ALL); // instead of default IdleStyling.NONE

The file is not immediately colored, which is as expected, and the syntax coloring runs in the background. You can freely scroll to the end of a large >10MB csv file,. At first it’s white and then the syntax colors will apear from top to bottom with a short delay. Also, smaller files still work as well, if you open them they are immediately in the correct colors.

I wonder though, why is IdleStyling.NONE the standard, why not always use IdleStyling.ALL by default?

Ekopalypse

@bas-de-reuver

to be honest, I have no experience with this feature.
I’ve never tested or researched what idle styling means for UI responsiveness in general, or when it was introduced and under what conditions at the time. If it really runs in the main Scintilla thread, then it could mean that on a machine that is always highly loaded, the editing part is significantly worse, meaning noticeable to a user.

Bas de Reuver

With a custom lexer, is it possible to update the syntax highlighting colors through code?

In my plug-in the user can select 4 pre-set colors schemes, and the plugin just overwrites the <plug-in name>.xml file with the new colors and then shows a dialog asking the user to restart Notepad++ so that the new colors are applied.

<?xml version="1.0" encoding="utf-8"?>
<NotepadPlus>
	<Languages><!-- etc.. --></Languages>
	<LexerStyles>
		<LexerType name="CSVLint" desc="CSV Linter and validator" excluded="no">
			<WordsStyle styleID="0" name="Default" fgColor="000000" bgColor="FFFFFF" fontName="" fontStyle="0" />
			<WordsStyle styleID="1" name="ColumnColor1" fgColor="000000" bgColor="E0E0FF" fontName="" fontStyle="0" />
			<WordsStyle styleID="2" name="ColumnColor2" fgColor="000000" bgColor="FFFF80" fontName="" fontStyle="0" />
			<WordsStyle styleID="3" name="ColumnColor3" fgColor="000000" bgColor="FFE0FF" fontName="" fontStyle="0" />
			<!-- etc.. -->

But is it possible to update the syntax highlighting colors immediately without restarting Notepad++? So update the color settings through code or a API call?

I know something might be possible, because when you go to Settings -> Style Configurator.. and change the syntax highlighting colors, Notepad++ will immediately update them visibly. Meaning that, if you have a file opened with that particular syntax highlighting then you will see the new colors being applied right away.

I’ve looked at the documentation but couldn’t find anything for this.