[c#] Adding a custom styler or lexer in C# for scintilla/notepad++

Bas de Reuver

Another question about the colors. For a custom lexer you also have to distribute a color settings xml file.
It’s easiest to generate it at first time start-up, see this post.

This works great, but different users will have different preferences, in particular when running in dark mode. I know uesrs can change this in the Settings > Style Configurator, but I would like to provide a few color presets that look pretty decent, see screenshot below and the LexerStyles xml. I know I could generate all 4 with 3 of them commented out or something like that, but not all users know how to find the XML file let alone edit it.

So is it possible to provide different XMLs for the different Notepad++ style themes? Or alternatively, can I somehow detect if Notepad++ is in dark mode? Then the plugin can just generate the approporiate dark mode colors to the xml.

Bas de Reuver

A custom lexer can automatically apply the syntax highlighting to files with a certain extension. This setting is visible in the menu under Settings > Style Configuration and then the textbox “Default ext.”.

However, the extension textbox is readonly, user cannot change this value.

So why is the Default ext. textbox read-only? When users want to disable the always automatically applying colors to .csv files (so make ext. empty), they have to go to the settings XML file and change it there.

PeterJones

@bas-de-reuver said in [c#] Adding a custom styler or lexer in C# for scintilla/notepad++:

So why is the Default ext. textbox read-only? When users want to disable the always automatically applying colors to .csv files (so make ext. empty), they have to go to the settings XML file and change it there.

This is just a guess (how can we be expected to know the “why”?): Maybe just to make it harder to accidentally change the default extension? (To avoid a billion “why don’t .cpp files get highlighted with the C++ language highlighting?” to find out that the user had intentionally or accidentally edited the default extension list.)

Or maybe because the developer’s intention was that default extensions would always be handled, and the user-extension was thus enough of a feature for adding new file types, but hadn’t considered wanting to remove a filetype by a “normal” user.

But maybe there is a technical reason – a guess for a possible technical reason: maybe there’s some internal variables that are only setup during the initial launch of Notepad++, which include looking at those default extensions for each lexer, and the developers didn’t want to re-work the logic to allow those to change every time that the dialog is edited.

Also, on a lexer-plugin like the CSVLint, which seems to be using a separate XML file compared to the builtin langs.xml, there might be the added technical hurdle of knowing whether to edit langs.xml or some lexer-plugin-specific file.

It probably wouldn’t hurt to make it a feature request to change that from read-only to user-editable (with the explanation that it’s hard to delete extensions)… but don’t be surprised if it never gets implemented, because it may not be as simple as “just make the box not read-only”. If you do, please link us to that feature request.

Michael Vincent

@bas-de-reuver said in [c#] Adding a custom styler or lexer in C# for scintilla/notepad++:

So why is the Default ext. textbox read-only? When users want to disable the always automatically applying colors to .csv files (so make ext. empty), they have to go to the settings XML file and change it there.

Agree with @PeterJones , not sure why, but there is a “workaround”. I set ext="" in the ‘CSVLint.xml’ file and the in my Style Configurator window, the “Default ext.” is blank, so I add csv in the “User ext.” text box. Now I can enable/disable auto-lexing by just editing the editable (non-read-only) “User ext.” text box.

I believe that is the desired behavior, if not the desired implementation to achieve it.

Cheers.

Bas de Reuver

@peterjones said in [c#] Adding a custom styler or lexer in C# for scintilla/notepad++:

Maybe just to make it harder to accidentally change the default extension? (To avoid a billion “why don’t .cpp files get highlighted with the C++ language highlighting?” to find out that the user had intentionally or accidentally edited the default extension list.)

Sounds like a good explanation. The actual problem is that the lexer is too slow on large csv files, which seems to freeze Notepad++ when opening like >10MB files. But the Lex() function implements the syntax highlighting and Notepad calls this function, including the start/end parameters.

So if I understand correctly Notepad++ takes care of efficiently calling this syntax highlighting function (see also this January 2016 update). afaik the custom lexer doesn’t need to implement extra functions or settings to ensure that the rendering is only done to the visible parts, is that correct?

If not, how can I ensure that the custom lexer doesn’t apply syntax highlighting to the entire file at once from beginning to end, but instead only renders colors for the visible area? Ideally the user scrolls through the file, and the colors are incrementally applied to the visible areas, is that possible?

Michael Vincent

@bas-de-reuver said in [c#] Adding a custom styler or lexer in C# for scintilla/notepad++:

Ideally the user scrolls through the file, and the colors are incrementally applied to the visible areas, is that possible?

Yes, but maybe not by default? Not sure how the Lex() thing works, but your plugin can see the “active” lines and act on them. See:

SCI_GETFIRSTVISIBLELINE
SCI_LINESONSCREEN

Cheers.

Ekopalypse

@bas-de-reuver

Afaik, calculating the area of the document to color is an internal Scintilla thing, and I suppose it makes a difference if you open a document where the cursor is at the very top or rather at the end of the document.
The tip @michael-vincent gave to use getfirstvisibleline and linesonscreen might help if

the lexer is line-based
all lines are visible
and wrapped lines are taken into account
I’m not sure what happens when you scroll up when you have only a bottom part lexed. Does Scintilla think that everything has already been edited?

Ekopalypse

The ILoader interface could help here.
First load only the “visible” part and then more and more. But I suppose without Npp taking this into account as well, it might be difficult to implement this in a safe way.

Bas de Reuver

@ekopalypse Thanks the SetIdleStyling seems to work as intended. When I set the following at the beginning, then large files will open faster

    var editor = new ScintillaGateway(PluginBase.GetCurrentScintilla());

    editor.SetIdleStyling(IdleStyling.ALL); // instead of default IdleStyling.NONE

The file is not immediately colored, which is as expected, and the syntax coloring runs in the background. You can freely scroll to the end of a large >10MB csv file,. At first it’s white and then the syntax colors will apear from top to bottom with a short delay. Also, smaller files still work as well, if you open them they are immediately in the correct colors.

I wonder though, why is IdleStyling.NONE the standard, why not always use IdleStyling.ALL by default?

Ekopalypse

@bas-de-reuver

to be honest, I have no experience with this feature.
I’ve never tested or researched what idle styling means for UI responsiveness in general, or when it was introduced and under what conditions at the time. If it really runs in the main Scintilla thread, then it could mean that on a machine that is always highly loaded, the editing part is significantly worse, meaning noticeable to a user.

Bas de Reuver

With a custom lexer, is it possible to update the syntax highlighting colors through code?

In my plug-in the user can select 4 pre-set colors schemes, and the plugin just overwrites the <plug-in name>.xml file with the new colors and then shows a dialog asking the user to restart Notepad++ so that the new colors are applied.

<?xml version="1.0" encoding="utf-8"?>
<NotepadPlus>
	<Languages><!-- etc.. --></Languages>
	<LexerStyles>
		<LexerType name="CSVLint" desc="CSV Linter and validator" excluded="no">
			<WordsStyle styleID="0" name="Default" fgColor="000000" bgColor="FFFFFF" fontName="" fontStyle="0" />
			<WordsStyle styleID="1" name="ColumnColor1" fgColor="000000" bgColor="E0E0FF" fontName="" fontStyle="0" />
			<WordsStyle styleID="2" name="ColumnColor2" fgColor="000000" bgColor="FFFF80" fontName="" fontStyle="0" />
			<WordsStyle styleID="3" name="ColumnColor3" fgColor="000000" bgColor="FFE0FF" fontName="" fontStyle="0" />
			<!-- etc.. -->

But is it possible to update the syntax highlighting colors immediately without restarting Notepad++? So update the color settings through code or a API call?

I know something might be possible, because when you go to Settings -> Style Configurator.. and change the syntax highlighting colors, Notepad++ will immediately update them visibly. Meaning that, if you have a file opened with that particular syntax highlighting then you will see the new colors being applied right away.

I’ve looked at the documentation but couldn’t find anything for this.

Ekopalypse

@bas-de-reuver

What you are looking for are the style definitions.
The styleID from the xml is the int style value from the methods.
But that doesn’t inform Npp (afaik, there is no official API that can be used to do this) that there are changes that need to be reaccounted for,
and you therefore need to do this with each bufferactivated event which means it is effectively done twice, once by Npp and once by your lexer.

Bas de Reuver

@Ekopalypse I’ve got some more time to work on the plug-in again. And I’m looking at the STYLE_LASTPREDEFINED but how do I use this, is this a notification message, or do I need to send it Scintilla using Win32.SendMessage(scintilla, .. ?

I’m trying to dynamically (at runtime) determine how many Styles are defined for the custom Lexer. The idea is that the user can decide to add more or fewer color styles in the XML, and the plug-in Lexer will use more or fewer of those styles, depending on how many there are.

So is it possible to get the maximum valid styleID? Or maybe a list of the Style names? So in the example below, it’s 10 styles, from Default to HigLightNumeric

Ekopalypse

@Bas-de-Reuver

STYLE_LASTPREDEFINED is used internally by Scintilla and is not publicly available through the API, but I’ll have to double check. Your lexer has 256 styles available, with Scintilla using IDs from 32-39 internally. Since it’s your lexer, you should be able to get the list of style IDs used via some configuration settings.
I’m at the end of my lunch break right now, but I’ll try to come back to this later in the day. See you then.

Ekopalypse

@Bas-de-Reuver

If I did my homework, STYLE_LASTPREDEFINED is only used internally by Scintilla and is only available in a header to inform implementing applications and lexers that this range, 32-STYLE_LASTPREDEFINED, should not be used.
As for the actual problem, from my point of view there is a problem. Npp reads the XML file before the user uses it, and there is currently no way to change this from Npp so that Npp would re-read this file. That means the user would have to restart Npp every time they changed it, or you provide that option via the lexer and a custom configuration menu.

Bas de Reuver

@Ekopalypse thanks for the answer and looking into this, really appreciate it. I am able to update the custom style of my lexer plug-in, using StyleSetBack and StyleSetFore. The style changes are immediately visible, so that is a nice Quality-Of-Life improvement I will put in the next update. 😀

// get handle for editor
var IScintillaGateway editor = new ScintillaGateway(PluginBase.GetCurrentScintilla());
	
// etc..
	
// update style immediately
var fgcolor = x0ff00ff; //pink
var fgcolor = x000ffff; //yellow, apparently it's stored b-g-r so not x0ffff00
	
editor.StyleSetBack(idx, new Colour(fgcolor));
editor.StyleSetFore(idx, new Colour(bgcolor));
editor.StyleSetBold(idx, bold);

However, I wasn’t looking for that in my previous post. And to be clear I also wasn’t looking to reload the XML dynamically or live-update any changes made externally to the XML settings file.

I meant, programmatically get the number of Style items currently in the lexer (also storedin in the XML file). Or maybe get a string-list of the currently available styles. I tried the method GetNamedStyles, expecting to return the number of Styles in the list

// color index
var test123 = editor.GetNamedStyles(); // always returns 1?

So in the lexer in the example screenshot, I expected it to return 10 but it returns 1(?)

Ekopalypse

@Bas-de-Reuver

The xml file is only there to tell Npp how the
properties of the various styles CSVLint uses are to be set
once a buffer is activated, reflecting of course what has been coded internally,
so GetNamedStyles returns what your lexer has specified and in fact
the CSVLintLexer has only one style defined.

You need to find a way to map your idx to the styles you
provided in the xml, and then code the associated functions

            public ILexerNamedStyles NamedStyles;
            public ILexerNameOfStyle NameOfStyle;
            public ILexerTagsOfStyle TagsOfStyle;
            public ILexerDescriptionOfStyle DescriptionOfStyle;

to your liking.

I hope I have understood the question correctly this time. :-)

Ekopalypse

@Bas-de-Reuver said in [c#] Adding a custom styler or lexer in C# for scintilla/notepad++:

StyleSetBack and StyleSetFore

btw … this is usually what Npp does when the buffer is activated or a user changes the configuration via the style configuration dialog.

Bas de Reuver

@Ekopalypse I can update the style using StyleSetBack and StyleSetFore and this works, the new colors are immediately visible. I also overwrite the new style to the XML styler file, so on next start-up it keeps the news colors.

However, I just noticed that when I don’t restart Notepad++ and switch to a different file, the colors in the other file are still in the old/original color style. And when I go back to the first file tab that’s now also back in the old color style. (Btw the colors are correctly changed when I restart Notepad++ because of also the XML update, but I don’t want the user to have to restart)

I suspect it has to do with getting a handle to the editor through GetCurrentScintilla function. But that doesn’t explain why all colors as changed back to the original, and not just for one tab.

Do I have to do anything additional in code to make the style change permanent? Seems like the lexer plugin also stored the original colors from startup (read form the XML) somewhere, but where is that?

Ekopalypse

@Bas-de-Reuver

… to another file, the colors in the other file are still in the old/original color style. And when I go back to the first file tab, the colors are also back in the old color style.

… I suspect it has to do with getting a handle on the editor via the GetCurrentScintilla function. But that doesn’t explain why all the colors were reset to the original, and not just for one tab.

Npp only knows that styles for this lexer should be changed by reading the XML when starting the application or if modified via the style dialog during runtime. Whenever switching to another tab, Npp will determine which lexer is used for this buffer and accordingly changes the styles via StyleSet… Functions. There is no other official way, that I know of, to make this known to Npp at runtime.

Do I need to do something additional in the code to make the style change permanent? It seems that the Lexer plugin also has the original colors from the start (read from the XML) stored somewhere, but where is that?

There is of course the way to find out where Npp keeps the data in memory and then modify accordingly, but you don’t want to go that way, it is not only highly error prone but more likely to crash Npp. One possibility would be to do the same as Npp, when a BufferActivated event is triggered, set the styles as you already did.