[c#] Adding a custom styler or lexer in C# for scintilla/notepad++
-
@Ekopalypse said in [c#] Adding a custom styler or lexer in C# for scintilla/notepad++:
I think you are on to something. If this solves the crash problem, then it means that the garbage collection
happened when the functions were renewed and, ultimately, the vtable_pointer also.
I have updated my fork regarding this, which also contains the implementation of the IDcoument interface.Sounds great, I hadn’t even noticed the “other view” issue as I haven’t used it. If you want to do a pullrequest, I’d be happy to accept it.
The LexerExample project would be pretty much done then, I only want to clean it up, remove all unused menu items and add one option to highlghts the numeric values in yellow or something. That will make the lexer “user-interactive” so to speak, because that is still also needed for the CSV lint lexer.
-
Thanks, I’ll do the PR tomorrow.
I would like to play through it again.
Feel free to change anything that is not really C#-like
and I would be happy, if you could drop me a short info when
you have done your cleanup, then I would write my VLang Lexer
in C# and use it for a while to see that we don’t have some hidden/unknown issues. -
@Ekopalypse Ok I’ve cleaned up the project and removed the unused menuitems, and I’ve added a menu item with a checkmark, you can click it to toggle on or off. Also I’ve changed the lexer so it can hightlight numeric values, which can be toggled on or off with a boolean.
However, I can read the property from the property list, see here, but how do I send a boolean value to the
PropertySetfunction of the iLexer? I mean what is the best way to invoke thePropertySetfunction of the iLexer from the main menu? See the line here, it’s commented/disabledIf I understand correctly, invoking the
PropertySetwill cause theLex()to also refresh (depending on the return value) which is good because it will then toggle on/off the numeric highlights. -
Thx, yes but no. You should NEVER call the methods from the
ILexer interface but there corresponding functions from the
scintilla object.
Like SetProperty and SetKeywords.
The ILexer methods are meant to be used by scintilla exclusively.Sorry I haven’t done the PR yet, but real life and other projects got in the way.
-
PR made - I hope I didn’t mix up too many things when I merged your changes and mine into one PR.
From a high level perspective, the Lexer implementation is now fully functional and my tests have shown no more crashes, but I guess only the future can tell.
-
@Ekopalypse said in [c#] Adding a custom styler or lexer in C# for scintilla/notepad++:
PR made - I hope I didn’t mix up too many things when I merged your changes and mine into one PR.
Thanks for the PR I’ve updated the Lexer Example project and I’ve also updated the CSV Lint plug-in and included the syntax highlighting. It works great for both csv and fixed width data 😃 (also quoted string values are now supported, which was an important issue)
About the syntax highlighting, I’ve tested it extensively and it works in large part, but there are some minor things I ran into:
-
Large files take quite a white for the lexer to render, like about 10 seconds for a 10 MB file. And then the lexer is also called for every edit the user does, so each edit to the textfile is slightly delayed, like >half/quarter second for each keypress. It’s workable but you definitely notice it. I’ve added a larger edi data file to test it (and a script to generate even larger files).
-
When changing an option in the lexer, it isn’t always immediately updated. The edifact lexer now has an option to give all numeric values get a different yellow color. When you click the menu item to toggle this on, sometimes it works immediately but sometimes you have to click in the document or switch tabs back and forth before it’s visible. I’ve set it correctly afaik through the
SetPropertyof the editor (see here). -
In the CSV lexer, sometimes the parts that are supposed to be unstyled (so just white) do contain a random color. I think it’s because the Lexer just skips the parts that should be empty, so with no styling (index=0), and then when you edit the document the styling gets shifted around. I suspect that the solution is to either clear all styling on every lexer update, or explicitly set the “no styling” (the latter is probably better/faster performance).
-
-
Large files take quite a …
That sounds like the lexer tries to lex the whole document instead of the range provided by scintilla.
When changing an option in the lexer, it isn’t always immediately updated
I didn’t notice this with my own lexer. As soon as I use SetProperty, PropertySet from the lexer is called
and changes do get updated via the ILexer interface.In the CSV lexer, sometimes
Yes, that’s the interesting part of lexing. What do you do with the parts that have changed.
Brute force would be to clean everything up - style everything - but of course that is time consuming.
Usually scintilla can make a pretty good guess as to which parts need to be reformatted,
but it can’t make sure it’s always right.
Personally, however, I have not noticed this in my projects yet.I’ll take another look at the above points and let you know.
-
@Ekopalypse said in [c#] Adding a custom styler or lexer in C# for scintilla/notepad++:
I tried the edifact lexer with a 20MB file and the first load
took about 7 seconds, but then each change was made
in 4-7 ms according towatch.ElapsedMilliseconds.
A good test to see if a lexer is fast enough is to press a key and hold it.
If you don’t see any stuttering while rendering, the lexer is fast enough.According to the property, it looks like it is used immediately,
but I have found that it can take a while, depending on where
you are in the current file, since it starts changing from position 0 (IntPtr.Zero).
A possible solution would be to start with the first visible line
to speed things up.As for the (re)styling, I haven’t found a good general approach.
I suppose this is something each lexer has to take care of itself. -
The CSV Lint plug-in uses the custom lexer and it’s working quite nicely, but a colleague pointed something out.
The default color styles for the plug-in uses background colors, so that you can clearly see the data columns. Btw it’s not always vertically aligned as columns, but that’s not the point, the plug-in is used to quickly eyeball the data and the colors help to spot any weird values.
Anyway currently the selected line, or cursor line, is always a blue/purple background color, and this blue background “erases” the column background colors. So when the user selects a line to edit values, the column colors on that line are not visible anymore and it’s harder to see which value is which column. You then have to move the line up or down to see the correct background colors again.
It would be nicer if the selected line has the same background colors except a little darker (or lighter or inverse colors or whatever) shade of the background colors. See mockup below, is it possible to do something like this in the lexer code?
I suspect it’s currently some sort of hard coded Notepad++ cursor background, because all the default lexers (XML, C++, JavaScript etc.) mostly use white background colors. And when they do use a different background color it is erased by blue in the same way.
-
A suggestion that isn’t answering what you asked…have you considered turning this option off?:
-
@Alan-Kilborn Thanks for the suggestion 👍 I’ve tried that, although when you turn it off you do miss it, you do notice that having the selected line highlight is a very convenient option.
But it works in keeping the background colors, so I’ll recomment that for now.
-
Yes, in general this can be changed by the lexer by using SCI_SETSELBACK api call.
In addition one can change this using the style configurator to match their themes. -
@Ekopalypse Thanks, but if I understand correctly, the
SCI_SETSELBACK()affects the color of any selected text (default=grey) not of the current line highlighting (default=blue). -
oops - did I link the wrong function, sorry. But there is one for the current selected line.
Let me check. -
-
I’ve tried both SCI_SETCARETLINEBACK and SCI_SETCARETLINEBACKALPHA with all kinds of parameter values, but as far as I can tell they don’t seem to do anything.
Win32.SendMessage(PluginBase.nppData._nppHandle, SciMsg.SCI_SETCARETLINEBACK, new Colour(255, 0, 0).Value, (IntPtr)0); Win32.SendMessage(PluginBase.nppData._nppHandle, SciMsg.SCI_SETCARETLINEBACKALPHA, (IntPtr)0, (IntPtr)0);The documentation also mentions
SC_ELEMENT_CARET_LINE_BACKbut that one is not available in theScintilla_ifacesfile.
Any idea how to make the caret line tansparant so that you can also see the background colors of a selected line? -
Another question about the colors. For a custom lexer you also have to distribute a color settings xml file.
It’s easiest to generate it at first time start-up, see this post.This works great, but different users will have different preferences, in particular when running in dark mode. I know uesrs can change this in the
Settings > Style Configurator, but I would like to provide a few color presets that look pretty decent, see screenshot below and the LexerStyles xml. I know I could generate all 4 with 3 of them commented out or something like that, but not all users know how to find the XML file let alone edit it.So is it possible to provide different XMLs for the different Notepad++ style themes? Or alternatively, can I somehow detect if Notepad++ is in dark mode? Then the plugin can just generate the approporiate dark mode colors to the xml.
-
A custom lexer can automatically apply the syntax highlighting to files with a certain extension. This setting is visible in the menu under
Settings > Style Configurationand then the textbox “Default ext.”.However, the extension textbox is readonly, user cannot change this value.
So why is the Default ext. textbox read-only? When users want to disable the always automatically applying colors to
.csvfiles (so make ext. empty), they have to go to the settings XML file and change it there. -
@bas-de-reuver said in [c#] Adding a custom styler or lexer in C# for scintilla/notepad++:
So why is the Default ext. textbox read-only? When users want to disable the always automatically applying colors to .csv files (so make ext. empty), they have to go to the settings XML file and change it there.
This is just a guess (how can we be expected to know the “why”?): Maybe just to make it harder to accidentally change the default extension? (To avoid a billion “why don’t
.cppfiles get highlighted with the C++ language highlighting?” to find out that the user had intentionally or accidentally edited the default extension list.)Or maybe because the developer’s intention was that default extensions would always be handled, and the user-extension was thus enough of a feature for adding new file types, but hadn’t considered wanting to remove a filetype by a “normal” user.
But maybe there is a technical reason – a guess for a possible technical reason: maybe there’s some internal variables that are only setup during the initial launch of Notepad++, which include looking at those default extensions for each lexer, and the developers didn’t want to re-work the logic to allow those to change every time that the dialog is edited.
Also, on a lexer-plugin like the CSVLint, which seems to be using a separate XML file compared to the builtin
langs.xml, there might be the added technical hurdle of knowing whether to editlangs.xmlor some lexer-plugin-specific file.It probably wouldn’t hurt to make it a feature request to change that from read-only to user-editable (with the explanation that it’s hard to delete extensions)… but don’t be surprised if it never gets implemented, because it may not be as simple as “just make the box not read-only”. If you do, please link us to that feature request.
-
@bas-de-reuver said in [c#] Adding a custom styler or lexer in C# for scintilla/notepad++:
So why is the Default ext. textbox read-only? When users want to disable the always automatically applying colors to .csv files (so make ext. empty), they have to go to the settings XML file and change it there.
Agree with @PeterJones , not sure why, but there is a “workaround”. I set
ext=""in the ‘CSVLint.xml’ file and the in my Style Configurator window, the “Default ext.” is blank, so I addcsvin the “User ext.” text box. Now I can enable/disable auto-lexing by just editing the editable (non-read-only) “User ext.” text box.I believe that is the desired behavior, if not the desired implementation to achieve it.
Cheers.
