Setting up cute Python syntax highlighting (UDL vs built-in Python lexer)

Mikhail V

I try to customize highligting for Python in NPP and I have found two methods:

Use built-in Python language configurator (in Style configurator -> Python).
Use “user-defined languages” method to define syntax manually.

First of all I noticed such things: the built-in syntax has a fixed set of keywords,
they are all in one big group and I cannot define different styles for separate keywords.

E.g. print, True, False, etc, should have different color, but how can I setup them separately?
Is it true it is impossible to do with built-in Python lexer? If so, any plans to improve that?

So I have googled that and found that “User defined language” method.
Well, I was stubborn enough and made it all through, but frankly speaking it could have
been made a bit easier.
My thoughts about it - there should be definitely one panel with a palette editor where I can
pick and edit colors directly for all style groups. It could simplify the whole process 100 times :-)

Ok, with UDL I am able to colorize keywords now. Cool, I was able to get a nice setup picture finally…
BUT - I cannot use folding now no matter what I try it is not working. :-o
Is it possible to define Python foldings with UDL? I found older questions
on this and the answer is that it is not possible. This is really sad.
Also I cannot find how to highlight def function name.

Actually I see vertical grey lines (indent marks) in Python correctly, so NPP already has
the information about indent? is it planned to include Python folding in the library?
I think the algorithm behind it is very simple - like detect indent change,
when next line indent is bigger, then include in fold, otherwise stop fold.

Now I am in such intersting situation: I have two options: built-in can folding and function/class names
but cannot keyword groups. And UDL can keywords but cannot folding and names. :-/
I personally like customizing but still folds are useful especially for 'def’s and long if-blocks.

Am I missing something which can improve one or both methods?

Claudia Frank

@Mikhail-V

builtin lexer
There are several piece which need to work together.
First you need to checkout if the builtin lexer supports multiple keyword sets.
In LexPython.cxx we see, that the python lexer supports two sets.
To create another set of keywords which then can be recognized by npp you
need to modify the langs.xml file. I.e. add another Keywords tag,
name it instre2 and provide the list of words (in sorted order).
Next you need to modify your used style file, which is either stylers.xml,
if the default theme is used, or one of the theme files. What you need to do
is to add another WordsStyle tag, name it like you want, assign the next
styleID number and, most important, add the keywordClass tag with value instre2.
Restart npp.
user defined language (udl)
Afaik the only way to make this work is to cheat,
meaning e.g. you have to use unique comments as closing folding identifiers.
Not named but another solution might be to use python script plugin.
You get all that you need to write your own lexer.
It might be not as fast as the builtin ones but you get the possibility to add additional features.

Cheers
Claudia

MaDill

Or use the Scintillua++ plugin. https://github.com/dail8859/ScintilluaPlusPlus/releases
This plugin was the most flexible and easiest way to create the syntax highlighting for my user language. It has a Phyton lexer and this you can adjust.

Mikhail V

@Claudia-Frank said:

@Mikhail-V

builtin lexer
There are several piece which need to work together.
First you need to checkout if the builtin lexer supports multiple keyword sets.
In LexPython.cxx we see, that the python lexer supports two sets.
To create another set of keywords which then can be recognized by npp you
need to modify the langs.xml file. I.e. add another Keywords tag,
name it instre2 and provide the list of words (in sorted order).
Next you need to modify your used style file, which is either stylers.xml,
if the default theme is used, or one of the theme files. What you need to do
is to add another WordsStyle tag, name it like you want, assign the next
styleID number and, most important, add the keywordClass tag with value instre2.
Restart npp.

I did this but it does not really work. More precisely, it works only partially - after I make changes to “langs.xml”
and stylers.xml there appears a new styler group under Python language, but the keywords in “instre2” list appeared in black color.
If I change the new styler color it does not affect the look of keywords.
However I have accidentally discovered that they are colored with the color from
“Global styles” -> “default style”
And one additional group is not enough anyway. I need some 4 groups at least to optimize the look.

user defined language (udl)
Afaik the only way to make this work is to cheat,
meaning e.g. you have to use unique comments as closing folding identifiers.

Yes that works. Now I am thinking that having custom tags for folding is in some sense better
solution, because I can adjust the chunks I want to fold manually.

Not named but another solution might be to use python script plugin.
You get all that you need to write your own lexer.
It might be not as fast as the builtin ones but you get the possibility to add additional features.

Interesting, I’ll look into it somewhen.

Claudia Frank

@Mikhail-V

strange, could you post your langs and stylers.xml?

Cheers
Claudia

Mikhail V

@Claudia-Frank

Sure. Just in case, my version :
Notepad++ v7.5 (32-bit)
Build time : Aug 15 2017 - 23:02:57

from langs.xml:

    <Language name="python" ext="py pyw" commentLine="#">
        <Keywords name="instre1">and as assert break class continue def del elif else except exec  finally for from global if import in is lambda  not or pass  raise return  try while with yield async await</Keywords>
        <Keywords name="instre2">print False None True </Keywords>
    </Language>

From stylers.xml :

 <LexerType name="python" desc="Python" ext="">
        <WordsStyle name="DEFAULT" styleID="0" fgColor="000000" bgColor="E7E7E2" fontName="Times New Roman" fontStyle="0" fontSize="" />
        <WordsStyle name="COMMENTLINE" styleID="1" fgColor="A4A596" bgColor="E7E7E2" fontName="" fontStyle="0" fontSize="" />
        <WordsStyle name="NUMBER" styleID="2" fgColor="000000" bgColor="E7E7E2" fontName="" fontStyle="0" fontSize="" />
        <WordsStyle name="STRING" styleID="3" fgColor="934900" bgColor="E7E7E2" fontName="" fontStyle="0" fontSize="" />
        <WordsStyle name="CHARACTER" styleID="4" fgColor="808080" bgColor="D2D2CC" fontName="" fontStyle="0" fontSize="" />
        <WordsStyle name="KEYWORDS" styleID="5" fgColor="AE0000" bgColor="E7E7E2" fontName="" fontStyle="0" fontSize="" keywordClass="instre1" />
        <WordsStyle name="KEYWORDS_A" styleID="20" fgColor="00FF00" bgColor="E7E7E2" fontName="" fontStyle="0" fontSize="" keywordClass="instre2" />
        <WordsStyle name="TRIPLE" styleID="6" fgColor="004000" bgColor="E7E7E2" fontName="" fontStyle="0" fontSize="" />
        <WordsStyle name="TRIPLEDOUBLE" styleID="7" fgColor="A2732D" bgColor="E7E7E2" fontName="" fontStyle="0" fontSize="" />
        <WordsStyle name="CLASSNAME" styleID="8" fgColor="008000" bgColor="FFFFFF" fontName="" fontStyle="1" fontSize="" />
        <WordsStyle name="DEFNAME" styleID="9" fgColor="000000" bgColor="E7E7E2" fontName="" fontStyle="1" fontSize="" />
        <WordsStyle name="OPERATOR" styleID="10" fgColor="000000" bgColor="E7E7E2" fontName="" fontStyle="0" fontSize="" />
        <WordsStyle name="IDENTIFIER" styleID="11" fgColor="000000" bgColor="E7E7E2" fontName="" fontStyle="0" fontSize="" />
        <WordsStyle name="COMMENTBLOCK" styleID="12" fgColor="008000" bgColor="E7E7E2" fontName="" fontStyle="0" fontSize="" />
        <WordsStyle name="DECORATOR" styleID="15" fgColor="FF8000" bgColor="D2D2CC" fontName="" fontStyle="2" fontSize="" />
    </LexerType>

So “KEYWORDS_A” appears in the configurator with keyword list, but the color is not linked to highlighting.
See screenshot:

screenshot

PeterJones

Does Debug Info show Local Conf mode: as ON or OFF? And which copy of langs.xml and stylers.xml did you edit

If Local Conf mode: OFF, then you want to edit %AppData%\Notepad++\___.xml
If Local Conf mode: ON, then you want to edit the ones in the same directory as the executable

Claudia Frank

@Mikhail-V

looks like the used styleID is the problem.
If I copy and paste your stylers.xml into mine, then I do have the same behavior
but if I use styleID=14 for keywords_a then it is working.

To be honest, not sure why this happens. Maybe it is used internally (didn’t check source code).

Cheers
Claudia

Mikhail V

@Claudia-Frank
Yes you are right. With styleID = 14 it works.
By the way, what is this mysterios style : WordsStyle name=“CHARACTER” ?
It does not change anything if I change its color.

@Peter-Jones
I have Local Conf mode ON, from zip archive.

Claudia Frank

@Mikhail-V

had to smile because I’ve asked myself the same question as I first discovered this.
doublequoted strings are string style and singlequoted strings are character style.
But still not sure where this comes from.

Cheers
Claudia

Scott Sumner

@Claudia-Frank said:

doublequoted strings are string style and singlequoted strings are character style.
But still not sure where this comes from

I may be offbase here because I’m not really following the conversation, but what this reminds me of is C/C++ where doublequoted runs of characters are null-terminated “strings” and single-quoted single elements are “characters”.

Claudia Frank

@Scott-Sumner

might be or maybe someone asked just for a feature to handle single quoted string independently.

Cheers
Claudia