Missing lexers from Lexilla?

Timur Kelman

Notepad++ v8.4 added Lexilla v5.1.5, but there are some lexers in Lexilla that aren’t in the Language menu of the Notepad++, f.e. SAS and Stata (added in Lexilla v4.1.1, see https://www.scintilla.org/LexillaHistory.html).

Is there a way to enable them? Am I missing something?

PeterJones

@timur-kelman ,

I was able to confirm in source code that SAS and Stata lexers are part of the Lexilla bundle in Notepad++.

To happen automatically, they would have to be added to ScintillaEditView::defineDocType

Unfortunately, if Notepad++ doesn’t bring it out, I don’t know if there’s an easy way to activate it or not. Maybe one of the regulars who better understands the Lexilla and lexing requirements would be able to say whether there’s a way to “activate” one of the non-exposed lexers that’s already built into Lexilla.

Ekopalypse

@timur-kelman

the only way I know of is to use something like this.
I assume that unlike the errorlist lexer, SAS and Stata have keywords that need to be set when the lexer is initialized.
If you want to go this route and need help making it work, let us know.

Alan Kilborn

@ekopalypse

I suppose I have forgotten what the script you linked to actually does. What is the errorlist?

I looked into the linked source and all I saw was “Makes the builtin errorlist lexer available for npp.” which doesn’t refresh my memory. :-(

Ekopalypse

@alan-kilborn

The error list lexer hides ANSI escape sequences and colors the following text accordingly.

For example, something like this


ERROR [clojure-lsp.kondo:281] - Error running clj-kondo on D:\tmp\src\t.clj
�[37mcom.oracle.svm.core.windows.WindowsJavaThreads.osThreadStartRoutine�[m  �[32m        WindowsJavaThreads.java:  138�[m
          �[37mcom.oracle.svm.core.thread.JavaThreads.threadStartRoutine�[m  �[32m               JavaThreads.java:  596�[m
                                               �[37mjava.lang.Thread.run�[m  �[32m                    Thread.java:  829�[m
                 �[37mjava.util.concurrent.ThreadPoolExecutor$Worker.run�[m  �[32m        ThreadPoolExecutor.java:  628�[m
                  �[37mjava.util.concurrent.ThreadPoolExecutor.runWorker�[m  �[32m        ThreadPoolExecutor.java: 1128�[m
                                �[37mjava.util.concurrent.FutureTask.run�[m  �[32m                FutureTask.java:  264�[m
                �[37mjava.util.concurrent.Executors$RunnableAdapter.call�[m  �[32m                 Executors.java:  515�[m
      �[37morg.eclipse.lsp4j.jsonrpc.json.ConcurrentMessageProcessor.run�[m  �[32mConcurrentMessageProcessor.java:  113�[m
        �[37morg.eclipse.lsp4j.jsonrpc.json.StreamMessageProducer.listen�[m  �[32m     StreamMessageProducer.java:   94�[m
 �[37morg.eclipse.lsp4j.jsonrpc.json.StreamMessageProducer.handleMessage�[m  �[32m     StreamMessageProducer.java:  194�[m
                   �[37morg.eclipse.lsp4j.jsonrpc.RemoteEndpoint.consume�[m  �[32m            RemoteEndpoint.java:  187�[m
        �[37morg.eclipse.lsp4j.jsonrpc.RemoteEndpoint.handleNotification�[m  �[32m            RemoteEndpoint.java:  220�[m
          �[37morg.eclipse.lsp4j.jsonrpc.services.GenericEndpoint.notify�[m  �[32m           GenericEndpoint.java:  152�[m
   �[37morg.eclipse.lsp4j.jsonrpc.services.GenericEndpoint.lambda$null$0�[m  �[32m           GenericEndpoint.java:   65�[m
                                                                �[37m...�[m  �[32m                                     �[m
                        �[33mlsp4clj.core.LSPTextDocumentService/�[1;33mdidOpen�[m  �[32m                       core.clj:  101�[m
             �[33mclojure-lsp.handlers.ClojureLSPFeatureHandler/�[1;33mdid-open�[m  �[32m                   handlers.clj:  496�[m
                                      �[33mclojure-lsp.handlers/�[1;33mdid-open�[m  �[32m                   handlers.clj:  128�[m
                       �[33mclojure-lsp.feature.file-management/�[1;33mdid-open�[m  �[32m            file_management.clj:   38�[m
                               �[33mclojure-lsp.kondo/�[1;33mrun-kondo-on-text!�[m  �[32m                      kondo.clj:  332�[m
                                       �[33mclojure-lsp.kondo/�[1;33mrun-kondo!�[m  �[32m                      kondo.clj:  272�[m
                                    �[33mclojure-lsp.kondo/run-kondo!/�[1;33mfn�[m  �[32m                      kondo.clj:  274�[m
                                                �[33mclj-kondo.core/�[1;33mrun!�[m  �[32m                       core.clj:  195�[m
                                  �[33mclj-kondo.impl.core/�[1;33mprocess-files�[m  �[32m                       core.clj:  543�[m
                            �[33mclj-kondo.impl.core/�[1;33mcopied-config-paths�[m  �[32m                       core.clj:  504�[m
                                  �[37msun.nio.fs.WindowsPath.relativize�[m  �[32m               WindowsPath.java:   42�[m
                                  �[37msun.nio.fs.WindowsPath.relativize�[m  �[32m               WindowsPath.java:  404�[m
�[1;31mjava.lang.IllegalArgumentException�[m: �[3m'other' has different root�[m

INFO [clojure-lsp.handlers:124] - :did-open 46ms
INFO [clojure-lsp.handlers:227] - :document-symbol 0ms

with activated lexer looks like this

Alan Kilborn

@ekopalypse

Ah, OK! If “ANSI escape” had been mentioned anywhere, that would have jogged my memory. Those two images would do well to tell the story if placed on the github page for the script.

Ekopalypse

@alan-kilborn - done :-)

PeterJones

@ekopalypse said in Missing lexers from Lexilla?:

If you want to go this route and need help making it work, let us know.

Well, I am not the OP… but I started playing with it. I looked up what the 15 SAS styles were in LexerStyles.iface, and combined that with the sas.properties info from a recent SciTE 5.22 (which matches NPP v7.4.1) to come up with some colors for those styles. I changed the extension to .sas. With that, I’ve gotten it to the point where I can get the SAS highlighting on numbers and operators and %-keywords and comments.

The sas.properties lists the following sets of keywords:

# Keywords
keywords.$(file.patterns.sas)=%let %do

# Block Keywords
keywords2.$(file.patterns.sas)=also cards class data input model ods proc var where

# Function Keywords
keywords3.$(file.patterns.sas)=%printz

# Statement Keywords
keywords4.$(file.patterns.sas)=run

How do I add the list of keywords for each of those 4 keywords entries using your similar PythonScript?

(I picked SAS instead of Stata, even though I know nothing about either, because SciTE included sas.properties but not stata.properties)

PeterJones

@peterjones said in Missing lexers from Lexilla?:

How do I add the list of keywords for each of those 4 keywords entries using your similar PythonScript?

I found it:

        editor.setKeyWords(0, "%let %do")
        editor.setKeyWords(1, "also cards class data input model ods proc var where")
        editor.setKeyWords(2, "%printz")
        editor.setKeyWords(3, "run")

PeterJones

For those who are curious, my script that seems to work for SAS is found at 23147-enable-sas-lexer.py in my scripts repo.

Ekopalypse

@peterjones

congrats, very nice. In the future it may be easier to create them with the newly added NPPM_CREATELEXER message.

PeterJones

Update from the far future: SAS was officially enabled in Notepad++ in v8.7.8, so you no longer need to use a script to enable SAS highlighting.

I wanted to add the Stata lexer at the same time I did SAS, but without a GitHub issue requesting it, I couldn’t justify it. If someone can create a GitHub Issue requesting Notepad++ enable the Stata lexer, I could add it to my TODO list, since I now have experience enabling those missing/hidden lexers in Notepad++. You can use https://github.com/notepad-plus-plus/notepad-plus-plus/issues/16148 as an example of an issue requesting a language be enabled – but I would ask that you include keyword lists: The Stata lexer has 2 keyword lists – one for normal keywords, one for data types – so if the Issue/request could include those two lists of keywords, it would make it much easier for someone who knows nothing about Stata to do the addition (since I wouldn’t have to go googling to try to figure out keywords and try to determine which should be called “normal keywords” and which as “types”). If you do create an Issue, please also paste a link here, because I don’t see all new Issues in the repo.