Perl language syntax highlighting troubles (bug or limitation ?)

Gilles Maisonneuve

All right I am going to migrate right away and I come back to give back my new debug info

@Ekopalypse : please do not take into account my previous information for the moment.

Meta Chuh

note that if you update your old 7.3.2 version with the 7.6.4 installer from >>> here <<<, you will need to reinstall your plugins using the new built in plugins admin menu.

(or migrate your old plugins manually, or with a script, what ever you decide to be the easiest for you. we will help you with that)

if you go for the batch script variant, the best script would be @dinkumoil 's MigrateNppPlugins.cmd, as it will migrate all past notepad++ releases, with all the different plugin folder structures that have existed.

you can find his MigrateNppPlugins.cmd script >>> here <<<

Gilles Maisonneuve

Notepad++ v7.6.4 (64-bit)
Build time : Mar 6 2019 - 02:58:24
Path : C:\Program Files\Notepad++\notepad++.exe
Admin mode : OFF
Local Conf mode : OFF
OS : Windows 7 (64-bit)
Plugins : ComparePlugin.dll DSpellCheck.dll HTMLTag.dll mimeTools.dll NppConverter.dll PluginManager.dll _CustomizeToolbar.dll

Note that the plugins installed by the additional plugin manager under C:\Users\gm\AppData\Roaming\Notepad++\plugins are not listed here. Only the ones installed under C:\Program Files\Notepad++\plugins are listed in debug info.

Additional plugins are :

total 3824
drw-rw-rw-   7 gm 0   12288 2019-03-15 01:55 config
drw-rw-rw-   3 gm 0       0 2019-03-15 01:44 doc
-rw-rw-rw-   1 gm 0  708096 2019-03-15 01:45 HexEditor.dll
drw-rw-rw-  15 gm 0    4096 2019-03-15 01:44 MenuIcons
-rw-rw-rw-   1 gm 0  125952 2019-03-15 01:45 MenuIcons.dll
-rw-rw-rw-   1 gm 0  368128 2019-03-15 01:55 NppSaveAsAdmin.dll
-rw-rw-rw-   1 gm 0  611328 2019-03-15 01:47 PluginManager.dll
drw-rw-rw-   4 gm 0       0 2019-03-15 01:44 PythonScript
-rw-rw-rw-   1 gm 0 1655808 2019-03-15 01:45 PythonScript.dll
-rw-rw-rw-   1 gm 0  130048 2019-03-15 01:45 RunMe.dll
-rw-rw-rw-   1 gm 0  287744 2019-03-15 01:45 SessionMgr.dll

Let me know if you need more information. It’s late now (>2AM) for me in Europe, I might not answer you until tomorrow.

Have a pleasant night.

Gilles

Gilles Maisonneuve

PS: corrected the python script installation: remove all from AppData, installed new under n++ install dire as explained by Meta Chuh:

+$ ls -l "C:\Program Files\Notepad++"\py*
-rw-rw-rw-  1 gm 0 3428352 2018-04-30 18:44 C:\Program Files\Notepad++\python27.dll
+$ ls -lr "C:\Program Files\Notepad++\plugins\PythonScript"
total 1684
drw-rw-rw-   3 gm 0       0 2019-03-15 02:11 scripts
-rw-rw-rw-   1 gm 0 1655808 2018-10-09 20:19 PythonScript.dll
drw-rw-rw-  18 gm 0   65536 2019-03-15 02:11 lib

Is that all right ?

Meta Chuh

@Gilles-Maisonneuve

Note that the plugins installed by the additional plugin manager under C:\Users\gm\AppData\Roaming\Notepad++\plugins are not listed here. Only the ones installed under C:\Program Files\Notepad++\plugins are listed in debug info.

yes, see the post above yours.

additional info:
the old plugin manager is not compatible with any notepad++ versions above 7.5.9. do not migrate or use this plugin, as it will write to the wrong folder locations.
for all newer plugins, please use the new built in plugins admin.
(plugins which are not listed in plugins admin will have to be installed manually, as they either have not been submitted to the official list by the plugin author, or they are trickier to install, like the pythonscript plugin, requiring python27.dll in the notepad++ binary folder)

Ekopalypse

@Gilles-Maisonneuve

looks good - now to make this work I would do the following.
Goto Plugins->Python Script->New Script, give it a meaningful name
and copy/paste the posted script. Save.
Next, set the PythonScript Initialisation from LAZY to ATSTARTUP
(Plugins->Python Script->Configuration)
While having the configuration window open, you see the new script in the
Scripts area if User Scripts is checked.
In addition you see two sections, Menu items and Toolbar icons.
If you add it to one or both of it, it will appear in the main Plugin->PythonScript menu
and at the toolbar (after npp restart)
This setting would allow it to manually start the script and do its job. Only one start
is needed and every document with an active perl lexer should be handled.

There is an additional step to be done if you want to have the script started automatically
every time npp starts. If you want to do this, then create another new script and
name it startup.py (name is important and DO NOT USE the already available
startup.py as this file might be overwritten once you update pythonscript plugin).
In the NEWLY created startup.py put one line of code
import NAME_OF_THE_SCRIPT_WITHOUT_EXTENSION
like
import EnhancePerlLexer (note, python is case sensitive).

If you want to modify/extend the script it should be only needed to change/add
something within the configuration area.
For example you want to change the color, add another regex or modify an existing one.

As usual, I failed doing a proper documentation. The line
# in the same color as defined by style id 5 using results from matchgroup 1 and 4
doesn’t make sense anymore.
Should be something like # own defined color and non-default match group(s) used

If there is anything unclear or you need help to modify the script to your needs or the
script is not exactly working as you like, do not hesitate to ask. Just a side note, npp
uses a PCRE2 compatible regex engine, it is not the exact perl regex version, so don’t
be confused when there are slightly different syntactical differences.

Have fun.

Ekopalypse

Short note before leaving home - a quick way to identify which style/color is used at a
certain position is to use this two commands in either a script or at the console
(if used within a script, put a print before each line)

editor.getStyleAt(editor.getCurrentPos())
editor.styleGetFore(editor.getStyleAt(editor.getCurrentPos()))

In the example the caret was set at line 2 just before the shift operator.
Note, caret needs to be set before or within a word but not at the end.
And note, this doesn’t return what the enhancement script does as this is
using indicators and not style api.

Gilles Maisonneuve

@Ekopalypse
Did what you wrote, got no result (no visible one I mean).

But perhaps the color you defined in the script (can’t find how you did it) is the same as the one I already had (white).
Both the ‘<<’ operator and the q* ones are in white (on my blue background defined by the vim Dark Blue theme) which matches absolutely nothing (not a single catégorie) of my Perl coloring scheme not of my Global Styles… funny.

We’ll see that perhaps next week (or this we if you’re at home) when (if) you have time.
Thanks for all.

Ekopalypse

@Gilles-Maisonneuve

maybe we check first if your pythonscript plugin installation is working correctly.
Can you open the console (Plugins->Python Script->Show Console)?
Do you see something like

Python 2.7.15 (v2.7.15:ca079a3ea3, Apr 30 2018, 16:30:26) [MSC v.1500 64 bit (AMD64)]
Initialisation took 78ms
Ready.

?
If so, can you type the following into the edit box at the bottom and press enter or run button?

notepad.new()

What happened?

Ekopalypse

@Gilles-Maisonneuve

We’ll see that perhaps next week (or this we if you’re at home) when (if) you have time.

I assume that I’m online again in about 16-20hours, mostly depends on what
my BOSS wants me to do in the garden :-D

Gilles Maisonneuve

@Ekopalypse
Hello, I’m back

Did the test for Python.

Got:

Python 2.7.15 (v2.7.15:ca079a3ea3, Apr 30 2018, 16:30:26) [MSC v.1500 64 bit (AMD64)]
Initialisation took 359ms
Ready.
>>> ?
File "<console>", line 1
?
^
SyntaxError: invalid syntax
>>> notepad.new()

It opened a new window. I changed the language of that window to ‘Perl’ (from 'Normal ‘Text’) but still the qr, qx, … remains in white which is none of Perl defined color for my 'vim dark blue" theme.
Is the color defined in your Python script ? And if so, where and how ?

Ekopalypse

@Gilles-Maisonneuve

Having a break right now - currently there is a lot to do at work so can’t really monitor
the community site.
First, great it looks like pythonscript has been correctly installed. I assume that notepad.new
resulted in a new document, hasn’t it?
When you say you changed the doc to perl and nothing happened, does it mean you
executed the script afterwards as well or did you choose to use the automatically start?
Maybe give a short description about how you created the script, like the name and settings you chose and the workflow how it should get started.

If you haven’t changed anything within the script then those lines define the color

regexes[(1, 5)] = (r'\bq[rwqx]{0,1}\b([^\h]).*?\1|(\bq[rwqx]{0,1}\b\h+(\w).*?\3)', [0])
regexes[(2, 5)] = (r'\bq[rwqx]{0,1}\b\h*(\(.+?\)|\[.+?\]|\{.+?\})', [0])

regexes[(3, (130,130,170))] = (r'(?s)((<<)"*(\w+?)"*;.*?\3)', [2])
regexes[(4, (130,130,170))] = (r'(?s)((<<)\h+"(\w+?)";.*?\3)', [2,3])

The first two, 1 and 2, would use the same color of style id 5 as defined by your used theme.
The next two, 3 and 4, explicitly define a color of (130,130,170) = rgb values.

So, at least the heredoc should be painted in a violet/gray color.

Working for another ~5 hour - so will be online in about ~8 hours again.

Gilles Maisonneuve

@Ekopalypse

list itemWhen I said " I changed the language of that window to ‘Perl’ ", I meant I specifically changed the language for that newly created window (the one creatd by python with “notepad.new()”) to Perl syntax and then entered some perl statements (namely qx, qr, qq) and the color was white.
then I executed the python script (manually, not in auto, suspecting autorun might not work) on (having the focus on) both Windows : the new one and the one with the previous Perl code I had in screen copy in one of my messages above. In both cases I could not get the color else than white.

Going to revert my theme to default one and also change the colors by hand in your code (Python) then re-execute everything.
I’ll let you know.

Don’t bother too much w/ my pb, concentrate on your job. I don’t want to be a pain in the neck for you just for a small detail. It’s just that I’d like to understand why it works for you and not for me, nasty curiosity.

Gilles

Gilles Maisonneuve

PS: did it, correction the color is not white but very light yellow
I commented out color code in your py script, changed to 0 0 0 RGB, NO CHANGE when I execute the script again (saved, changed thme back to default, left n++, restarted it, executed the py script manually… no change…)

bit confused.

Ekopalypse

@Gilles-Maisonneuve

Don’t bother too much w/ my pb, concentrate on your job

:-) That’s for sure - we are allowed to surf the web as long as work related stuff is
priority 1 and we take this very seriously.

bit confused.

Actually me too.
If you don’t mind - would you try to do the following.

Restart npp (Only one single instance should run)
Click Plugins->PythonScript->Show Console
Click Plugins->PythonScript->Scripts->NAME_OF_YOUR_SCRIPT
-> check if there is an error reported in the console, if so - stop here and post it
If no error has been written to console
Open a perl file or a new file and change the language to PERL and put some code in it
-> check if there is an error reported in the console, if so - stop here and post it
If no error has been written to console
execute the following in the console run edit box

editor.getLexer()

Does it return 6 as specified in the script (BUILTIN_LEXER_ID = 6)?
If it does, execute the following in the console run edit box

EnhanceBuiltinLexer

What does it return?

Gilles Maisonneuve

restarted NPP, only one instance: checked
python console:

Python 2.7.15 (v2.7.15:ca079a3ea3, Apr 30 2018, 16:30:26) [MSC v.1500 64 bit (AMD64)]
Initialisation took 109ms
Ready.
Plugins/PythonScript/Scripts->EnhancePerlLexer.py: checked. Result: NOTHING in the console, no error but no confirmation that the py script as run either, is it normal?
Opened a new window, made it Perl syntaxing. . Did not do it. Here is the code for the colors in your script that I modified:

regexes[(1, 5)] = (r’\bq[rwqx]{0,1}\b([^\h]).?\1|(\bq[rwqx]{0,1}\b\h+(\w).?\3)', [0])

regexes[(2, 5)] = (r’\bq[rwqx]{0,1}\b\h*((.+?)|[.+?]|{.+?})', [0])

own defined color and non-default match group(s) used

regexes[(1, (0,0,0))] = (r’\bq[rwqx]{0,1}\b([^\h]).?\1|(\bq[rwqx]{0,1}\b\h+(\w).?\3)‘, [0])
regexes[(2, (0,0,0))] = (r’\bq[rwqx]{0,1}\b\h*((.+?)|[.+?]|{.+?})‘, [0])
regexes[(3, (0,0,0))] = (r’(?s)((<<)"(\w+?)";.?\3)‘, [2])
regexes[(4, (128,128,128))] = (r’(?s)((<<)\h+“(\w+?)”;.?\3)', [2,3])

Executed editor.getLexer(), got a return of 1; executed EnhanceBuiltinLexer, results:

Python 2.7.15 (v2.7.15:ca079a3ea3, Apr 30 2018, 16:30:26) [MSC v.1500 64 bit (AMD64)]
Initialisation took 125ms
Ready.
>>> editor.getLexer()
1
>>> EnhanceBuiltinLexer
<class '__main__.EnhanceBuiltinLexer'>

Seems that I get the wrong return code… don’t I ?

Ekopalypse

@Gilles-Maisonneuve

Seems that I get the wrong return code… don’t I ?

Indeed - would you mind executing the following in the console run edit box?

editor.getLexerLanguage()

no error but no confirmation that the py script as run either, is it normal?

yes.

Ekopalypse

@Gilles-Maisonneuve

I get 1 returned if the language is set to Normal Text.
Sorry for asking, but sure you have set the language to Perl?

Gilles Maisonneuve

@Ekopalypse

>>> editor.getLexerLanguage()
'perl'

Would it be (more) convenient for you to connect directly on my machine ? (DWservice) ?

Ekopalypse

@Gilles-Maisonneuve

Could it be that you have two views open, one with a perl document and one with a normal text document?

Perl language syntax highlighting troubles (bug or limitation ?)

regexes[(1, 5)] = (r’\bq[rwqx]{0,1}\b([^\h]).?\1|(\bq[rwqx]{0,1}\b\h+(\w).?\3)', [0])

regexes[(2, 5)] = (r’\bq[rwqx]{0,1}\b\h*((.+?)|[.+?]|{.+?})', [0])

own defined color and non-default match group(s) used