Integration of a script writen in Python

Alan Kilborn

@tho-gru said in Integration of a script writen in Python:

Now I am lokking for a “good” solution of executing scripts during startup.

There’s an existing file called startup.py in PythonScript.
To run my demo script from above automatically on startup, you could do something like this; add these lines to startup.py :

import UrlAltClick
UrlAltClick.UAC()

tho-gru

@Alan-Kilborn thanks for your hint.

In startup.py I entered

import TgrUrlAltClick
TgrUrlAltClick.UAC()

This works a little bit:

The script TgrUrlAltClick.py is looaded
The Python Console does not open on startup although self.debug is still True
ALT-click works for links that are not matched, the Windows standard works fine in this case.
ALT-click for links matching the regular expression get the following error message

20210626-190826.832000 UriIndicatorAltClick indicator click callback
20210626-190826.832000 file:///C:\tmp\anchor-test-file.shtm#part3
20210626-190826.832000 URI matches :-)
Traceback (most recent call last):
  File "C:\Users\ZZunder\AppData\Roaming\Notepad++\plugins\Config\PythonScript\scripts\TgrUrlAltClick.py", line 94, in indicator_release_callback
    self.handle_uri(uri_text)
  File "C:\Users\ZZunder\AppData\Roaming\Notepad++\plugins\Config\PythonScript\scripts\TgrUrlAltClick.py", line 110, in handle_uri
    console.run(defaultBrowser + ' ' + uri_text)
NameError: global name 'console' is not defined

I did not expect that automatic starting of a python script which is working when started manually causes so many problems.

Kind Regards
Thomas

Alan Kilborn

Well, I certainly don’t claim to be an expert on the console instance.
In fact, I don’t know anything about it at all.
All I do in my scripts in the Console window is output debugging info with print() function calls.
So…what I’m saying is that I don’t know how to help you resolve errors relating to this.
Maybe someone else with some knowledge can chime in?

But I will point out that my demo didn’t introduce such console manipulations, your modifications must have. At which point, I turn it fully over to you to debug. :-)

I will say that console.run() seems a strange call to me to launch a browser. How about using ShellExecute or subprocess.Popen for this purpose? Probably the subprocess stuff gives you better control over exactly what you want/need to run.

PeterJones

@tho-gru said in Integration of a script writen in Python:

The Python Console does not open on startup although self.debug is still True

Did you notice the error message that your output showed? Specifically, it said

NameError: global name 'console' is not defined

If console is not defined, how do you expect console.show() to work?

Looking in your imports in the first script that you shared, you have the line:

from Npp import editor, SCINTILLANOTIFICATION, notepad

Notice that you don’t ever import the console symbol from Npp. You probably need to use:

from Npp import editor, SCINTILLANOTIFICATION, notepad, console

tho-gru

@PeterJones thanks for clarification. That makes sense.

I am wondering why this Python script works when executed via the menu (Plugins -> Python Script -> Scripts -> TgrUrlAltClick). If the console module was reported as missing during my tests when I started the script via the menu I might have fixed it before working on the startup.py. So I assume that the loaded modules differs for scripts started from startup and scripts started via the menu. How to inform the plugin developer about this issue?

@Alan-Kilborn thanks for the hint of using subprocess.call. I use it now.

Finally I got everything working. TgrUrlAltClick.py looks like:

# -*- coding: utf-8 -*-

#
# A work around for 
# https://github.com/notepad-plus-plus/notepad-plus-plus/issues/10071
#
# test cases:
#   work around must be used
#   file:///C:\tmp\anchor-test-file.html#part3
#   file:///C:\tmp\anchor-test-file.htm#part3
#   file:///C:\tmp\anchor-test-file.shtml#part3
#   file:///C:\tmp\anchor-test-file.shtm#part3
#
#   handled by the Windows standard
#   file:///C:\tmp\anchor-test-file.xhtml#part3
#   file:///C:\tmp\anchor-test-file.xht#part3
#   file:///C:\tmp\anchor-test-file.hta#part3
#
#   anchors seems to work for HTTP links (at least in Firefox)
#   and are handled correctly by Windows
#   https://docs.opnsense.org/manual/gateways.html#gateways
#

#
# Code based on an idea of Alan Kilborn https://community.notepad-plus-plus.org/user/alan-kilborn
# https://community.notepad-plus-plus.org/topic/21395/integration-of-a-script-writen-in-python/4
#

from __future__ import print_function
from Npp import editor, SCINTILLANOTIFICATION, notepad
from datetime import datetime
from re import match
import ctypes
import subprocess

import TgrRegistry

# URL ALT click
class UAC(object):

    def __init__(self):
        self.URL_INDIC = 8  # URL_INDIC is used in N++ source code
        self.ALT_MODIFIER = 4
        self.alt_held_at_click = False
        self.installed = False
        self.debug = True
        self.now = ""
        self.install()

    def install(self):
        if not self.installed:
            # https://www.scintilla.org/ScintillaDoc.html#SCN_INDICATORCLICK
            editor.callback(self.indicator_click_callback, [SCINTILLANOTIFICATION.INDICATORCLICK])
            # https://www.scintilla.org/ScintillaDoc.html#SCN_INDICATORRELEASE
            editor.callback(self.indicator_release_callback, [SCINTILLANOTIFICATION.INDICATORRELEASE])
            self.installed = True

    def uninstall(self):
        if self.installed:
            editor.clearCallbacks(self.indicator_click_callback)
            editor.clearCallbacks(self.indicator_release_callback)
            self.installed = False

    def is_installed(self):
        return self.installed

    def is_debug_active(self):
        return self.debug

    def get_indicator_range(self, indic_number):
        # similar to ScintillaEditView::getIndicatorRange() in N++ source
        # https://github.com/notepad-plus-plus/notepad-plus-plus/blob/8f38707d33d869a5b8f5014dbb18619b166486a0/PowerEditor/src/ScitillaComponent/ScintillaEditView.h#L562
        curr_pos = editor.getCurrentPos()
        indic_mask = editor.indicatorAllOnFor(curr_pos)
        if (indic_mask & (1 << indic_number)) != 0:
            start_pos = editor.indicatorStart(indic_number, curr_pos)
            end_pos = editor.indicatorEnd(indic_number, curr_pos)
            if curr_pos >= start_pos and curr_pos <= end_pos:
                return (start_pos, end_pos)
        return (0, 0)

    def indicator_click_callback(self, args):
        # example: INDICATORCLICK: {'position': 12294, 'idFrom': 0, 'modifiers': 4, 'code': 2023, 'hwndFrom': 1577146}
        if self.debug:
            self.now = datetime.now().strftime("%Y%m%d-%H%M%S.%f")
            print('{0} UriIndicatorAltClick indicator click callback'.format(self.now))
        self.alt_held_at_click = (args['modifiers'] & self.ALT_MODIFIER) != 0

    def indicator_release_callback(self, args):
        # example: INDICATORRELEASE: {'position': 12294, 'idFrom': 0, 'modifiers': 0, 'code': 2024, 'hwndFrom': 1577146}
        if self.alt_held_at_click:
            self.alt_held_at_click = False
            (start_pos, end_pos) = self.get_indicator_range(self.URL_INDIC)
            if start_pos <> end_pos:        # if click on indicator that is URL_INDIC
                uri_text = editor.getTextRange(start_pos, end_pos)
                self.handle_uri(uri_text)
        if self.debug:
            print('{0} UriIndicatorAltClick indicator release callback'.format(self.now))

    def handle_uri(self, uri_text):
        if self.debug:
            #notepad.messageBox(uri_text, '')
            print("{0} clicked link: uri_text{1}".format(self.now, uri_text))
        htmlExtensions = ("html", "htm", "shtml", "shtm")
        htmlExtReString = '({0})'.format("|".join(htmlExtensions))
        ReString = 'file://.*\\.{0}#.*'.format(htmlExtReString)
        if self.debug: print("{0} regular expression: ReString = {1}".format(self.now, ReString))
        if match(ReString, uri_text):
            if self.debug:
                print('{0} URI matches :-)'.format(self.now))
            reg = TgrRegistry.TgrRegistry()
            defaultBrowser = reg.getDefaultBrowser()
            if self.debug: print("{0} defaultBrowser = {1}\n{0} uri_text = {2}".format(self.now, defaultBrowser, uri_text))
            subprocess.call([defaultBrowser, uri_text])
        else:
            if self.debug:
                print('{0} URI does not match >>> standard action'.format(self.now))
            # the following code was created by sasummer https://github.com/sasumner
            SW_SHOW = 5
            ctypes.windll.Shell32.ShellExecuteA(None, 'open', uri_text, None, None, SW_SHOW)

if __name__ == '__main__':

    if 'uac' not in globals():
        uac = UAC()  # will automatically "install" it
        if uac.debug: console.show()
    else:
        # each running the script toggles install/uninstall:
        uac.uninstall() if uac.is_installed() else uac.install()
        print('uac installed?: {0}'.format(uac.is_installed()))

TgrRegistry.py is:

# -*- coding: utf-8 -*-

from __future__ import print_function
import itertools

# based on
# https://stackoverflow.com/questions/28128446/how-do-i-use-python-to-retrieve-registry-values
# original author: https://stackoverflow.com/users/205580/eryk-sun

try:
    from winreg import *
except ImportError: # Python 2
    from _winreg import *


class TgrRegistry:

    def __init__(self):
            self.KEY_READ_64 = KEY_READ | KEY_WOW64_64KEY

    def getDefaultBrowser(self):
        # 1. step: get name of default browser
        keystr = r"SOFTWARE\Clients\StartMenuInternet"
        key = OpenKey(HKEY_CURRENT_USER, keystr, 0, self.KEY_READ_64)
        keyName = QueryValueEx(key, "")[0]
        CloseKey(key)
        
        # 2. step: get executable name of the default browser
        keystrBrowser = r"SOFTWARE\Clients\StartMenuInternet" + "\\" + keyName + r"\shell\open\command"
        keyBrowser = OpenKey(HKEY_CURRENT_USER, keystrBrowser, 0, self.KEY_READ_64)
        openBrowserCommand = QueryValueEx(keyBrowser, "")[0]
        CloseKey(keyBrowser)

        # convert the UTF-8 string from the registry into ASCII
        openBrowserCommand = openBrowserCommand.decode("utf-8").encode("ascii")
        # strip quotes
        if openBrowserCommand[0] == '"' and openBrowserCommand[len(openBrowserCommand) - 1] == '"':
            openBrowserCommand = openBrowserCommand[1 : len(openBrowserCommand) - 1]
        return openBrowserCommand

And startup.py is:

import TgrUrlAltClick
TgrUrlAltClick.UAC()

As this was my first real python project I am pretty sure to use python more often.

Kind Regards
Thomas

Alan Kilborn

@tho-gru said in Integration of a script writen in Python:

So I assume that the loaded modules differs for scripts started from startup and scripts started via the menu. How to inform the plugin developer about this issue?

TBH, I have never myself noticed any problem along these lines.
But if you want to create an issue concerning it, the PythonScript plugin website is: https://github.com/bruderstein/PythonScript

Alan Kilborn

This thread reminds me of an older discussion by @mkupper that I wanted to follow up on at some point.
That older discussion was HERE.

A recent version of Notepad++ (not sure which one – maybe the one the above link mentions) added the ability to do “custom links” via this setting:

To my knowledge, before this there has never been a great way to add a link in a N++ document that, when activated, will open another document into N++.
Sure, one could use file:// but then that would depend upon having a filepath in the link that is associated with Notepad++.
But…I’m not big on making N++ associations in the OS.

So, what I have set up allows me to put text like this in a document:

and when I activate this link, the intended file opens in Notepad++.
Obviously this required the addition of edit: to the URI customized schemes: box.

I’ve also set up Excel documents to be activatable the same way (but of course opening in Excel, not N++):

Often the files I have to use at work (shared files on a network drive) have spaces in their paths, e.g. W:\test\my Excel file.xlsx

This causes my link scheme to break down:

:-(

But that is a fairly simple thing to fix by doing this to the link text:

and having a macro that takes selected text (the real path, with spaces) in N++ and replaces spaces with %20 and then using the modified version in the edit: link.
Actually the macro does the prepending of the edit: text as well.

So, anyway, just showing an additional possibility (for scripting), a bit above and beyond what the OP wanted to do with linked text in this thread.

Ekopalypse

@tho-gru said in Integration of a script writen in Python:

I am wondering why this Python script works when executed via the menu

because it is then defined in the main namespace, but if
you import your script, then that script is defined in its own namespace.

A word of warning if you use the default startup.py file.
The default startup.py will be overwritten when PS is updated.
Therefore, create your own startup.py file by simply creating a new script and naming it startup.py.
If you use the version installed by PluginAdmin, it is unlikely that this version will be updated, but the PS3 version will be updated from time to time.

Ekopalypse

@PeterJones @Alan-Kilborn

Guess who I take all the inspiration from? :-D
From everyone who posted any scripts and Python code in general.
I think that’s how most of us learn new languages.
So from my point of view, reusing their ideas, even parts of their code is normal, I would say.
I’m not saying they don’t deserve the credits, it’s just impossible to really know everyone involved.
So, to everyone who has ever posted any code, even in other languages, thank you. :-D

Alan Kilborn

@Ekopalypse said in Integration of a script writen in Python:

A word of warning if you use the default startup.py file.
The default startup.py will be overwritten when PS is updated.
Therefore, create your own startup.py file by simply creating a new script and naming it startup.py.
If you use the version installed by PluginAdmin, it is unlikely that this version will be updated, but the PS3 version will be updated from time to time.

Interesting. I don’t think I’ve ever known this.

So there’s a startup.py in ...\plugins\PythonScript\scripts\ folder.
If you modify it, it will work but this one is the one in danger of being overwritten on a PS update.

If you move the file to ...\plugins\Config\PythonScript\scripts\ folder then it still works just fine but is out of danger from being clobbered by a PS update.

Is that accurate information?

PeterJones

@Alan-Kilborn said in Integration of a script writen in Python:

Is that accurate information?

As far as I understand it.

The first, in ...\plugins\PythonScript\scripts\ folder, is the “machine scripts” instance; the second, in ...\plugins\Config\PythonScript\scripts\, is the “user scripts” version (using the nomenclature from the Python Script > Configuration… dialog).

Moreover, if you have both startup files, both will be listed in your Plugins > Python Script > Scripts menu, with the second (the “user scripts” version from the Config hierarchy) will have (User) appended to the displayed name:

I just ran an experiment with the two: I put in a print statement (well, console.write()) in both startup.py scripts; the machine-scripts instance runs first, followed by the user-scripts instance. (That’s what I thought happened, but wanted to make sure before saying it here, because the ...PythonScript/doc/usage.html#startup section is not explicit about it using both, or what order they run in.)

Ekopalypse

Alan, yes, it is as Peter explained.

Alan Kilborn

@Ekopalypse @PeterJones

Looking back on it, I probably knew most of that. :-)
But I did not know that the non-user startup.py can get clobbered.

So I’d say the best course of action for a normal user is to ignore the non-user (aka machine scripts) one, and, if you need to run stuff on startup, create your own startup.py (create it just like any other new script you’d make) and put your stuff in it.

The temptation is to just quickly throw stuff into the file that already exists – don’t do it.

Ekopalypse

@Alan-Kilborn - absolutely

tho-gru

Thanks for your additions.

I already used the startup (user) script.

Kind Regards,
Thomas

tho-gru

I opened the issue 205 for the difference in “preloaded” modules.

Alan Kilborn

@tho-gru

I think the reason for the “difference” was explained HERE, so you may want to withdraw your issue.

PeterJones

@tho-gru ,

To use different words than @Alan-Kilborn and @Ekopalypse have already used to try to explain this to you:

As was explained to you above by @Ekopalypse (in case @Alan-Kilborn’s link doesn’t work well – links to individual posts in the NodeBB-based forum don’t highlight the post, and sometimes don’t scroll to the part of the post you expect them to – you can look in this topic for where @Ekopalypse said, “because it is then defined in the main namespace”) before you created the issue 205, when you run a script from the PythonScript Scripts menu, it is in the __main__ namespace, so if your startup.py has already imported those default symbols, then the names are already defined in that __main__ namespace, and so are available to your script. However, when you have a line in startup.py that imports your script (the import TgrUrlAltClick), Python (not the PythonScript plugin) puts that whole file in its own namespace (TgrUrlAltClick, in your example). Thus, in the context of that imported script, the TgrUrlAltClick namespace has never imported the symbols from the Npp module, and that is why the Npp-defined objects notepad, console, and editor are not available to functions in your imported script.

The PythonScript plugin cannot and will not redefine the Python language import mechanism to inject the __main__ namespace symbols into whatever scripts you happen to import via startup.py.

The PythonScript plugin cannot and will not do any magic in startup.py that will be able to recognize when an import in startup.py is really trying to auto-execute a user-defined script rather than importing a standard Python library, so cannot and will not try to magically inject the __main__ namespace symbols into the scripts you happen to import via startup.py.

You can and should and are expected to import the symbols that you need from the Npp module into any script that you desire to import from startup.py or from any other script, because that is the only way to guarantee that any script you write will have access to those symbols whether the script is run from the PythonScript Scripts menu or by being imported as a module from another script. Adding the line from Npp import * (or a reduced version if you don’t need all the Npp symbols) into all your scripts is not an onerous task on your part, and importing modules and symbols is an expected aspect of any Python development, not just PythonScript development.

(A copy of this has also been posted in your issue)

tho-gru

@PeterJones I fully understand that I need to import symbols from other modules by typing import ....

Unfortunately (now I understand why) I did not get an error message while executing the script via the menu. My complain is not the need of importing modules I am using in my script. My complain is that I want to get same error messages independent how I can start a python script.

I currently see two possible solutions to achieve this:

hide (unload or what ever possible) the console module for scripts started via the menu
“preload” the console module to be available for all script started via the startup.py (user or system)
Of course the should be done for all “preloaded” modules available in scripts started via the menu.

This should simply remind beginners to import the modules needed.

Kind Regards
Thomas

PeterJones

@tho-gru said in Integration of a script writen in Python:

@PeterJones I fully understand that I need to import symbols from other modules by typing import ....

Not fully, because your option 2 below seemed to indicate you still think it’s possible to pre-load it.

I currently see two possible solutions to achieve this:

hide (unload or what ever possible) the console module for scripts started via the menu

Then change the from Npp import * from the machine and user startup.py to import Npp yourself. That way, the errors will come from the beginning for you, without influencing other people who do not want to have to do the imports when using the PythonScript console in immediate mode. (If you do this, calls to the standard objects inside the startup.py scripts will have to be prepended with Npp.)

“preload” the console module to be available for all script started via the startup.py (user or system)
Of course the should be done for all “preloaded” modules available in scripts started via the menu.

As already explained in detail above, but I’ll try to phrase it differently, because you have not understood, despite saying that you understand: The startup.py currently preload the Npp instance variables into the __main__ namespace, and thus for every script started from the menu, those constants are pre-loaded. But that cannot affect scripts that you import into your startup.py script, because that’s not the way Python import works – when you import a module in a given, it imports into the current namespace seen by that file; startup.py are in the __main__ namespace, so they import symbols into the __main__ namespace; scripts run from the menu are in the __main__ namespace, so they import symbols into the __main__ namespace; but a script like TgrUrlAltClick.py that is imported into some other script (like using import TgrUrlAltClick in starup.py to import a script rather than a standard module) is in its own namespace, so none of the functions defined inside TgrUrlAltClick.py will see the __main__ namespace symbols because functions inside TgrUrlAltClick.py are in the TgrUrlAltClick namespace in this condition.

What you are asking in point#2 is the equivalent of saying that when you have the following situation, it should work instead of throwing a NameError exception.

C:\usr\local\apps\python-3.9.6-embed-amd64>type mainFile.py
#!python
# encoding=utf-8

from sys import path
from MyOtherScript import myFunction
myFunction()

C:\usr\local\apps\python-3.9.6-embed-amd64>type MyOtherScript.py
#!python
# encoding=utf-8

print("Hello from namespace '{}'!\n".format(globals()['__name__']))

def myFunction():
    print(path)

C:\usr\local\apps\python-3.9.6-embed-amd64>python mainFile.py
Hello from namespace 'MyOtherScript'!

Traceback (most recent call last):
  File "C:\usr\local\apps\python-3.9.6-embed-amd64\mainFile.py", line 6, in <module>
    myFunction()
  File "C:\usr\local\apps\python-3.9.6-embed-amd64\MyOtherScript.py", line 7, in myFunction
    print(path)
NameError: name 'path' is not defined

This should simply remind beginners to import the modules needed.

Beginners don’t import their scripts into startup.py; beginners run their scripts manually from the menu. People who want to do more advanced actions, like automatically running their script from startup.py by importing the script via a command in startup.py need to learn about things like namespaces and importing, and need to actually understand those concepts … or at least be willing to take the advice to “always from Npp import * in every script if you’re going to do anything but run your script from the menu”.