Perl subroutine calltips - with PythonScript



  • Taking inspiration from @Ekopalypse Python “IDE” I’ve been playing with, I tried to use PythonScript to create a Perl subroutine calltip provider for the current Perl script. It “works”:

    11f4dde4-5155-4d7f-bd08-4c0d94d953bf-image.png

    Typing the ( open parenthesis triggers the lookup and as you can see in the example, the function “signature” is displayed in the calltip.

    In case you’re interested, I have “perl_ide.py” in my $(NPP_DIR)/plugins/PythonScript/scripts directory and load it in my “startup.py” script in the same directory with:

    from perl_ide import PerlIDE
    __plide = PerlIDE()
    __plide.initialize()
    

    And the script itself - feel free to offer improvements - I’m not a Python expert by any means!

    ''' Basic functionality which one would expect from a full-fledged IDE '''
    import re
    from Npp import (
        editor, notepad,
        SCINTILLANOTIFICATION, NOTIFICATION, LANGTYPE
        )
    
    class PerlIDE(object):
    
        def __init__(self):
            self.is_perl = False
            self.excluded_styles = [1, 3, 4, 6, 7, 12, 16, 17, 18, 19]
    
    
        def initialize(self):
            editor.callbackSync(self.on_charadded, [SCINTILLANOTIFICATION.CHARADDED])
            notepad.callback(self.on_buffer_activated, [NOTIFICATION.BUFFERACTIVATED])
            self.is_perl = notepad.getCurrentLang() == LANGTYPE.PERL
    
    
        def on_buffer_activated(self, args):
            self.is_perl = notepad.getCurrentLang() == LANGTYPE.PERL
            self.path = notepad.getCurrentFilename()
    
    
        def on_charadded(self, args):
            if self.is_perl:
                c = chr(args['ch'])
                if c in '\r\n:-+*/#=)':
                    return
    
                pos = editor.getCurrentPos()
                if editor.getStyleAt(pos) in self.excluded_styles:
                    return
    
                if c == '(':
                    # subName will contain the subroutine's name from the current typing line 
                    # by finding the last open paren `(` and then capturing the word (\w+) 
                    # before.  For example:
                    #     if ( not defined( $session = getCMSession(
                    # should capture `getCMSession`     
                    subName = '\0'
                    callTip = ""
                    line = editor.getCurLine()
                    sub = re.search('^.*[\s\t\=](\w+)\s*\(', line)
                    if sub:
                        subName = sub.group(1)
                    else:
                        return
    
                    # get the entire file into an array, 1 file line per array index
                    lines = editor.getText().split("\n")
    
                    # find the start of the sub, form example above, we're looking for 
                    # something like:
                    #   sub getCMSession {
                    start = 0
                    for line, text in enumerate(lines):
                        match = re.search(rf"^\s*sub\s+{subName}\s*", text)
                        if match:
                            start = line
                            break
                    search = lines[start+1:]
    
                    # new array `search` now starts at the line after the subroutine start
                    # we need to find the start of the next sub so we know where to stop 
                    # searching.  The next sub starts at a line like:
                    #     "sub "
                    # without the double-quotes, but the 'sub' must be followed by 
                    # some whitespace
                    end = 0
                    for line, text in enumerate(search):
                        match = re.search('^\s*sub\s+', text)
                        if match:
                            end = line
                            break
                    sub = search[0:end]
    
                    # the new array `sub` now contains each line of the full subName 
                    # subroutine's code, let's search for a function signature looking thing
                    for line, text in enumerate(sub):
                        match = re.search('^\s*my\s+.*\s*\=\s*\@\_\;', text)
                        if match:
                            callTip = match.group(0)
                            editor.callTipShow(pos, callTip)
                            break
    
    

    Cheers.



  • @Michael-Vincent said in Perl subroutine calltips - with PythonScript:

      File "C:\usr\local\apps\notepad++\plugins\Config\PythonScript\scripts\perl_ide.py", line 67
        match = re.search(rf"^\s*sub\s+{subName}\s*", text)
                                                   ^
    SyntaxError: invalid syntax
    

    Did some brackets [] or backslashes \\ disappear? This is what it looks like in the Forum:
    2eb67e9e-f74a-4572-91ed-38b6832b656b-image.png

    Oh, or actually, rf"..." looks suspect. r"..." was able to run through startup.py without error.

    Unfortunately, changing that to r"..." didn’t work – I don’t get a calltip… (and changing it to f"..." gives the syntax error) =
    2b80b6b6-b7de-4b94-bc7e-c524da900c08-image.png

    Any idea what I’ve done wrong?

    edit: I’ve tried both with Auto-completion on the () enabled and disabled., and auto-completion-on-each-input enabled and disabled… tried just in case those were interfering**/edit**



  • @PeterJones said in Perl subroutine calltips - with PythonScript:

    Oh, or actually, rf"…" looks suspect. r"…" was able to run through startup.py without error.

    @Michael-Vincent
    If you’re running Pythonscript 1.x (can’t tell from info provided), isn’t it true that the Python 2.x it uses isn’t going to like “f-strings”?



  • @Alan-Kilborn ,

    I am on PythonScript 1.5.4.

    I have not yet switched to one of the v3.0.x-alpha releases.

    I hadn’t noticed that in the @Ekopalypse original, he said,

    You need to pip-install pyflakes and jedi and use the PS3 alpha version.

    For @Michael-Vincent’s, it doesn’t look like pyflakes or jedi are required… but I am now assuming that PS3-alpha is required.



  • @PeterJones

    Sorry, not sure why I @Michael-Vincent -ed that last comment I made.



  • @PeterJones said in Perl subroutine calltips - with PythonScript:

    I am on PythonScript 1.5.4.

    Yes, that may be the problem, I’m using “PythonScript = 3.0.4.0” so Python3.

    The error you’re getting is because I’m trying to create a REGEX with a variable. the rf"..." was recommended on StackOverflow or some other Google search for the Python3 way to do it. If you can convert that to Python2, what I’m trying to get is:

    REGEX = ^\s*sub\s+ + the value of the variable subName + \s*

    Cheers.



  • @Michael-Vincent ,

    Yeah, I am in the midst of debugging a PS1.5.4-compatible version. I’ll probably use r"^\s*sub\s+{}\s+".format(subName) to accomplish the similar formatting.

    I have found what I think is a bug, however: if sub getCMSession is the last subroutine in your module/script, then the end regex ^\s*sub\s+ will fail, so end will be 0, which means it doesn’t get the contents of the sub. I’m am working on a workaround for that.

    Once I’m done, I will post perl_ide_ps154.py for those (me) who haven’t updated to PS3-alpha.



  • I was closer to having it working than I thought:
    b51664ca-c01d-41d0-9461-dbcaa5f77421-image.png

    ''' Basic functionality which one would expect from a full-fledged IDE
    
    By: Notepad++ Community Forum member: @Michael-Vincent (aka vinsworldcom@perlmonks)
    
    https://community.notepad-plus-plus.org/topic/20340/perl-subroutine-calltips-with-pythonscript
    
    
    '''
    
    import re
    from Npp import (
        editor, notepad,
        SCINTILLANOTIFICATION, NOTIFICATION, LANGTYPE
        )
    
    class PerlIDE(object):
    
        def __init__(self):
            self.is_perl = False
            self.excluded_styles = [1, 3, 4, 6, 7, 12, 16, 17, 18, 19]
    
    
        def initialize(self):
            editor.callbackSync(self.on_charadded, [SCINTILLANOTIFICATION.CHARADDED])
            notepad.callback(self.on_buffer_activated, [NOTIFICATION.BUFFERACTIVATED])
            self.is_perl = notepad.getCurrentLang() == LANGTYPE.PERL
    
    
        def on_buffer_activated(self, args):
            self.is_perl = notepad.getCurrentLang() == LANGTYPE.PERL
            self.path = notepad.getCurrentFilename()
    
    
        def on_charadded(self, args):
            if self.is_perl:
                c = chr(args['ch'])
                if c in '\r\n:-+*/#=)':
                    return
    
                pos = editor.getCurrentPos()
                if editor.getStyleAt(pos) in self.excluded_styles:
                    return
    
                if c == '(':
                    # subName will contain the subroutine's name from the current typing line
                    # by finding the last open paren `(` and then capturing the word (\w+)
                    # before.  For example:
                    #     if ( not defined( $session = getCMSession(
                    # should capture `getCMSession`
                    subName = '\0'
                    callTip = ""
                    line = editor.getCurLine()
                    sub = re.search(r'^.*[\s\t\=](\w+)\s*\(', line)
                    if sub:
                        subName = sub.group(1)
                    else:
                        return
    
                    # get the entire file into an array, 1 file line per array index
                    lines = editor.getText().split("\n")
    
                    # find the start of the sub, form example above, we're looking for
                    # something like:
                    #   sub getCMSession {
                    myMatchString = r"^\s*sub\s+{}\s*".format(subName)
                    start = 0
                    for line, text in enumerate(lines):
                        match = re.search(myMatchString, text)
                        if match:
                            start = line
                            break
                    search = lines[start+1:]
    
                    # new array `search` now starts at the line after the subroutine start
                    # we need to find the start of the next sub so we know where to stop
                    # searching.  The next sub starts at a line like:
                    #     "sub "
                    # without the double-quotes, but the 'sub' must be followed by
                    # some whitespace
                    myMatchString = r'^\s*sub\s+'
                    end = None
                    for line, text in enumerate(search):
                        match = re.search(myMatchString, text)
                        if match:
                            end = line
                            break
                    if end is None:
                        # use the whole search array if the end of the sub wasn't found
                        sub = search
                    else:
                        sub = search[0:end]
    
                    # the new array `sub` now contains each line of the full subName
                    # subroutine's code, let's search for a function signature looking thing
                    for line, text in enumerate(sub):
                        match = re.search(r'^\s*my\s+.*\s*\=\s*\@\_\;', text)
                        if match:
                            callTip = match.group(0)
                            editor.callTipShow(pos, callTip)
                            break
    

    Some of the changes – especially changing the regex strings to separate variables, were done just to aid in debugging, so I could print my search string to make sure it was as expected. The critical change was using r"...".format() instead of rf"...", and changing the logic in the search-for-end that will return all the way to the EOF if it doesn’t find another sub after the start sub.

    This one works for me with PS v1.5.4.

    Thanks again for the starting point.

    (Now you just need to come up with the logic to search all the included library modules as well… or, um, maybe not.)



  • Pyflakes and Jedi are only needed for Python auto-completion and syntax checking.
    The f-string syntax was introduced, afair in Python 3.7 and therefore PythonScript3 would be required,
    which currently uses Python 3.8 under the hood. Yes, the syntax '{}...'.format(...) can be used with PS 1.5



  • @PeterJones said in Perl subroutine calltips - with PythonScript:

    (Now you just need to come up with the logic to search all the included library modules as well… or, um, maybe not.)

    I thought I read somewhere: “only perl can parse Perl”. I originally wrote this in PerlScript since it was more familiar to me and then just had my PythonScript so a subprocess(['perl', 'perl_ide.pl' ... but that was awful slow and launched a CMD window - so I ported the whole thing to Python.

    I’ve been looking at Perl Language Servers but that would require a Notepad++Language Server client (the topic in the post I linked to at the beginning). One mentions integrating plsense which looks amazing!

    I liked @Ekopalypse approach of this lightweight way to go with a scripting plugin. I have my cTags plugin helping with autocomplete across my Perl scripts in the same hierarchy and NppEventExec can be set to perl -c on file save. Not sure what else we can do to parse Perl with Python to make this “Perl IDE” better, but certainly open to suggestions!

    Cheers.



  • @Michael-Vincent

    does perl, itself, has a feature of parsing. By that I mean builtin not by
    using perl -module.pm script.pm



  • @Ekopalypse ,

    “builtin” – not quite; perl -c does a compile check, but doesn’t give you the full parse…

    There is the CORE module B::Deparse, which runs as perl -MO=Deparse script.pl, which comes closer – though I’ve never used it except when replicating experiments shown by perlmonks experts, so I don’t really know how to use it. (CORE is defined as the set of modules that ship with Perl, by default, and are normally assumed to be present; though there are minimalist distributions of Perl which don’t include all of CORE.)

    But with Perl, you can do some pretty powerful stuff (like define an anonymous subroutine, then manually assign it to a slot in any namespace, so that from the user’s point of view, there is no difference between that and a subroutine that had been defined by a traditional sub name { ... } block. I don’t know how well Deparse works with those.

    The thought behind “only perl can parse Perl” is that, by Perl’s very nature, if you create an executable that’s capable of parsing any arbitrary correct Perl code, that executable is effectively an implementation of the perl executable.



  • @Ekopalypse said in Perl subroutine calltips - with PythonScript:

    does perl, itself, has a feature of parsing

    @PeterJones probably knows better than I (EDIT: HE ALSO TYPES FASTER!!!) but I don’t think so. perl -c will “compile / check” to make sure your code is valid without actually running it. That would be the equivalent to maybe the “lint” feature in your Python IDE script.

    To actually parse the Perl and return methods for objects, you’d need something like plsense modified to work in this environment. There is also Perl::LanguageServer which seems to be the most complete Perl Language Server but note it does not work on Windows (so says the README) and it requires a Language Server Client - NppLSP - but as discussed, it’s very alpha.

    Cheers.



  • i was thinking - can we use perl.dll in some way?
    It seems to export a lot of stuff.



  • it has

    ...
     806  325 00023D20 Perl_parse_arithexpr
     807  326 00023D90 Perl_parse_barestmt
     808  327 00023D60 Perl_parse_block
     809  328 00023D50 Perl_parse_fullexpr
     810  329 00024130 Perl_parse_fullstmt
     811  32A 00023DC0 Perl_parse_label
     812  32B 00023D40 Perl_parse_listexpr
     813  32C 00024160 Perl_parse_stmtseq
     814  32D 00023D30 Perl_parse_termexpr
     815  32E 00039A50 Perl_parse_uniprop_string
     816  32F 0009B3C0 Perl_parser_dup
    ...
    


  • @Ekopalypse

    there is even an exported function RunPerl

    EXTERN_C DllExport int
    RunPerl(int argc, char **argv, char **env)
    {
        int exitstatus;
        PerlInterpreter *my_perl, *new_perl = NULL;
        bool use_environ = (env == environ);
    
    #ifdef PERL_GLOBAL_STRUCT
    #define PERLVAR(prefix,var,type) /**/
    #define PERLVARA(prefix,var,type) /**/
    #define PERLVARI(prefix,var,type,init) PL_Vars.prefix##var = init;
    #define PERLVARIC(prefix,var,type,init) PL_Vars.prefix##var = init;
    #include "perlvars.h"
    #undef PERLVAR
    #undef PERLVARA
    #undef PERLVARI
    #undef PERLVARIC
    #endif
    
        PERL_SYS_INIT(&argc,&argv);
    
        if (!(my_perl = perl_alloc()))
    	return (1);
        perl_construct(my_perl);
        PL_perl_destruct_level = 0;
    
        /* PERL_SYS_INIT() may update the environment, e.g. via ansify_path().
         * This may reallocate the RTL environment block. Therefore we need
         * to make sure that `env` continues to have the same value as `environ`
         * if we have been called this way.  If we have been called with any
         * other value for `env` then all environment munging by PERL_SYS_INIT()
         * will be lost again.
         */
        if (use_environ)
            env = environ;
    
        if (!perl_parse(my_perl, xs_init, argc, argv, env)) {
    #if defined(TOP_CLONE) && defined(USE_ITHREADS)		/* XXXXXX testing */
    	new_perl = perl_clone(my_perl, 1);
    	(void) perl_run(new_perl);
    	PERL_SET_THX(my_perl);
    #else
    	(void) perl_run(my_perl);
    #endif
        }
    
        exitstatus = perl_destruct(my_perl);
        perl_free(my_perl);
    #ifdef USE_ITHREADS
        if (new_perl) {
    	PERL_SET_THX(new_perl);
    	exitstatus = perl_destruct(new_perl);
    	perl_free(new_perl);
        }
    #endif
    
        PERL_SYS_TERM();
    
        return (exitstatus);
    }
    


  • @Ekopalypse ,

    I don’t know the guts (I was going to say “very well”, but that would be understatement; I think the unqualified comes closest to the truth), but perldoc perlembed shows how to embed the perl interpreter into an application, and takes a bit about perl_parse and perl_run.

    But I wouldn’t know where to begin on using those functions, or even the output of -MO=Deparse, and implementing a stronger version of what @Michael-Vincent has shown. And even if I did, I already have my huge project of the Win32::Mechanize::NotepadPlusPlus which isn’t quite to where I want it, and the PerlScript plugin wrapper that I haven’t even started writing yet – and I cannot take on another big project until I’ve got those to a reasonably-functional level.



  • @PeterJones

    no problem … I was curious and played already a bit with it.
    If I’m right, then 5 lines of pythonscript might give you an embedded perl.



  • @Ekopalypse said in Perl subroutine calltips - with PythonScript:

    5 lines of pythonscript might give you an embedded perl.

    Oh, the irony. :-)



  • @PeterJones

    :-D

    this seems to do something, not sure what but it doesn’t crash my npp … so … still crossing fingers ;-)

    from ctypes import CDLL, POINTER, c_int, c_char_p
    
    perllib = CDLL(r'd:\perl\bin\perl531.dll')
    
    ["-le", "print 'something \"with\" quotes';"]
    
    perllib.RunPerl.restype = c_int
    perllib.RunPerl.argtypes = c_int, POINTER(c_char_p), POINTER(c_char_p)
    args = (c_char_p * 2)(b"-le", b"print 'something \"with\" quotes';")
    print(perllib.RunPerl(len(args),args, None))
    

Log in to reply