Perl subroutine calltips - with PythonScript

Alan Kilborn

@PeterJones said in Perl subroutine calltips - with PythonScript:

Oh, or actually, rf"…" looks suspect. r"…" was able to run through startup.py without error.

@Michael-Vincent
If you’re running Pythonscript 1.x (can’t tell from info provided), isn’t it true that the Python 2.x it uses isn’t going to like “f-strings”?

PeterJones

@Alan-Kilborn ,

I am on PythonScript 1.5.4.

I have not yet switched to one of the v3.0.x-alpha releases.

I hadn’t noticed that in the @Ekopalypse original, he said,

You need to pip-install pyflakes and jedi and use the PS3 alpha version.

For @Michael-Vincent’s, it doesn’t look like pyflakes or jedi are required… but I am now assuming that PS3-alpha is required.

Alan Kilborn

@PeterJones

Sorry, not sure why I @Michael-Vincent -ed that last comment I made.

Michael Vincent

@PeterJones said in Perl subroutine calltips - with PythonScript:

I am on PythonScript 1.5.4.

Yes, that may be the problem, I’m using “PythonScript = 3.0.4.0” so Python3.

The error you’re getting is because I’m trying to create a REGEX with a variable. the rf"..." was recommended on StackOverflow or some other Google search for the Python3 way to do it. If you can convert that to Python2, what I’m trying to get is:

REGEX = ^\s*sub\s+ + the value of the variable subName + \s*

Cheers.

PeterJones

@Michael-Vincent ,

Yeah, I am in the midst of debugging a PS1.5.4-compatible version. I’ll probably use r"^\s*sub\s+{}\s+".format(subName) to accomplish the similar formatting.

I have found what I think is a bug, however: if sub getCMSession is the last subroutine in your module/script, then the end regex ^\s*sub\s+ will fail, so end will be 0, which means it doesn’t get the contents of the sub. I’m am working on a workaround for that.

Once I’m done, I will post perl_ide_ps154.py for those (me) who haven’t updated to PS3-alpha.

PeterJones

I was closer to having it working than I thought:

''' Basic functionality which one would expect from a full-fledged IDE

By: Notepad++ Community Forum member: @Michael-Vincent (aka vinsworldcom@perlmonks)

https://community.notepad-plus-plus.org/topic/20340/perl-subroutine-calltips-with-pythonscript


'''

import re
from Npp import (
    editor, notepad,
    SCINTILLANOTIFICATION, NOTIFICATION, LANGTYPE
    )

class PerlIDE(object):

    def __init__(self):
        self.is_perl = False
        self.excluded_styles = [1, 3, 4, 6, 7, 12, 16, 17, 18, 19]


    def initialize(self):
        editor.callbackSync(self.on_charadded, [SCINTILLANOTIFICATION.CHARADDED])
        notepad.callback(self.on_buffer_activated, [NOTIFICATION.BUFFERACTIVATED])
        self.is_perl = notepad.getCurrentLang() == LANGTYPE.PERL


    def on_buffer_activated(self, args):
        self.is_perl = notepad.getCurrentLang() == LANGTYPE.PERL
        self.path = notepad.getCurrentFilename()


    def on_charadded(self, args):
        if self.is_perl:
            c = chr(args['ch'])
            if c in '\r\n:-+*/#=)':
                return

            pos = editor.getCurrentPos()
            if editor.getStyleAt(pos) in self.excluded_styles:
                return

            if c == '(':
                # subName will contain the subroutine's name from the current typing line
                # by finding the last open paren `(` and then capturing the word (\w+)
                # before.  For example:
                #     if ( not defined( $session = getCMSession(
                # should capture `getCMSession`
                subName = '\0'
                callTip = ""
                line = editor.getCurLine()
                sub = re.search(r'^.*[\s\t\=](\w+)\s*\(', line)
                if sub:
                    subName = sub.group(1)
                else:
                    return

                # get the entire file into an array, 1 file line per array index
                lines = editor.getText().split("\n")

                # find the start of the sub, form example above, we're looking for
                # something like:
                #   sub getCMSession {
                myMatchString = r"^\s*sub\s+{}\s*".format(subName)
                start = 0
                for line, text in enumerate(lines):
                    match = re.search(myMatchString, text)
                    if match:
                        start = line
                        break
                search = lines[start+1:]

                # new array `search` now starts at the line after the subroutine start
                # we need to find the start of the next sub so we know where to stop
                # searching.  The next sub starts at a line like:
                #     "sub "
                # without the double-quotes, but the 'sub' must be followed by
                # some whitespace
                myMatchString = r'^\s*sub\s+'
                end = None
                for line, text in enumerate(search):
                    match = re.search(myMatchString, text)
                    if match:
                        end = line
                        break
                if end is None:
                    # use the whole search array if the end of the sub wasn't found
                    sub = search
                else:
                    sub = search[0:end]

                # the new array `sub` now contains each line of the full subName
                # subroutine's code, let's search for a function signature looking thing
                for line, text in enumerate(sub):
                    match = re.search(r'^\s*my\s+.*\s*\=\s*\@\_\;', text)
                    if match:
                        callTip = match.group(0)
                        editor.callTipShow(pos, callTip)
                        break

Some of the changes – especially changing the regex strings to separate variables, were done just to aid in debugging, so I could print my search string to make sure it was as expected. The critical change was using r"...".format() instead of rf"...", and changing the logic in the search-for-end that will return all the way to the EOF if it doesn’t find another sub after the start sub.

This one works for me with PS v1.5.4.

Thanks again for the starting point.

(Now you just need to come up with the logic to search all the included library modules as well… or, um, maybe not.)

Ekopalypse

Pyflakes and Jedi are only needed for Python auto-completion and syntax checking.
The f-string syntax was introduced, afair in Python 3.7 and therefore PythonScript3 would be required,
which currently uses Python 3.8 under the hood. Yes, the syntax '{}...'.format(...) can be used with PS 1.5

Michael Vincent

@PeterJones said in Perl subroutine calltips - with PythonScript:

(Now you just need to come up with the logic to search all the included library modules as well… or, um, maybe not.)

I thought I read somewhere: “only perl can parse Perl”. I originally wrote this in PerlScript since it was more familiar to me and then just had my PythonScript so a subprocess(['perl', 'perl_ide.pl' ... but that was awful slow and launched a CMD window - so I ported the whole thing to Python.

I’ve been looking at Perl Language Servers but that would require a Notepad++Language Server client (the topic in the post I linked to at the beginning). One mentions integrating plsense which looks amazing!

I liked @Ekopalypse approach of this lightweight way to go with a scripting plugin. I have my cTags plugin helping with autocomplete across my Perl scripts in the same hierarchy and NppEventExec can be set to perl -c on file save. Not sure what else we can do to parse Perl with Python to make this “Perl IDE” better, but certainly open to suggestions!

Cheers.

Ekopalypse

@Michael-Vincent

does perl, itself, has a feature of parsing. By that I mean builtin not by
using perl -module.pm script.pm

PeterJones

@Ekopalypse ,

“builtin” – not quite; perl -c does a compile check, but doesn’t give you the full parse…

There is the CORE module B::Deparse, which runs as perl -MO=Deparse script.pl, which comes closer – though I’ve never used it except when replicating experiments shown by perlmonks experts, so I don’t really know how to use it. (CORE is defined as the set of modules that ship with Perl, by default, and are normally assumed to be present; though there are minimalist distributions of Perl which don’t include all of CORE.)

But with Perl, you can do some pretty powerful stuff (like define an anonymous subroutine, then manually assign it to a slot in any namespace, so that from the user’s point of view, there is no difference between that and a subroutine that had been defined by a traditional sub name { ... } block. I don’t know how well Deparse works with those.

The thought behind “only perl can parse Perl” is that, by Perl’s very nature, if you create an executable that’s capable of parsing any arbitrary correct Perl code, that executable is effectively an implementation of the perl executable.

Michael Vincent

@Ekopalypse said in Perl subroutine calltips - with PythonScript:

does perl, itself, has a feature of parsing

@PeterJones probably knows better than I (EDIT: HE ALSO TYPES FASTER!!!) but I don’t think so. perl -c will “compile / check” to make sure your code is valid without actually running it. That would be the equivalent to maybe the “lint” feature in your Python IDE script.

To actually parse the Perl and return methods for objects, you’d need something like plsense modified to work in this environment. There is also Perl::LanguageServer which seems to be the most complete Perl Language Server but note it does not work on Windows (so says the README) and it requires a Language Server Client - NppLSP - but as discussed, it’s very alpha.

Cheers.

Ekopalypse

i was thinking - can we use perl.dll in some way?
It seems to export a lot of stuff.

Ekopalypse

it has

...
 806  325 00023D20 Perl_parse_arithexpr
 807  326 00023D90 Perl_parse_barestmt
 808  327 00023D60 Perl_parse_block
 809  328 00023D50 Perl_parse_fullexpr
 810  329 00024130 Perl_parse_fullstmt
 811  32A 00023DC0 Perl_parse_label
 812  32B 00023D40 Perl_parse_listexpr
 813  32C 00024160 Perl_parse_stmtseq
 814  32D 00023D30 Perl_parse_termexpr
 815  32E 00039A50 Perl_parse_uniprop_string
 816  32F 0009B3C0 Perl_parser_dup
...

Ekopalypse

@Ekopalypse

there is even an exported function RunPerl

EXTERN_C DllExport int
RunPerl(int argc, char **argv, char **env)
{
    int exitstatus;
    PerlInterpreter *my_perl, *new_perl = NULL;
    bool use_environ = (env == environ);

#ifdef PERL_GLOBAL_STRUCT
#define PERLVAR(prefix,var,type) /**/
#define PERLVARA(prefix,var,type) /**/
#define PERLVARI(prefix,var,type,init) PL_Vars.prefix##var = init;
#define PERLVARIC(prefix,var,type,init) PL_Vars.prefix##var = init;
#include "perlvars.h"
#undef PERLVAR
#undef PERLVARA
#undef PERLVARI
#undef PERLVARIC
#endif

    PERL_SYS_INIT(&argc,&argv);

    if (!(my_perl = perl_alloc()))
	return (1);
    perl_construct(my_perl);
    PL_perl_destruct_level = 0;

    /* PERL_SYS_INIT() may update the environment, e.g. via ansify_path().
     * This may reallocate the RTL environment block. Therefore we need
     * to make sure that `env` continues to have the same value as `environ`
     * if we have been called this way.  If we have been called with any
     * other value for `env` then all environment munging by PERL_SYS_INIT()
     * will be lost again.
     */
    if (use_environ)
        env = environ;

    if (!perl_parse(my_perl, xs_init, argc, argv, env)) {
#if defined(TOP_CLONE) && defined(USE_ITHREADS)		/* XXXXXX testing */
	new_perl = perl_clone(my_perl, 1);
	(void) perl_run(new_perl);
	PERL_SET_THX(my_perl);
#else
	(void) perl_run(my_perl);
#endif
    }

    exitstatus = perl_destruct(my_perl);
    perl_free(my_perl);
#ifdef USE_ITHREADS
    if (new_perl) {
	PERL_SET_THX(new_perl);
	exitstatus = perl_destruct(new_perl);
	perl_free(new_perl);
    }
#endif

    PERL_SYS_TERM();

    return (exitstatus);
}

PeterJones

@Ekopalypse ,

I don’t know the guts (I was going to say “very well”, but that would be understatement; I think the unqualified comes closest to the truth), but perldoc perlembed shows how to embed the perl interpreter into an application, and takes a bit about perl_parse and perl_run.

But I wouldn’t know where to begin on using those functions, or even the output of -MO=Deparse, and implementing a stronger version of what @Michael-Vincent has shown. And even if I did, I already have my huge project of the Win32::Mechanize::NotepadPlusPlus which isn’t quite to where I want it, and the PerlScript plugin wrapper that I haven’t even started writing yet – and I cannot take on another big project until I’ve got those to a reasonably-functional level.

Ekopalypse

@PeterJones

no problem … I was curious and played already a bit with it.
If I’m right, then 5 lines of pythonscript might give you an embedded perl.

PeterJones

@Ekopalypse said in Perl subroutine calltips - with PythonScript:

5 lines of pythonscript might give you an embedded perl.

Oh, the irony. :-)

Ekopalypse

@PeterJones

:-D

this seems to do something, not sure what but it doesn’t crash my npp … so … still crossing fingers ;-)

from ctypes import CDLL, POINTER, c_int, c_char_p

perllib = CDLL(r'd:\perl\bin\perl531.dll')

["-le", "print 'something \"with\" quotes';"]

perllib.RunPerl.restype = c_int
perllib.RunPerl.argtypes = c_int, POINTER(c_char_p), POINTER(c_char_p)
args = (c_char_p * 2)(b"-le", b"print 'something \"with\" quotes';")
print(perllib.RunPerl(len(args),args, None))

Ekopalypse

@PeterJones

can you give me a qiuck “Win32::Mechanize::NotepadPlusPlus for dummies äh ekopalypses” to open a new npp buffer?

PeterJones

@Ekopalypse ,

From the command line, to just open a new file tab:

perl -le "use Win32::Mechanize::NotepadPlusPlus qw/:main/; notepad->newFile();"

If you want to call a script.pl instead,

use Win32::Mechanize::NotepadPlusPlus qw/:main/; 
notepad->newFile();

But that assumes that perl has the right @INC (include library paths) to find Win32::Mechanize::NotepadPlusPlus.