RFC: Track previous position

Michael Vincent

The Location Navigate plugin provides a position tracking for all the previous places your caret was while navigating a document or series of documents. I believe it’s based on number of Scintilla character positions away from the previous position that it logs a new entry.

I could never get Location Navigate to work for me so created Changed Lines plugin heavily borrowing from Location Navigate only the essential change tracking (margin coloring) features. I then added a circular buffer to track previous positions based on a WM_TIMER of 2 seconds of “lingering” meant log the position. It “worked” - meaning worked good enough for me, with some bugginess but overall OK for the times I needed it.

Both Location Navigate and Changed Lines are pretty much useless now since Scintilla and N++ have integrated native change history tracking. You can even navigate between the changes - my Changed Lines plugin add a few features to this as mainly now I just use it to augment native change history - all of its change history tracking is disabled - and I keep it to keep the previous location tracking. I can’t help but wonder if there is a better way.

The Scintilla change history is based on the undo buffer but that is not exposed in a way to find the position of undos - though I did ask. Barring that, a plugin for this functionality seems the way to go.

I thought about doing it in PythonScript - much easier for me to “memory manage” the data structure:

filename
line number
currently opened or closed (to skip if closed)
timestamp (maybe to age out)
…

And of course implementation details:

how long is the list
in memory or write to disk / database
keep on close / re-open of Notepad++ (implies a save to disk at session close)
…

And the biggest problem is triggering a location save. I did it with a WM_TIMER event and hooking SCN_MODIFIED but the Scintilla notification fires for every single typed character and the SCN_DWELL events seem nice, but they are mouse only. I don’t like the idea of a position differential like Location Navigate used as that is highly subjective based on line lengths in files (think long lines log file and you may have successive positions on the same line - which may be useful??).

And of course adding lines to a file would mean revisiting each logged entry and adjusting the “line” number saved so it reflects its new location - something Location Navigate tried to do but I never bothered in my implementation (too hard).

I’m not asking anyone to write this - I already did (not well) in my plugin and have a good idea on how to implement in PythonScript if I choose to. This is as the title implies a “Request for Comments” - any thoughts? Does anyone like / need / use this feature with a plugin or in another editor (e.g., VS Code has something similar)? Do you think PythonScript could handle it or better to keep in a “proper” C++ plugin?

Discuss …

Cheers.

Alan Kilborn

@Michael-Vincent

I used Location Navigate long ago, strictly for its changed-lines aspect. As soon as I knew about your Changed Lines plugin, I switched to that and forgot about Location Navigate.

What you’re proposing is bringing back the core functionality of Location Navigate – the ability to jump around (presumably backward and forward) through where you’ve been (your caret, that is).

I don’t have this need, but, like other things I think I don’t need, maybe I would like your implementation and use it.

As far as what to develop it in, scripting (in Python) versus true plugin (in presumably C++), there are pros and cons each way.

Let’s face it, there are going to be people that might want the functionality that would not go near it if it were a script. Scripting would likely be easier to develop. Like I said, pros and cons…

But, I applaud your apparent enthusiasm, so whichever way you go, you’ll learn some things, and hopefully something good comes out the other end when you’re done. :-)

Michael Vincent

@Alan-Kilborn said in RFC: Track previous position:

As soon as I knew about your Changed Lines plugin, I switched to that

Did you ever use the position history feature of Changed Lines?

Or maybe as I said it was so buggy that you tried and didn’t like it? I’m not sure it’s worth saving vs. starting anew - and if starting fresh, I think PythonScript is the way to go to get a proof of concept working. Though anytime I get something working in PythonScript - I just keep using it since it’s so seamless - it’s like a plugin factory for me!

Cheers.

Alan Kilborn

@Michael-Vincent said in RFC: Track previous position:

Did you ever use the position history feature of Changed Lines?

Nope…because I never felt I needed it. I did, however, jump between changed-lines sections, but that’s a different thing.

I think PythonScript is the way to go to get a proof of concept working.

I think this is an exceptionally good idea. :-)

Ekopalypse

I would say that this is exactly the area where PS is at its best as it performs one step per action and in this case no slowdown is to be expected, but in my usage I usually need a more programming language oriented solution like “jump to the definition of the function”. So using TreeSitter approaches is more the direction for me as I hardly ever edit different positions in a file that are out of context and then have to jump back there.

Michael Vincent

@Ekopalypse said in RFC: Track previous position:

TreeSitter

Do you have a plug-in or PythonScript that integrates TreeSitter with N++?

Cheers.

Michael Vincent

@Michael-Vincent said in RFC: Track previous position:

PythonScript that integrates TreeSitter with N++

I wrote a very simple syntax highlighting one. Use Plugins => Python Script => New Script and save as tree-sitter.py:

import re

from Npp import editor, editor1, editor2, notepad, SCINTILLANOTIFICATION

class TreeSitter():
    """
    Follow a `tree-sitter` parse output (in `editor2`) and highlight syntax
    (in `editor1`).
    """
    def __init__(self):
        super().__init__()
        self._edCallbacks = {
            self._on_updateui: [
                SCINTILLANOTIFICATION.UPDATEUI
            ]
        }
        self._ENABLED = False
        self.DEBUG = False

    def _on_updateui(self, args):
        if notepad.getCurrentView() != 1:
            return

        if self.DEBUG:
            print(args)
        if args['hwndFrom'] != editor2.hwnd:
            return

        line = editor2.getCurLine()
        match = re.search(r'(\w+)\s\[(\d+)\,\s(\d+)\]\s\-\s\[(\d+)\,\s(\d+)\]', line)

        if (match is None) or (len(match.groups()) != 5):
            return

        # nm = match.group(1)
        sl = int(match.group(2))
        sc = int(match.group(3))
        el = int(match.group(4))
        ec = int(match.group(5))
        if self.DEBUG:
            print(f"[{sl}, {sc}] - [{el}, {ec}]")

        editor1.gotoLine(sl)
        spos = editor1.findColumn(sl, sc)
        editor1.gotoLine(el)
        epos = editor1.findColumn(el, ec)

        editor1.setSelectionStart(spos)
        editor1.setSelectionEnd(epos)

    def start(self):
        """Start the service"""
        for cb in self._edCallbacks:
            editor.callback(cb, self._edCallbacks[cb])
        self._ENABLED = True

    def stop(self):
        """Stop the service"""
        for cb in self._edCallbacks:
            editor.clearCallbacks(cb)
        self._ENABLED = False

if __name__ == '__main__':
    try:
        isinstance(ts, TreeSitter)
        print("Tree-Sitter `ts' already enabled")
    except NameError:
        ts = TreeSitter()
        ts.start()

To use, you’ll need to have Tree-Sitter installed for Windows and have a proper parsing library installed as well. That is a bit of a headache, so here’s what I did:

Download the latest tree-sitter binary release for Windows
Put it in a directory - I used ‘C:\usr\bin\tree-sitter’
unzip the .gz file
rename the extracted file to tree-sitter.exe
create a directory named ‘src’ in the directory where tree-sitter.exe resides
run tree-sitter.exe init-config
that should tell you it created a file config.json in %USERPROFILE%\AppData\Roaming\tree-sitter
open that file in Notepad++ and edit the “parser-directories” to contain only a single directory - the path to the ‘src’ directory you created in step 5. For my example, the entry looks like:

{
  "parser-directories": [
    "C:\\usr\\bin\\tree-sitter\\src"
  ],

cd into the “src” directory and git clone some tree-sitter grammar parsers for languages you are interested in, for example:

git clone https://github.com/tree-sitter/tree-sitter-c
git clone https://github.com/tree-sitter/tree-sitter-cpp
git clone https://github.com/tree-sitter/tree-sitter-python

You should be ready to go now. Take a simple Python file - you can use the tree-sitter.py PythonScript from above.

Run Tree-Sitter on that file and output to a file:

tree-sitter.exe parse $(NPP_DIRECTORY)\plugins\Config\PythonScript\scripts\tree-sitter.py > %TEMP%\tree-sitter.trs

where $(NPP_DIRECTORY) is the path to your Notepad++ directory where the tree-sitter.py script is. Note, you can use any Python file to parse for grammar, you don’t need to use that one, I’m just using it as an example as I know it will exist on your system. The %TEMP% is your Windows temporary files directory.

Open both the file you parsed (e.g., tree-sitter.py) and the output (e.g., tree-sitter.trs) in Notepad++. Put the Python file in the Editor1 spot (usually the left view) and put the TRS file in the Editor2 spot (usually the right view). Run the PythonScript from the menu - “Plugins => Python Script => Scripts => tree-sitter”.

As you scroll through the TRS file, the appropriate sections of the Python file should highlight:

When you are done, you can use ts.stop() in the PythonScript console to remove the callbacks.

Cheers.

Ekopalypse

@Michael-Vincent

I just install it via pip install tree-sitter and pip install tree-sitter-languages and of course use PS3 with the preferred local installation to access it.

Btw … I made a small demo here but you’ve already mastered it :-)

Michael Vincent

@Ekopalypse said in RFC: Track previous position:

I just install it via pip install tree-sitter and pip install tree-sitter-languages and of course use PS3 with the preferred local installation to access it.

Much nicer. Modified script playground.py to work with that setup:

import os
import re
from tree_sitter_languages import get_parser

from Npp import editor, editor1, editor2, notepad, MENUCOMMAND, SCINTILLANOTIFICATION

class TreeSitterPlayground():
    """
    Follow a `tree-sitter` parse output (in `editor2`) and highlight syntax
    (in `editor1`).
    """
    def __init__(self):
        super().__init__()
        self._edCallbacks = {
            self._on_updateui: [
                SCINTILLANOTIFICATION.UPDATEUI
            ]
        }
        self._cst = []

    def _on_updateui(self, args):
        if notepad.getCurrentView() != 1:
            return

        if self.DEBUG:
            print(args)
        if args['hwndFrom'] != editor2.hwnd:
            return

        line = editor2.getCurLine()
        match = re.search(r'Node\stype\=(.*?)\,\sstart_point\=\((\d+)\,\s(\d+)\)\,\send_point\=\((\d+)\,\s(\d+)\)', line)
        if (match is None) or (len(match.groups()) != 5):
            return

        # nm = match.group(1)
        sl = int(match.group(2))
        sc = int(match.group(3))
        el = int(match.group(4))
        ec = int(match.group(5))
        if self.DEBUG:
            print(f"[{sl}, {sc}] - [{el}, {ec}]")

        editor1.gotoLine(sl)
        spos = editor1.findColumn(sl, sc)
        editor1.gotoLine(el)
        epos = editor1.findColumn(el, ec)

        editor1.setSelectionStart(spos)
        editor1.setSelectionEnd(epos)

    def _dump_tree(self, node, indent=0):
        self._cst.append(f"{' '*indent}{node}")
        if node.child_count > 0:
            for c in node.children:
                self._dump_tree(c, indent+1)

    def parse(self):
        """Parse the current file."""
        self._cst = []

        # file name, path, extension parsing
        full = notepad.getCurrentFilename()
        path, file = os.path.split(full)
        name, ext = os.path.splitext(file)

        # get file language type and adjust for Tree-Sitter parser names
        filetype = notepad.getLanguageName(notepad.getLangType()).lower()
        if filetype == 'c++':
            filetype = 'cpp'
        elif filetype == 'shell':
            filetype = 'bash'

        # parse the file
        try:
            tree = get_parser(filetype).parse(editor.getText().encode())
        except:
            print(f"tree_sitter_languages `{filetype}' not found")
            return

        # recurse the CST, create the output string and put it in N++
        self._dump_tree(tree.root_node)
        cst = '\n'.join(self._cst)
        notepad.new()
        editor.setText(cst)
        notepad.saveAs(f"{os.environ['TEMP']}/{name}.trs")

        # Make sure the CST is in view 1 and the source file is in view 0
        if notepad.getCurrentView() == 0:
            notepad.menuCommand(MENUCOMMAND.VIEW_GOTO_ANOTHER_VIEW)
        else:
            notepad.open(full)
            notepad.menuCommand(MENUCOMMAND.VIEW_GOTO_ANOTHER_VIEW)
            notepad.open(f"{os.environ['TEMP']}/{name}.trs")
 
    def start(self):
        """Start the service"""
        for cb in self._edCallbacks:
            editor.callback(cb, self._edCallbacks[cb])
        self._ENABLED = True

    def stop(self):
        """Stop the service"""
        for cb in self._edCallbacks:
            editor.clearCallbacks(cb)
        self._ENABLED = False
                   
if __name__ == '__main__':
    try:
        isinstance(tsp, TreeSitterPlayground)
        print("Tree-Sitter Playground `tsp' already enabled")
    except NameError:
        tsp = TreeSitterPlayground()
        tsp.start()
        print("Tree-Sitter Playground instantiated as `tsp'")

    tsp.parse()

Now, just open a file in Notepad++, run the “Plugins => Python Script => Scripts => playground” and it will try to parse the current file and setup the output CST (syntax tree) in view 1 and the source file in view 0 to allow scrolling through the CST and auto-highlighting the source in view 0 as per my screenshot in the previous post.

Cheers.