Microsoft Language Server Protocol (LSP): A possible revolution for Notepad++?



  • I’ve made some thoughts about implementing a LSP server.
    All the cool stuff like auto-completion, signature help etc… aren’t part of this yet.
    This is more about the general workflow and potential risks.
    If someone has already read the specification or is aware how this is going to work,
    or has general ideas, then I would appreciate any input.

    Just in case it isn’t obvious - this would be the workflow for python code.
    def On… are the callbacks which are triggered by npp, the init function
    is the one which is going to be called as the very first function at all.

    def __init__():
        # read config file
        # create a dict of configured lspservers and their startup parameters
        #     sanity check - check if known lspservers do still exist as file on disk
        #     if not, log error and ignore config entry
        # create a dict of known languages npp/lsp
        #     dict key notepad.getCurrentLang().name should return the lsp equivalent
        # 
        
    def OnBufferActivated(args):
        #   if notepad.getCurrentLang().name not in lspclient.dictOfLSPServers
        #       return - nothing to do
        #   else 
        #       if lspserver is not running
        #           start lspserver
        #           if process is running
        #               send initialize request
        #               wait for response (how long??)
        #               if initialize response received
        #                   send initialized
        #               else
        #                   what to do?? send a second time?? document says NO
        #                   ignore the lspserver for the rest of npp runtime??
        #           else
        #               log error and (ignore the lspserver for the rest of npp runtime??)
        #
        #       if didOpen was not already sent
        #           send didOpen
        #       else
        #           send didChange ?? there seem to be no equivalent of buffer activated
        #                             should we send a didClose each time the active buffer changes??
        #       From the docs:
        #           The document open notification is sent from the client to the server to signal newly opened text documents. 
        #           The document’s truth is now managed by the client and the server must not try to read the document’s truth 
        #           using the document’s Uri. Open in this sense means it is managed by the client. 
        #           It doesn’t necessarily mean that its content is presented in an editor. 
        #           An open notification must not be sent more than once without a corresponding close notification send before. 
        #           This means open and close notification must be balanced and the max open count for a particular textDocument is one. 
        #           Note that a server’s ability to fulfill requests is independent of whether a text document is open or closed.
        #
        #
        #   The overall potential risk here is that it takes some time before a lspserver is completely started
        #   which means the user might start typing and expects interaction which isn't possible at this time yet.
        #   Maybe a startup parameter flag startLSPServer on npp startup could be helpful but this means lspserver
        #   gets started even if it is not needed. 
        #   In addition, I assume that publishDiagnostics notification is send upon didOpen/Change notification
        #   which would be another hint to have this run in its own thread but what if one constantly switches buffers?
        #   I assume queuing and checking if the notification is still valid is needed.
        
    
    def OnFileBeforeSave(args):
        # send willSave notification
        # don't understand the meaning of the willSaveWaitUntil request. 
        # Why should one allow to modify source code before closing the file - does not make sense to me.
        # From the docs
        #   The document will save request is sent from the client to the server before the document is actually saved. 
        #   The request can return an array of TextEdits which will be applied to the text document before it is saved. 
        #   Please note that clients might drop results if computing the text edits took too long or if a server 
        #   constantly fails on this request. This is done to keep the save fast and reliable.
    
    def OnFileClosed(args):
        # send didClose notification
    
    def OnFileOpened(args):
        # normally we would send didOpen notification from here but because this callback is not reliable,
        # you won't see this callback on npp startup, we use OnBufferActivated callback to send the didOpen notification
    
    def OnFileSaved(args):
        # send didSave notification
    
    def OnLangChanged(args):
        # trigger an OnBufferActivated callback should be all we need to do, to be on the save side
    
    def OnReady(args):
        # maybe do some init here instead of atstartup.py - is it reliable??
        
    def OnShutdown(args):
        # send shutdown request and exit notification


  • Ok - I found some time and created a python script based LSP client and it works in its early alpha stage quiet well.
    Now I need some help - as npp is a general purpose editor it supports a wide range of languages and this means
    my nppLspClient would, possibly, have to handle those different LSP servers.
    As I don’t know all of the possible languages :-) I would like to ask for help in identifying, creating and running those servers and
    most important providing some test scripts/programs in order for me to be able to test it.
    Just in case someone wants to point me to https://microsoft.github.io/language-server-protocol/implementors/servers/
    I already know it :-) but, for example, I don’t know how I can create and run a typescript based LSP server.
    Of course I could start investigating this but I would prefer to spend the time to work on the current client
    and someone who knows typescript would possibly have the answer already, so if you can help - please do so :-D

    Thank you.



  • I’m interested in supporting your development by making some experiments with the followings:

    VHDL-Tool

    rust_hdl

    Do you think those could be of help?



  • @pidgeon777

    sorry, totally missed this one - mostly related to the fact that
    I cannot login with my google account anymore but … that’s another story …

    I don’t know how vhdtool can be used as this is, currently,
    a linux only version but the rust_hdl seems interesting.

    Currently there are a couple of manual steps which are needed to be done and in debug mode it is very slow. Would you mind
    to provide a simple sample vhdl script with a short description
    what you expect to see/work and I’ll give it a try because if it
    doesn’t work like the python lsp server than it is easier to debug
    locally instead of communicating over the community.
    Once I can make it work I would provide you with the info how
    to setup PythonScript plugin and the lsp client in order to do more detailed tests on your side.



  • @Ekopalypse
    @dail

    This old discussion almost going on 2 years - any progress? What were the challenges with NppLSP - technical or just time? Is the “python script based LSP client” still in existence - any improvements / testing?

    Cheers.



  • @Michael-Vincent

    I have moved to using Jedi, Flakes … directly through the PythonScript
    plugin instead of through an external process, but the code still exists, yes.
    I haven’t checked/tested it with the latest Npp/PS yet,
    but I will do it and let you know.
    Are you looking for a specific programming language?
    If so, can you give me a sample script and hints how to
    set up the LSP server for this language?
    I have also neglected the further development of the LSP protocol.
    Let’s see what else needs to be added.



  • @Ekopalypse said in Microsoft Language Server Protocol (LSP): A possible revolution for Notepad++?:

    Are you looking for a specific programming language?

    Perl would be my goto as it’s the language I best understand from both a coding and setup / installation perspective. There is Perl::LanguageServer, but without a client in an IDE, I’ve never set it up or tested it.

    I don’t use PythonScript, but could if it would have a Python solution - I do a lot of little Python scripts nowadays and it’s not my favorite - I just don’t know it as well as Perl so I code slower. Auto-complete and “Intellisense-like” info would help so that’s why I was looking this up. I do only Python3 however - is N++ PythonScript still at Python2?

    Cheers.



  • @Michael-Vincent

    There is an alpha version available which is stable.
    I’ve been using PS3 since its introduction, the only caveat is that it only supports utf8-encoded scintilla buffers.
    Since I only use utf8 this is not a problem for me and I have had no problems so far.
    I deleted my perl installation some time ago, can you give me a tip
    which one I should use to have a smooth (not to worry about) installation.



  • @Ekopalypse said in Microsoft Language Server Protocol (LSP): A possible revolution for Notepad++?:

    I’ve been using PS3 since its introduction, the only caveat is that it only supports utf8-encoded scintilla buffers.

    Arg, I use ANSI / Unix file endings as default. I suppose I could try using UTF8 files and see if that introduces any encoding issues with the scripts and environments I’m running. PS3 would be essential though for me.

    I deleted my perl installation some time ago, can you give me a tip
    which one I should use to have a smooth (not to worry about) installation.

    Shame :-) Maybe @PeterJones can weigh in too being a Perl expert, but I prefer Strawberry Perl . I’m not using the latest version - just haven’t upgraded my install in some time, but the latest available should be fine and stable.

    I’m also willing to do some testing myself if you prefer to “give” me your PythonScript (v3) setup with “Jedi, Flakes” and I can try it out on some Python scripts.

    I think ideally, like you mentioned above, Language Server Protocol in Notepad++ will probably best be implemented through a proper plugin and be able to connect to any Language Server (e.g., Python, Perl like we’re talking, but also C/C++ and others). Of course, that’s a pretty heavy lift methinks. Hoping @dail has some update on NppLSP, but based on GitHub activity, I think not ;-(

    Cheers.



  • I am on my way now, I will come back to it some time later today. ~4-5 hours.



  • @Michael-Vincent

    I do only Python3 however - is N++ PythonScript still at Python2?

    I use ANSI / Unix file endings as default. I suppose I could try using UTF8 files and see if that introduces any encoding issues with the scripts and environments I’m running. PS3 would be essential though for me.

    Curious about this.
    First, line-endings don’t matter (in this).
    Second, definitely try UTF8; the sooner the world can forget about “ANSI” the better off things will be. :-)

    Lastly, for “pythonscripting” I don’t see a big difference between Python2 and Python3, syntactically. Sure, if you are doing some “deeper” things, then maybe, but for the vast majority of scripts I’ve done, when I finally move to PS3 (haven’t yet I guess due to laziness) I don’t expect to have to change anything.

    So I guess my point is, if you want to stick with ANSI, it seems reasonable to work with the Python2 version of PS, as such an “investment” isn’t “lost” later.

    @Ekopalypse may have some further thoughts on this; am I off-base in my thinking, here?



  • @Alan-Kilborn said in Microsoft Language Server Protocol (LSP): A possible revolution for Notepad++?:

    Lastly, for “pythonscripting” I don’t see a big difference between Python2 and Python3, syntactically

    Agree, it’s not that hard to port our Py2 to Py3 stuff. What I wasn’t sure about was using PythonScript2 to do language parsing and analysis on Python3 scripts I’ll be writing for work stuff - not to be run as Notepad++ Python scripts. That may not make a difference, but wanted to clarify.

    Cheers.



  • @Michael-Vincent said in Microsoft Language Server Protocol (LSP): A possible revolution for Notepad++?:

    @Ekopalypse
    @dail

    This old discussion almost going on 2 years - any progress? What were the challenges with NppLSP - technical or just time? Is the “python script based LSP client” still in existence - any improvements / testing?

    Cheers.

    I obviously haven’t touched the code in quite a while and don’t have any immediate plans to pick it back up any time soon.

    The struggles I had at the time were severe lack of documentation. Yes there was documentation, but they covered the basics and individual pieces. Also it came down to just having enough time as well to fight through the documentation.

    Specifically what was lacking was the higher level usage-case. What gets sent before other messages, what parameter effects later behavior, when to send these commands, recommended best practices, etc.

    On top of this was the problem of transitioning to a newer specification…so some LSP servers supported one, and not the other. Other LSP servers partially supported one of them with certain features not supported, or had broken implementations.

    My hope is that over the past 2 years the environment has improved.



  • @dail

    Thank you for the feedback!



  • I guess the most critical part of changing from 2 to 3 is how python handles strings and this is the
    reason that PS3 is still in alpha because, as far as I understand, this isn’t currently handled in boost::python.

    @Michael-Vincent
    not sure I understand correctly what you are trying to achieve.
    If you want to get auto completion and such stuff to work with python3 scripts in notepad++,
    you normally have to use PS3 plugin.
    Using PS2 plugin would mean you have to start a python3 process, run this in combination with a
    a jedi.py script which then parses your notepad++ .py file and outputs it findings to stdout which then is
    parsed back to PS2. Doesn’t sound desirable.



  • @Ekopalypse said in Microsoft Language Server Protocol (LSP): A possible revolution for Notepad++?:

    If you want to get auto completion and such stuff to work with python3 scripts in notepad++,
    you normally have to use PS3 plugin.

    Yes, that’s my desire. I’m not interested in writing Notepad++ PythonScript automation, I’m interested in using Notepad++ to write scripts in version 3 of the Python language to be used for my work projects and while writing those scripts, I’d like to have some “Language Server” features (i.e., autocomplete, etc) above and beyond the standard Notepad++ “function name completion” through the use of a Language Server “plugin” (or in this case, PS3 adapted script capability you speak of).

    I hope that makes sense now in exhaustive detail. I realize the confusion around “PythonScript” and “python scripts” :-)

    Cheers.



  • @Michael-Vincent
    ok, this is my “lsp” script (called python_ide.py)

    ''' Basic functionality which one would expect from a full-fledged IDE '''
    import sys
    from Npp import (
        editor, editor1, editor2, notepad,
        SCINTILLANOTIFICATION, NOTIFICATION, LANGTYPE
        )
    
    from pyflakes import api
    from pyflakes import reporter as rep
    import jedi
    jedi.preload_module(['os',])
    
    
    class IDE(rep.Reporter):
    
        def __init__(self):
            rep.Reporter.__init__(self, sys.stdout, sys.stderr)
            self.auto_indent = True
            self.check_types = False
            self.linting = True
            self.reporter = None
            self.is_python = False
            self.STYLE_WARNING = 55
            self.excluded_styles = [1, 3, 4, 6, 7, 12, 16, 17, 18, 19]
            self.prev_completion_list = []
    
    
        def initialize(self):
            editor1.styleSetFont(self.STYLE_WARNING, 'Consolas')
            editor1.styleSetItalic(self.STYLE_WARNING,True)
            editor1.styleSetFore(self.STYLE_WARNING, (200,160,12))
            editor1.styleSetBack(self.STYLE_WARNING, notepad.getEditorDefaultBackgroundColor())
            editor2.styleSetFont(self.STYLE_WARNING, 'Consolas')
            editor2.styleSetItalic(self.STYLE_WARNING,True)
            editor2.styleSetFore(self.STYLE_WARNING, (200,160,12))
            editor2.styleSetBack(self.STYLE_WARNING, notepad.getEditorDefaultBackgroundColor())
            editor.callbackSync(self.on_charadded, [SCINTILLANOTIFICATION.CHARADDED])
            editor.callbackSync(self.on_modified, [SCINTILLANOTIFICATION.MODIFIED])
            notepad.callback(self.on_buffer_activated, [NOTIFICATION.BUFFERACTIVATED])
            notepad.callback(self.on_file_saved, [NOTIFICATION.FILESAVED])
            self.is_python = notepad.getCurrentLang() == LANGTYPE.PYTHON
    
    
        def on_modified(self, args):
            if args['modificationType'] & 0x100000 == 0x100000 and args['text'] in ['\r','\n', '\r\n']:
                self._indent(args['position'], args['text'])
    
    
        def on_buffer_activated(self, args):
            self.is_python = notepad.getCurrentLang() == LANGTYPE.PYTHON
            self.path = notepad.getCurrentFilename()
    
    
        def on_file_saved(self, args):
            if self.is_python:
                editor.annotationClearAll()
                self.lint()
    
    
        def on_charadded(self, args):
            if self.is_python:
                c = chr(args['ch'])
                if c in '\r\n:-+*/#=)':
                    return
    
                pos = editor.getCurrentPos()
                if editor.getStyleAt(pos) in self.excluded_styles:
                    return
    
                source = editor.getText() + c
                line = editor.lineFromPosition(pos) + 1
                column = editor.getColumn(pos)
    
                if c == '(':
                    script = jedi.Script(code=source).get_signatures(line, column)
                    self.show_calltip(script, pos)
                else:
                    if c == '.' and editor.callTipActive():
                        editor.callTipCancel()
                    script = jedi.Script(code=source).complete(line, column)
                    self.autocomplete(script, pos)
    
    
        def unexpectedError(self, filename, msg):
            # HACK: to avoid encoding declaration in Unicode string error
            if msg != 'problem decoding source':
                print(f'File "{filename}", line {0}, unexpected error:{msg}')
    
    
        def syntaxError(self, filename, msg, lineno, offset, text):
            line = text.splitlines()[-1]
            if offset is not None:
                offset -= (len(text) - len(line)) + 1
            else:
                offset=''
    
            print(f'File "{filename}", line {lineno},{offset} Error:{msg}')
            print(f'    => {line}')
            self.show_error(filename,lineno, msg)
    
    
        def flake(self, message):
            msg = message.message % message.message_args
            for line_ignore_msg in ['# ignore_undefined_name', '# ignore_unused_import',]:
                if line_ignore_msg in editor.getLine(message.lineno-1 if message.lineno>0 else message.lineno):
                    return
            self.show_error(message.filename, message.lineno, msg)
    
    
        def show_error(self, filename, line_no, msg):
            line_no = line_no-1 if line_no>0 else line_no
            editor.annotationSetText(line_no, f'{msg}')
            editor.annotationSetVisible(2)
            editor.annotationSetStyle(line_no, self.STYLE_WARNING)
            editor.ensureVisible(line_no)
            editor.gotoLine(line_no)
            editor.verticalCentreCaret()
    
    
        def lint(self):
            if self.linting:
                filename = notepad.getCurrentFilename()
                if filename.endswith('.py'):
                    api.check(editor.getText(), filename, self)
    
    
        def autocomplete(self, completions, pos):
            if completions:
                completion_list = sorted([x.name for x in completions if not x.name.startswith('_')])
                if completion_list:
                    if self.prev_completion_list != completion_list:
                        self.prev_completion_list = completion_list
                        word_start = editor.wordStartPosition(pos, True)
                        already_typed = pos - word_start
                        already_typed = 0 if already_typed < 0 else already_typed
                        editor.autoCShow(already_typed, ' '.join(completion_list))
    
    
        def show_calltip(self, call_tips, pos):
            if call_tips:
                tips = ', '.join([x.name for x in call_tips[0].params])
                tips += '\n'
                if tips.strip():
                    editor.callTipShow(pos, f' {tips[:-1]} ')
    
    
        def _indent(self, position, text):
            if self.is_python and self.auto_indent:
                indent = editor.getLineIndentation(editor.lineFromPosition(position))
                if (editor.getCharAt(position-1) == 58 and  # 58 == :
                    editor.getStyleAt(position-1) not in self.excluded_styles):
                    tabwidth = editor.getTabWidth()
                    text += ' '*(indent//tabwidth+1)*tabwidth
                else:
                    text += ' '*indent
                editor.changeInsertion(text)  
    

    and this is what I have in user startup.py

    from python_ide import IDE
    __ide = IDE()
    __ide.initialize()
    

    You need to pip-install pyflakes and jedi and use the PS3 alpha version.
    A word of warning, although I don’t think it will affect you.
    I had crashes when using this together with Npp objects.
    As long as you only use standard Python scripts/modules, as you mentioned,
    you should not have this problem.



  • @Ekopalypse said in Microsoft Language Server Protocol (LSP): A possible revolution for Notepad++?:

    You need to pip-install pyflakes and jedi and use the PS3 alpha version.

    Sorry for the following probably basic Python questions, but I’m getting this error when running my startup.py:

    Traceback (most recent call last):
      File "C:\usr\bin\npp64\plugins\PythonScript\scripts\startup.py", line 31, in <module>
        from python_ide import IDE
      File "C:\usr\bin\npp64\plugins\PythonScript\scripts\python_ide.py", line 8, in <module>
        from pyflakes import api
    ModuleNotFoundError: No module named 'pyflakes'
    

    I’m assuming this is because PyFlakes is not installed in my PythonScript3 distro. I did a:

    python3 -mpip install pyflakes jedi

    and it worked to install to my OS python:

    PS VinsWorldcom C:\usr\bin\npp64\plugins\PythonScript > python3 -V
    Python 3.6.3
    PS VinsWorldcom C:\usr\bin\npp64\plugins\PythonScript > python3
    Python 3.6.3 (v3.6.3:2c5fed8, Oct  3 2017, 18:11:49) [MSC v.1900 64 bit (AMD64)] on win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import jedi
    >>> import pyflakes
    

    So I’m guessing I need to have PythonScript3 find my local Python3 libraries or install PyFlakes and Jedi in the PythonScript3 lib/ directory.

    Any tips?

    Cheers.



  • @Michael-Vincent

    which PS3 plugin did you install? Depending on it you have to use the correct python version and copy
    the pythonXX.dll to the PS plugin directory.
    Like I have the latest PS, hence I use python 3.8

    0d58770e-7268-4d24-a723-d0f18267a42f-image.png



  • @Michael-Vincent

    Now I’m thinking that, if I remember correctly, you have to use python 3.8. PS3 started with this version.


Log in to reply