Community
    • Login

    PythonScript: Different behavior in script vs in immediate mode

    Scheduled Pinned Locked Moved Notepad++ & Plugin Development
    14 Posts 3 Posters 1.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • PeterJonesP
      PeterJones @PeterJones
      last edited by

      NPP v7.3.3-32bit and PS 1.5.4-32bit work on both the scripts, so it’s some change in Notepad++ itself that is causing that SendMessage to behave differently. I don’t know what change in Notepad++ would affect whether or not it receives seemingly-identical SendMessage signals, but …

      1 Reply Last reply Reply Quote 0
      • Alan KilbornA
        Alan Kilborn @PeterJones
        last edited by Alan Kilborn

        @PeterJones said in PythonScript: Different behavior in script vs in immediate mode:

        I am trying to implement getStatusBar(sectionNum) using win32 api calls, since NPPM_* doesn’t include NPPM_GETSTATSUBAR despite having NPPM_SETSTATUSBAR, and thus PythonScript implements editor.setStatusBar(sectionNum, text) but not editor.getStatusBar(sectionNum)

        So I dug through my code archive, and I found something that achieves the goal. Curiously, some notes about it lead back to that very same Claudia posting you cited.

        I gave this old code a run, expecting to see some interesting things like you wrote about. But no, it ran just fine (N++ 8.1.3, 32-bit, PS 1.5.4). I even gave it a spin under a relatively recent N++ (7.9.5) in 64-bit mode with PS1.5 – and it ran nicely there too.

        Here’s my version of the code (I’m sure “my” is a loose term, as I’m confident I wrote maybe 10%):

        # -*- coding: utf-8 -*-
        from __future__ import print_function
        
        from Npp import *
        import ctypes
        from ctypes.wintypes import BOOL, HWND, WPARAM, LPARAM, UINT
        
        def npp_get_statusbar(statusbar_item_number):
        
            WNDENUMPROC = ctypes.WINFUNCTYPE(BOOL, HWND, LPARAM)
            FindWindowW = ctypes.windll.user32.FindWindowW
            FindWindowExW = ctypes.windll.user32.FindWindowExW
            SendMessageW = ctypes.windll.user32.SendMessageW
            LRESULT = LPARAM
            SendMessageW.restype = LRESULT
            SendMessageW.argtypes = [ HWND, UINT, WPARAM, LPARAM ]
            EnumChildWindows = ctypes.windll.user32.EnumChildWindows
            GetClassNameW = ctypes.windll.user32.GetClassNameW
            create_unicode_buffer = ctypes.create_unicode_buffer
        
            SBT_OWNERDRAW = 0x1000
            WM_USER = 0x400; SB_GETTEXTLENGTHW = WM_USER + 12; SB_GETTEXTW = WM_USER + 13
        
            npp_get_statusbar.STATUSBAR_HANDLE = None
        
            def get_result_from_statusbar(statusbar_item_number):
                assert statusbar_item_number <= 5
                retcode = SendMessageW(npp_get_statusbar.STATUSBAR_HANDLE, SB_GETTEXTLENGTHW, statusbar_item_number, 0)
                length = retcode & 0xFFFF
                type = (retcode >> 16) & 0xFFFF
                assert (type != SBT_OWNERDRAW)
                text_buffer = create_unicode_buffer(length)
                retcode = SendMessageW(npp_get_statusbar.STATUSBAR_HANDLE, SB_GETTEXTW, statusbar_item_number, ctypes.addressof(text_buffer))
                retval = '{}'.format(text_buffer[:length])
                return retval
        
            def EnumCallback(hwnd, lparam):
                curr_class = create_unicode_buffer(256)
                GetClassNameW(hwnd, curr_class, 256)
                if curr_class.value.lower() == "msctls_statusbar32":
                    npp_get_statusbar.STATUSBAR_HANDLE = hwnd
                    return False  # stop the enumeration
                return True  # continue the enumeration
        
            npp_hwnd = FindWindowW(u"Notepad++", None)
            #print('npph:', npp_hwnd)
            EnumChildWindows(npp_hwnd, WNDENUMPROC(EnumCallback), 0)
            #print('sbh:', npp_get_statusbar.STATUSBAR_HANDLE)
            if npp_get_statusbar.STATUSBAR_HANDLE: return get_result_from_statusbar(statusbar_item_number)
            assert False
        
        print(npp_get_statusbar(0))
        print(npp_get_statusbar(1))
        

        and here’s some sample output from the PS Console:

        Python file
        length : 2,439    lines : 55
        

        If this doesn’t help, I can dig in deeper, but right now, I’m not sure what I’d look at.

        Final note:
        To be honest, I searched my code archive first for SB_GETTEXTLENGHTHW, but didn’t get any hits. I found out that’s because you spelled it wrong (and I just copied from your post into my Find what) – an extra H that is really hard to see!

        PeterJonesP 2 Replies Last reply Reply Quote 1
        • PeterJonesP
          PeterJones @Alan Kilborn
          last edited by PeterJones

          @Alan-Kilborn said in PythonScript: Different behavior in script vs in immediate mode:

          I gave this old code a run, expecting to see some interesting things like you wrote about. But no, it ran just fine (N++ 8.1.3, 32-bit, PS 1.5.4). I even gave it a spin under a relatively recent N++ (7.9.5) in 64-bit mode with PS1.5 – and it ran nicely there too.

          Well, you just invalidated my next direction, because I had shown myself that 32b v7.6.3 + v1.5.4 worked, but 64b v7.6.3 + v1.5.4 failed, so I thought it was a bitness issue. But if v7.9.5-64 + PS1.5 works for you, then it’s not (just) bitness.

          Your script works for me under v8.1.4-64 and PS1.5.4, so I’ll try to figure out what’s different there. A quick glance tells me that it’s probably the SendMessageW.restype/.argtypes, which you define but I don’t. But I’ll know more when I get another chance to experiment.

          Thanks for the help.

          SB_GETTEXTLENGHTHW, but didn’t get any hits. I found out that’s because you spelled it wrong (and I just copied from your post into my Find what) – an extra H that is really hard to see!

          It was right in my code. :-)

          1 Reply Last reply Reply Quote 1
          • EkopalypseE
            Ekopalypse
            last edited by

            Just a note: pay attention to how SendMessage is defined.
            Since the PythonScript plugin behaves more or less like the Python REPL,
            i.e. it remembers what has been used, it is important to make sure that one script
            does not overwrites function definitions that have already been defined by another script.

            Alan KilbornA 1 Reply Last reply Reply Quote 3
            • Alan KilbornA
              Alan Kilborn @Ekopalypse
              last edited by

              @Ekopalypse said in PythonScript: Different behavior in script vs in immediate mode:

              Just a note: pay attention to how SendMessage is defined.

              Since the PythonScript plugin behaves more or less like the Python REPL,

              i.e. it remembers what has been used, it is important to make sure that one script

              does not overwrites function definitions that have already been defined by another script.

              So…actually I have that kind of stuff in a file that I import from startup.py. So that way I think I am following your advice…every script gets the same definitions.

              The downside is that if I want to share a script, I have to copy some definitions from that file into the script to share before publishing. AND I always test it in a “fresh” N++/PS, both 32 and 64 bit.

              EkopalypseE 1 Reply Last reply Reply Quote 3
              • EkopalypseE
                Ekopalypse @Alan Kilborn
                last edited by

                @Alan-Kilborn

                Yes, that’s how it should be done to avoid misconduct.

                PeterJonesP 1 Reply Last reply Reply Quote 0
                • PeterJonesP
                  PeterJones @Alan Kilborn
                  last edited by

                  @Alan-Kilborn ,

                  adding

                  from ctypes.wintypes import BOOL, HWND, WPARAM, LPARAM, UINT
                  LRESULT = LPARAM
                  ...
                  SendMessage.restype = LRESULT
                  SendMessage.argtypes = [ HWND, UINT, WPARAM, LPARAM ]
                  

                  to the script I posted above makes it work with NPPv8.1.4-64 + PSv1.5.4-64, so it was just parameter/return-type issues.

                  pay attention to how SendMessage is defined.

                  I had confirmed that there wasn’t a previous definition of SendMessage that was conflicting – and as my prints above showed, the same SendMessage was being called both in the script and in the immediate-mode. That makes me a little surprised that it gave different results in immediate mode… but I’m glad it did; if I had never gotten a good return value, I might have given up without every posting for help.

                  But yes, my plan was to add encapsulation (similar to what Alan showed in his script), so in the end, name collisions wouldn’t be a problem

                  Alan KilbornA 1 Reply Last reply Reply Quote 0
                  • Alan KilbornA
                    Alan Kilborn @PeterJones
                    last edited by Alan Kilborn

                    @PeterJones said in PythonScript: Different behavior in script vs in immediate mode:

                    I had confirmed that there wasn’t a previous definition of SendMessage that was conflicting

                    It is interesting.
                    Without any predefinition, the argtypes for SendMessageW is None.
                    If it doesn’t work without the end-user defining some argtypes first, it seems, well, less than useful.
                    Or maybe I’m missing something.
                    Or maybe ctypes is just too magical and mystical for me.

                    PeterJonesP 1 Reply Last reply Reply Quote 1
                    • PeterJonesP
                      PeterJones @Alan Kilborn
                      last edited by

                      @Alan-Kilborn said in PythonScript: Different behavior in script vs in immediate mode:

                      Or maybe ctypes is just too magical and mystical for me.

                      Probably for all of us.

                      I compared the results of SendMessage.argtypes = [ HWND, UINT, WPARAM, LPARAM ] and then printing SendMessage.argtypes under both 32bit and 64bit.

                      • 64bit = [<class 'ctypes.c_void_p'>, <class 'ctypes.c_ulong'>, <class 'ctypes.c_ulonglong'>, <class 'ctypes.c_longlong'>]
                      • 32bit = [<class 'ctypes.c_void_p'>, <class 'ctypes.c_ulong'>, <class 'ctypes.c_ulong'>, <class 'ctypes.c_long'>]

                      Since the 32bit is long-based all the way across, then it probably has no compatibility problems (which is why it worked for me); whereas 64bit requires longlong-based for the WPARAM and LPARAM, so there was probably a data size issue (without telling it that, maybe Python was only sending 8bytes of data rather than 16bytes, which was then not a valid SendMessage data packet.)

                      1 Reply Last reply Reply Quote 0
                      • PeterJonesP
                        PeterJones @Ekopalypse
                        last edited by

                        @Ekopalypse / @Alan-Kilborn

                        Do either of you know (or anyone else who reads this thread): for the ctypes.create_unicode_buffer, when you assign the results to a variable, if that variable later gets deleted (or goes out of scope), does the virtual buffer that was created also get freed, or will there be a memory leakage issue?

                        In Perl, the AllocateVirtualBuffer has a paired FreeVirtualBuffer to make sure that the allocated memory gets freed to prevent leakage.

                        So is going out of scope or using del sufficient to free that memory in ctypes? Or is there a paired ctypes.delete_unicode_buffer() or some such phrasing that I wasn’t finding?

                        When I was debugging a related function that was allocating buffers in pythonscript, I was occasionally crashing Notepad++. When I clean things up and encapsulate, maybe that portion will work right; but if not, I wanted to make sure I was using best-practices on the mutable memory allocation.

                        1 Reply Last reply Reply Quote 1
                        • EkopalypseE
                          Ekopalypse
                          last edited by

                          I’m not sure if I understand this discussion correctly, but as I understand it, interacting with the C/C++ API using ctypes works like this.
                          In general, it is better to create function signatures like.

                          SendMessage = ctypes.WinDLL('user32').SendMessageW
                          SendMessage.restype = LRESULT
                          SendMessage.argtypes = [ HWND, UINT, WPARAM, LPARAM ]
                          

                          instead of calling SendMessage with some parameters and letting ctypes try to figure out what and how to interpret them.
                          You have to worry about different sized pointers/types for x86 and x64 architectures.
                          Using e.g. wintypes.LPARAM etc… helps in such cases, but when interacting with non-Windows C APIs, you have to take care of it manually.

                          The SendMessage API itself is a special case anyway, as it is not always immediately clear what is being done.
                          The function is … a message handler … that delivers the provided information to the addressed window handle.
                          Only the receiver knows what to do with this message, e.g. is LPARAM a pointer to something or is it a value.
                          Which brings me to the point of how big such a message is. I would expect it to be 28 bytes on 64bit and 16 bytes on x86.

                          hwnd = pointer (8 or 4 bytes)
                          msg = unsigned int (always 4 bytes)
                          wparam = unsigned INT_PTR (8 or 4 bytes - NOTE, this is not a pointer but an architecture dependent integer)
                          lparam = signed INT_PTR (8 or 4 bytes - not a pointer, but an architecture dependent integer)
                          

                          As for create_unicode_buffer, there is no counterpart that releases the buffer, other than the normal
                          variable-goes-out-of-scope-run-garbage-collection mechanism.
                          That is, as long as there is a reference to a variable, the GC (garbage collector) does not release it, which also means that del neither releases objects nor tells the GC to do so, but merely reduces the reference count to that object.
                          If it is 0 after that, GC kicks in and does its work.
                          So, in my opinion, the del statement is only needed when objects are created outside the GC controlled area.
                          I have not yet found such a case with the PythonScript plugin.

                          PeterJonesP 1 Reply Last reply Reply Quote 2
                          • PeterJonesP
                            PeterJones @Ekopalypse
                            last edited by PeterJones

                            Status Bar Manipulation Allowing Extra Status Bar Section

                            Complete example:

                            • implements a class for StatusBar manipulation
                            • includes a method which adds an extra StatusBar section
                            • includes an example callback to force it to a particular string
                              (this could be changed so the callback puts some piece of particular
                              info into the status bar)
                            • shows a 10sec example of it being used; note, because Notepad++ resets
                              the status bar every time it changes tabs, you can see it flashing
                              frequently. After that 10s, the callback will be cleared.

                            You can take that concept and get rid of the clear-after-10s, which will mean that the status bar will continue to add that section as needed and display your information.

                            Because it’s fighting Notepad++ for control of the status bar, things like toggling to a different tab or different view (or from one of the panels back to an editor tab) will cause Notepad++ to reset the status bar, then the UPDATEUI will trigger the callback to update the statusbar. This may cause unpleasant flashing of the status bar if you are changing tabs or panels frequently.0

                            # encoding=utf-8
                            """StatusBar Manipulation
                            
                            !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
                            !!! USER WARNING: STATUS BAR MAY FLASH DURING UPDATE !!!
                            !!! FLASH-SENSITIVE USERS SHOULD NOT USE THIS SCRIPT !!!
                            !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
                            
                            Complete example:
                            * implements a class for StatusBar manipulation
                            * includes a method which adds an extra StatusBar section
                            * includes an example callback to force it to a particular string
                                (this could be changed so the callback puts some piece of particular
                                info into the status bar)
                            * shows a 10sec example of it being used; note, because Notepad++ resets
                                the status bar every time it changes tabs, you can see it flashing
                                frequently
                            """
                            
                            from Npp import *
                            import ctypes
                            from ctypes.wintypes import BOOL, HWND, WPARAM, LPARAM, UINT
                            from struct import pack, unpack
                            from time import sleep
                            
                            class _SB(object):
                                """refer to these values as _SB.xxxx elsewhere"""
                                LRESULT = LPARAM
                                WNDENUMPROC = ctypes.WINFUNCTYPE(BOOL, HWND, LPARAM)
                                WM_USER = 0x400
                                SB_SETPARTS = WM_USER + 4
                                SB_GETPARTS = WM_USER + 6
                                SB_SETTEXTA = WM_USER + 1
                                SB_SETTEXTW = WM_USER + 11
                                SB_GETTEXTA = WM_USER + 2
                                SB_GETTEXTW = WM_USER + 13
                                SB_GETTEXTLENGTHA = WM_USER + 3
                                SB_GETTEXTLENGTHW = WM_USER + 12
                                SBT_OWNERDRAW = 0x1000
                                ctypes.windll.user32.SendMessageW.restype = LRESULT
                                ctypes.windll.user32.SendMessageW.argtypes = [ HWND, UINT, WPARAM, LPARAM ]
                            
                            _SB() # call it to initialize
                            
                            
                            class NppStatusBar(object):
                                """implement a class wrapper around the status bar"""
                            
                                def __init__(self):
                                    self._APP = None
                                    self._HANDLE = None
                            
                                    def EnumCallback(hwnd, lparam):
                                        curr_class = ctypes.create_unicode_buffer(256)
                                        ctypes.windll.user32.GetClassNameW(hwnd, curr_class, 256)
                                        if curr_class.value.lower() == "msctls_statusbar32":
                                            self._HANDLE = hwnd
                                            #console.write("\t{:32s}0x{:08x}\n".format("sbWnd:", hwnd))
                                            return False
                                        return True
                            
                                    # console.write("\n" + __file__ + "::" + str(self.__class__) + "\n")
                                    self._APP = ctypes.windll.user32.FindWindowW(u"Notepad++", None)
                                    # console.write("\t{:32s}0x{:08x}\n".format("Notepad++ hWnd:", self._APP))
                                    ctypes.windll.user32.EnumChildWindows(self._APP, _SB.WNDENUMPROC(EnumCallback), 0)
                                    # console.write("\t{:32s}0x{:08x}\n".format("StatusBar hWnd:", self._HANDLE))
                            
                                    self.__debugStatusBarSections()
                            
                                def __debugStatusBarSections(self):
                                    for sec in [STATUSBARSECTION.DOCTYPE,STATUSBARSECTION.DOCSIZE,STATUSBARSECTION.CURPOS,STATUSBARSECTION.EOFFORMAT,STATUSBARSECTION.UNICODETYPE,STATUSBARSECTION.TYPINGMODE]:
                                        self.getStatusBarText(sec)
                            
                                def getStatusBarText(self, sec):
                                    section = int(sec)
                                    retcode = ctypes.windll.user32.SendMessageW(self._HANDLE, _SB.SB_GETTEXTLENGTHW, section, 0)
                                    length = retcode & 0xFFFF
                                    sbtype = (retcode>>16) & 0xFFFF
                                    assert (sbtype != _SB.SBT_OWNERDRAW)
                                    text_buffer = ctypes.create_unicode_buffer(length)
                                    retcode = ctypes.windll.user32.SendMessageW(self._HANDLE, _SB.SB_GETTEXTW, section, ctypes.addressof(text_buffer))
                                    text = '{}'.format(text_buffer[:length])
                                    del text_buffer
                                    # console.write("\tSendMessage(0x{:08x}, 0x{:04x}, {:d}, {:d}) => 0x{:04x} 0x{:04x} \"{:s}\"\n".format(self._HANDLE, _SB.SB_GETTEXTLENGTHW, section, 0, sbtype, length, text))
                                    return text
                            
                                def setStatusBarText(self, sec, txt):
                                    section = int(sec)
                                    if section <= 5:
                                        notepad.setStatusBar(STATUSBARSECTION.values[sec], txt)
                                    else:
                                        nChars = len(txt)
                                        text_buffer = ctypes.create_unicode_buffer(nChars)
                                        text_buffer[:nChars] = txt[:nChars]
                                        # console.write(repr(text_buffer))
                                        retcode = ctypes.windll.user32.SendMessageW(self._HANDLE, _SB.SB_SETTEXTW, section, ctypes.addressof(text_buffer))
                                        del text_buffer
                                        # console.write("\t...\n")
                                        # sleep(1)
                                        # console.write("\t... done\n")
                            
                                def getStatusBarNumberOfSections(self):
                                    nParts = ctypes.windll.user32.SendMessageW(self._HANDLE, _SB.SB_GETPARTS, 0, 0)
                                    return nParts & 0xFFFF
                            
                                def getStatusBarParts(self):
                                    nParts = ctypes.windll.user32.SendMessageW(self._HANDLE, _SB.SB_GETPARTS, 0, 0)
                                    # console.write("getStatusBarParts() -> nParts = {}\n".format(nParts))
                                    nBytes = 4 * nParts
                                    buf = ctypes.create_string_buffer(nBytes)
                                    retcode = ctypes.windll.user32.SendMessageW(self._HANDLE, _SB.SB_GETPARTS, nParts, ctypes.addressof(buf))
                                    #retcode = SendMessage(sb7_StatusBarCallback.STATUSBAR_HANDLE, SB_GETPARTS, nParts, buf)
                                    # console.write("\tretcode = {}\n".format(retcode))
                                    ints = unpack('i'*nParts, buf[:nBytes])
                                    # console.write("\tbuf = {:s} = {:s}\n".format(repr(buf[:nBytes]), ints))
                                    del buf
                                    return ints
                            
                                def setStatusBarParts(self, *args):
                                    # console.write("setStatusBarParts({:s})\n".format(args))
                                    nParts = len(args)
                                    nBytes = 4 * nParts
                                    buf = ctypes.create_string_buffer(nBytes)
                                    buf[:nBytes] = pack('i'*nParts, *args)
                                    # console.write("\tedit buf = {:s} = {:s}\n".format(repr(buf[:nBytes]), unpack('i'*nParts, buf[:nBytes]) ))
                                    retcode = ctypes.windll.user32.SendMessageW(self._HANDLE, _SB.SB_SETPARTS, nParts, ctypes.addressof(buf))
                                    # console.write("\tretcode = {}\n".format(retcode))
                            
                                def addStatusBarSection(self, text, width):
                                    # console.write("addStatusBarSection({:s})\n".format(text))
                                    oldParts = self.getStatusBarParts()
                                    nParts = len(oldParts)
                                    subtract = int(width / nParts)
                                    scaled = map(lambda x: x-subtract, oldParts)
                                    scaled.append( oldParts[-1] )
                                    self.setStatusBarParts( *scaled )
                                    self.setStatusBarText( nParts, text )
                            
                                def example_setSeventhSection(self, txt, width):
                                    """
                                    If there are 7 sections, update text;
                                    if less, add a section and
                                    """
                                    n = self.getStatusBarNumberOfSections()
                                    if n==7:
                                        self.setStatusBarText(n-1, txt)
                                    else:
                                        self.addStatusBarSection(txt, width)
                            
                            if __name__ == '__main__':
                                def example_callback(args):
                                    sb.example_setSeventhSection("Example", 360)
                            
                                console.show()
                                sb = NppStatusBar()
                                editor.callback(example_callback, [SCINTILLANOTIFICATION.UPDATEUI])
                                example_callback(None) # call it once to update the UI manually
                                console.write("For the next 10s, should say Example... \n")
                                console.write("... even if you UpdateUI (change tabs, etc)\n")
                                sleep(10)
                                editor.clearCallbacks(example_callback)
                                console.write("... DONE.  The next UpdateUI will clear it.\n")
                            

                            –
                            edit: added if __name__ == '__main__': and indentation, to make it importable

                            1 Reply Last reply Reply Quote 1
                            • guy038G guy038 referenced this topic on
                            • guy038G guy038 referenced this topic on
                            • guy038G guy038 referenced this topic on
                            • guy038G guy038 referenced this topic on
                            • guy038G guy038 referenced this topic on
                            • guy038G guy038 referenced this topic on
                            • guy038G guy038 referenced this topic on
                            • guy038G guy038 referenced this topic on
                            • guy038G guy038 referenced this topic on
                            • First post
                              Last post
                            The Community of users of the Notepad++ text editor.
                            Powered by NodeBB | Contributors