PythonScript: Different behavior in script vs in immediate mode

[Alan Kilborn]

@PeterJones said in PythonScript: Different behavior in script vs in immediate mode:

I am trying to implement getStatusBar(sectionNum) using win32 api calls, since NPPM_* doesn’t include NPPM_GETSTATSUBAR despite having NPPM_SETSTATUSBAR, and thus PythonScript implements editor.setStatusBar(sectionNum, text) but not editor.getStatusBar(sectionNum)

So I dug through my code archive, and I found something that achieves the goal. Curiously, some notes about it lead back to that very same Claudia posting you cited.

I gave this old code a run, expecting to see some interesting things like you wrote about. But no, it ran just fine (N++ 8.1.3, 32-bit, PS 1.5.4). I even gave it a spin under a relatively recent N++ (7.9.5) in 64-bit mode with PS1.5 – and it ran nicely there too.

Here’s my version of the code (I’m sure “my” is a loose term, as I’m confident I wrote maybe 10%):

# -*- coding: utf-8 -*-
from __future__ import print_function

from Npp import *
import ctypes
from ctypes.wintypes import BOOL, HWND, WPARAM, LPARAM, UINT

def npp_get_statusbar(statusbar_item_number):

    WNDENUMPROC = ctypes.WINFUNCTYPE(BOOL, HWND, LPARAM)
    FindWindowW = ctypes.windll.user32.FindWindowW
    FindWindowExW = ctypes.windll.user32.FindWindowExW
    SendMessageW = ctypes.windll.user32.SendMessageW
    LRESULT = LPARAM
    SendMessageW.restype = LRESULT
    SendMessageW.argtypes = [ HWND, UINT, WPARAM, LPARAM ]
    EnumChildWindows = ctypes.windll.user32.EnumChildWindows
    GetClassNameW = ctypes.windll.user32.GetClassNameW
    create_unicode_buffer = ctypes.create_unicode_buffer

    SBT_OWNERDRAW = 0x1000
    WM_USER = 0x400; SB_GETTEXTLENGTHW = WM_USER + 12; SB_GETTEXTW = WM_USER + 13

    npp_get_statusbar.STATUSBAR_HANDLE = None

    def get_result_from_statusbar(statusbar_item_number):
        assert statusbar_item_number <= 5
        retcode = SendMessageW(npp_get_statusbar.STATUSBAR_HANDLE, SB_GETTEXTLENGTHW, statusbar_item_number, 0)
        length = retcode & 0xFFFF
        type = (retcode >> 16) & 0xFFFF
        assert (type != SBT_OWNERDRAW)
        text_buffer = create_unicode_buffer(length)
        retcode = SendMessageW(npp_get_statusbar.STATUSBAR_HANDLE, SB_GETTEXTW, statusbar_item_number, ctypes.addressof(text_buffer))
        retval = '{}'.format(text_buffer[:length])
        return retval

    def EnumCallback(hwnd, lparam):
        curr_class = create_unicode_buffer(256)
        GetClassNameW(hwnd, curr_class, 256)
        if curr_class.value.lower() == "msctls_statusbar32":
            npp_get_statusbar.STATUSBAR_HANDLE = hwnd
            return False  # stop the enumeration
        return True  # continue the enumeration

    npp_hwnd = FindWindowW(u"Notepad++", None)
    #print('npph:', npp_hwnd)
    EnumChildWindows(npp_hwnd, WNDENUMPROC(EnumCallback), 0)
    #print('sbh:', npp_get_statusbar.STATUSBAR_HANDLE)
    if npp_get_statusbar.STATUSBAR_HANDLE: return get_result_from_statusbar(statusbar_item_number)
    assert False

print(npp_get_statusbar(0))
print(npp_get_statusbar(1))

and here’s some sample output from the PS Console:

Python file
length : 2,439    lines : 55

If this doesn’t help, I can dig in deeper, but right now, I’m not sure what I’d look at.

Final note:
To be honest, I searched my code archive first for SB_GETTEXTLENGHTHW, but didn’t get any hits. I found out that’s because you spelled it wrong (and I just copied from your post into my Find what) – an extra H that is really hard to see!

[PeterJones]

@Alan-Kilborn said in PythonScript: Different behavior in script vs in immediate mode:

I gave this old code a run, expecting to see some interesting things like you wrote about. But no, it ran just fine (N++ 8.1.3, 32-bit, PS 1.5.4). I even gave it a spin under a relatively recent N++ (7.9.5) in 64-bit mode with PS1.5 – and it ran nicely there too.

Well, you just invalidated my next direction, because I had shown myself that 32b v7.6.3 + v1.5.4 worked, but 64b v7.6.3 + v1.5.4 failed, so I thought it was a bitness issue. But if v7.9.5-64 + PS1.5 works for you, then it’s not (just) bitness.

Your script works for me under v8.1.4-64 and PS1.5.4, so I’ll try to figure out what’s different there. A quick glance tells me that it’s probably the SendMessageW.restype/.argtypes, which you define but I don’t. But I’ll know more when I get another chance to experiment.

Thanks for the help.

SB_GETTEXTLENGHTHW, but didn’t get any hits. I found out that’s because you spelled it wrong (and I just copied from your post into my Find what) – an extra H that is really hard to see!

It was right in my code. :-)

[Ekopalypse]

Just a note: pay attention to how SendMessage is defined.
Since the PythonScript plugin behaves more or less like the Python REPL,
i.e. it remembers what has been used, it is important to make sure that one script
does not overwrites function definitions that have already been defined by another script.

[Alan Kilborn]

@Ekopalypse said in PythonScript: Different behavior in script vs in immediate mode:

Just a note: pay attention to how SendMessage is defined.

Since the PythonScript plugin behaves more or less like the Python REPL,

i.e. it remembers what has been used, it is important to make sure that one script

does not overwrites function definitions that have already been defined by another script.

So…actually I have that kind of stuff in a file that I import from startup.py. So that way I think I am following your advice…every script gets the same definitions.

The downside is that if I want to share a script, I have to copy some definitions from that file into the script to share before publishing. AND I always test it in a “fresh” N++/PS, both 32 and 64 bit.

[Ekopalypse]

@Alan-Kilborn

Yes, that’s how it should be done to avoid misconduct.

[PeterJones]

@Alan-Kilborn ,

adding

from ctypes.wintypes import BOOL, HWND, WPARAM, LPARAM, UINT
LRESULT = LPARAM
...
SendMessage.restype = LRESULT
SendMessage.argtypes = [ HWND, UINT, WPARAM, LPARAM ]

to the script I posted above makes it work with NPPv8.1.4-64 + PSv1.5.4-64, so it was just parameter/return-type issues.

pay attention to how SendMessage is defined.

I had confirmed that there wasn’t a previous definition of SendMessage that was conflicting – and as my prints above showed, the same SendMessage was being called both in the script and in the immediate-mode. That makes me a little surprised that it gave different results in immediate mode… but I’m glad it did; if I had never gotten a good return value, I might have given up without every posting for help.

But yes, my plan was to add encapsulation (similar to what Alan showed in his script), so in the end, name collisions wouldn’t be a problem

[Alan Kilborn]

@PeterJones said in PythonScript: Different behavior in script vs in immediate mode:

I had confirmed that there wasn’t a previous definition of SendMessage that was conflicting

It is interesting.
Without any predefinition, the argtypes for SendMessageW is None.
If it doesn’t work without the end-user defining some argtypes first, it seems, well, less than useful.
Or maybe I’m missing something.
Or maybe ctypes is just too magical and mystical for me.

[PeterJones]

@Alan-Kilborn said in PythonScript: Different behavior in script vs in immediate mode:

Or maybe ctypes is just too magical and mystical for me.

Probably for all of us.

I compared the results of SendMessage.argtypes = [ HWND, UINT, WPARAM, LPARAM ] and then printing SendMessage.argtypes under both 32bit and 64bit.

• 64bit = [<class 'ctypes.c_void_p'>, <class 'ctypes.c_ulong'>, <class 'ctypes.c_ulonglong'>, <class 'ctypes.c_longlong'>]
• 32bit = [<class 'ctypes.c_void_p'>, <class 'ctypes.c_ulong'>, <class 'ctypes.c_ulong'>, <class 'ctypes.c_long'>]

Since the 32bit is long-based all the way across, then it probably has no compatibility problems (which is why it worked for me); whereas 64bit requires longlong-based for the WPARAM and LPARAM, so there was probably a data size issue (without telling it that, maybe Python was only sending 8bytes of data rather than 16bytes, which was then not a valid SendMessage data packet.)

[PeterJones]

@Ekopalypse / @Alan-Kilborn

Do either of you know (or anyone else who reads this thread): for the ctypes.create_unicode_buffer, when you assign the results to a variable, if that variable later gets deleted (or goes out of scope), does the virtual buffer that was created also get freed, or will there be a memory leakage issue?

In Perl, the AllocateVirtualBuffer has a paired FreeVirtualBuffer to make sure that the allocated memory gets freed to prevent leakage.

So is going out of scope or using del sufficient to free that memory in ctypes? Or is there a paired ctypes.delete_unicode_buffer() or some such phrasing that I wasn’t finding?

When I was debugging a related function that was allocating buffers in pythonscript, I was occasionally crashing Notepad++. When I clean things up and encapsulate, maybe that portion will work right; but if not, I wanted to make sure I was using best-practices on the mutable memory allocation.

[Ekopalypse]

I’m not sure if I understand this discussion correctly, but as I understand it, interacting with the C/C++ API using ctypes works like this.
In general, it is better to create function signatures like.

SendMessage = ctypes.WinDLL('user32').SendMessageW
SendMessage.restype = LRESULT
SendMessage.argtypes = [ HWND, UINT, WPARAM, LPARAM ]

instead of calling SendMessage with some parameters and letting ctypes try to figure out what and how to interpret them.
You have to worry about different sized pointers/types for x86 and x64 architectures.
Using e.g. wintypes.LPARAM etc… helps in such cases, but when interacting with non-Windows C APIs, you have to take care of it manually.

The SendMessage API itself is a special case anyway, as it is not always immediately clear what is being done.
The function is … a message handler … that delivers the provided information to the addressed window handle.
Only the receiver knows what to do with this message, e.g. is LPARAM a pointer to something or is it a value.
Which brings me to the point of how big such a message is. I would expect it to be 28 bytes on 64bit and 16 bytes on x86.

hwnd = pointer (8 or 4 bytes)
msg = unsigned int (always 4 bytes)
wparam = unsigned INT_PTR (8 or 4 bytes - NOTE, this is not a pointer but an architecture dependent integer)
lparam = signed INT_PTR (8 or 4 bytes - not a pointer, but an architecture dependent integer)

As for create_unicode_buffer, there is no counterpart that releases the buffer, other than the normal
variable-goes-out-of-scope-run-garbage-collection mechanism.
That is, as long as there is a reference to a variable, the GC (garbage collector) does not release it, which also means that del neither releases objects nor tells the GC to do so, but merely reduces the reference count to that object.
If it is 0 after that, GC kicks in and does its work.
So, in my opinion, the del statement is only needed when objects are created outside the GC controlled area.
I have not yet found such a case with the PythonScript plugin.

[PeterJones]

Status Bar Manipulation Allowing Extra Status Bar Section

Complete example:

• implements a class for StatusBar manipulation
• includes a method which adds an extra StatusBar section
• includes an example callback to force it to a particular string
  (this could be changed so the callback puts some piece of particular
  info into the status bar)
• shows a 10sec example of it being used; note, because Notepad++ resets
  the status bar every time it changes tabs, you can see it flashing
  frequently. After that 10s, the callback will be cleared.

You can take that concept and get rid of the clear-after-10s, which will mean that the status bar will continue to add that section as needed and display your information.

Because it’s fighting Notepad++ for control of the status bar, things like toggling to a different tab or different view (or from one of the panels back to an editor tab) will cause Notepad++ to reset the status bar, then the UPDATEUI will trigger the callback to update the statusbar. This may cause unpleasant flashing of the status bar if you are changing tabs or panels frequently.0

# encoding=utf-8
"""StatusBar Manipulation

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!! USER WARNING: STATUS BAR MAY FLASH DURING UPDATE !!!
!!! FLASH-SENSITIVE USERS SHOULD NOT USE THIS SCRIPT !!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Complete example:
* implements a class for StatusBar manipulation
* includes a method which adds an extra StatusBar section
* includes an example callback to force it to a particular string
    (this could be changed so the callback puts some piece of particular
    info into the status bar)
* shows a 10sec example of it being used; note, because Notepad++ resets
    the status bar every time it changes tabs, you can see it flashing
    frequently
"""

from Npp import *
import ctypes
from ctypes.wintypes import BOOL, HWND, WPARAM, LPARAM, UINT
from struct import pack, unpack
from time import sleep

class _SB(object):
    """refer to these values as _SB.xxxx elsewhere"""
    LRESULT = LPARAM
    WNDENUMPROC = ctypes.WINFUNCTYPE(BOOL, HWND, LPARAM)
    WM_USER = 0x400
    SB_SETPARTS = WM_USER + 4
    SB_GETPARTS = WM_USER + 6
    SB_SETTEXTA = WM_USER + 1
    SB_SETTEXTW = WM_USER + 11
    SB_GETTEXTA = WM_USER + 2
    SB_GETTEXTW = WM_USER + 13
    SB_GETTEXTLENGTHA = WM_USER + 3
    SB_GETTEXTLENGTHW = WM_USER + 12
    SBT_OWNERDRAW = 0x1000
    ctypes.windll.user32.SendMessageW.restype = LRESULT
    ctypes.windll.user32.SendMessageW.argtypes = [ HWND, UINT, WPARAM, LPARAM ]

_SB() # call it to initialize


class NppStatusBar(object):
    """implement a class wrapper around the status bar"""

    def __init__(self):
        self._APP = None
        self._HANDLE = None

        def EnumCallback(hwnd, lparam):
            curr_class = ctypes.create_unicode_buffer(256)
            ctypes.windll.user32.GetClassNameW(hwnd, curr_class, 256)
            if curr_class.value.lower() == "msctls_statusbar32":
                self._HANDLE = hwnd
                #console.write("\t{:32s}0x{:08x}\n".format("sbWnd:", hwnd))
                return False
            return True

        # console.write("\n" + __file__ + "::" + str(self.__class__) + "\n")
        self._APP = ctypes.windll.user32.FindWindowW(u"Notepad++", None)
        # console.write("\t{:32s}0x{:08x}\n".format("Notepad++ hWnd:", self._APP))
        ctypes.windll.user32.EnumChildWindows(self._APP, _SB.WNDENUMPROC(EnumCallback), 0)
        # console.write("\t{:32s}0x{:08x}\n".format("StatusBar hWnd:", self._HANDLE))

        self.__debugStatusBarSections()

    def __debugStatusBarSections(self):
        for sec in [STATUSBARSECTION.DOCTYPE,STATUSBARSECTION.DOCSIZE,STATUSBARSECTION.CURPOS,STATUSBARSECTION.EOFFORMAT,STATUSBARSECTION.UNICODETYPE,STATUSBARSECTION.TYPINGMODE]:
            self.getStatusBarText(sec)

    def getStatusBarText(self, sec):
        section = int(sec)
        retcode = ctypes.windll.user32.SendMessageW(self._HANDLE, _SB.SB_GETTEXTLENGTHW, section, 0)
        length = retcode & 0xFFFF
        sbtype = (retcode>>16) & 0xFFFF
        assert (sbtype != _SB.SBT_OWNERDRAW)
        text_buffer = ctypes.create_unicode_buffer(length)
        retcode = ctypes.windll.user32.SendMessageW(self._HANDLE, _SB.SB_GETTEXTW, section, ctypes.addressof(text_buffer))
        text = '{}'.format(text_buffer[:length])
        del text_buffer
        # console.write("\tSendMessage(0x{:08x}, 0x{:04x}, {:d}, {:d}) => 0x{:04x} 0x{:04x} \"{:s}\"\n".format(self._HANDLE, _SB.SB_GETTEXTLENGTHW, section, 0, sbtype, length, text))
        return text

    def setStatusBarText(self, sec, txt):
        section = int(sec)
        if section <= 5:
            notepad.setStatusBar(STATUSBARSECTION.values[sec], txt)
        else:
            nChars = len(txt)
            text_buffer = ctypes.create_unicode_buffer(nChars)
            text_buffer[:nChars] = txt[:nChars]
            # console.write(repr(text_buffer))
            retcode = ctypes.windll.user32.SendMessageW(self._HANDLE, _SB.SB_SETTEXTW, section, ctypes.addressof(text_buffer))
            del text_buffer
            # console.write("\t...\n")
            # sleep(1)
            # console.write("\t... done\n")

    def getStatusBarNumberOfSections(self):
        nParts = ctypes.windll.user32.SendMessageW(self._HANDLE, _SB.SB_GETPARTS, 0, 0)
        return nParts & 0xFFFF

    def getStatusBarParts(self):
        nParts = ctypes.windll.user32.SendMessageW(self._HANDLE, _SB.SB_GETPARTS, 0, 0)
        # console.write("getStatusBarParts() -> nParts = {}\n".format(nParts))
        nBytes = 4 * nParts
        buf = ctypes.create_string_buffer(nBytes)
        retcode = ctypes.windll.user32.SendMessageW(self._HANDLE, _SB.SB_GETPARTS, nParts, ctypes.addressof(buf))
        #retcode = SendMessage(sb7_StatusBarCallback.STATUSBAR_HANDLE, SB_GETPARTS, nParts, buf)
        # console.write("\tretcode = {}\n".format(retcode))
        ints = unpack('i'*nParts, buf[:nBytes])
        # console.write("\tbuf = {:s} = {:s}\n".format(repr(buf[:nBytes]), ints))
        del buf
        return ints

    def setStatusBarParts(self, *args):
        # console.write("setStatusBarParts({:s})\n".format(args))
        nParts = len(args)
        nBytes = 4 * nParts
        buf = ctypes.create_string_buffer(nBytes)
        buf[:nBytes] = pack('i'*nParts, *args)
        # console.write("\tedit buf = {:s} = {:s}\n".format(repr(buf[:nBytes]), unpack('i'*nParts, buf[:nBytes]) ))
        retcode = ctypes.windll.user32.SendMessageW(self._HANDLE, _SB.SB_SETPARTS, nParts, ctypes.addressof(buf))
        # console.write("\tretcode = {}\n".format(retcode))

    def addStatusBarSection(self, text, width):
        # console.write("addStatusBarSection({:s})\n".format(text))
        oldParts = self.getStatusBarParts()
        nParts = len(oldParts)
        subtract = int(width / nParts)
        scaled = map(lambda x: x-subtract, oldParts)
        scaled.append( oldParts[-1] )
        self.setStatusBarParts( *scaled )
        self.setStatusBarText( nParts, text )

    def example_setSeventhSection(self, txt, width):
        """
        If there are 7 sections, update text;
        if less, add a section and
        """
        n = self.getStatusBarNumberOfSections()
        if n==7:
            self.setStatusBarText(n-1, txt)
        else:
            self.addStatusBarSection(txt, width)

if __name__ == '__main__':
    def example_callback(args):
        sb.example_setSeventhSection("Example", 360)

    console.show()
    sb = NppStatusBar()
    editor.callback(example_callback, [SCINTILLANOTIFICATION.UPDATEUI])
    example_callback(None) # call it once to update the UI manually
    console.write("For the next 10s, should say Example... \n")
    console.write("... even if you UpdateUI (change tabs, etc)\n")
    sleep(10)
    editor.clearCallbacks(example_callback)
    console.write("... DONE.  The next UpdateUI will clear it.\n")

–
edit: added if __name__ == '__main__': and indentation, to make it importable