Intercept Scintilla keydown events in C#

BlaxOne

For a C# plugin project (using the NppPlugin .NET package template) I need to detect when the user hits the Tab key, then call some method (editing the text in the current Scintilla actually) and cancel (or “consume”) that keydown event.
For a custom dialog or control this is quite simple, with code like that in the dialog class:

const int WM_KEYDOWN = 0x100;
const int WM_SYSKEYDOWN = 0x104;

protected override void WndProc(ref Message m)
{
    if (m.Msg == WM_KEYDOWN || m.Msg == WM_SYSKEYDOWN)
    {
        Keys keyCode = (Keys)m.WParam & Keys.KeyCode;
        if (keyCode == Keys.Tab)
        {
            DoStuff();
            return;
        }
    }
    base.WndProc(ref m);
}

But when the editor is in focus this code is not executed of course. How to process the window messages for the current Scintilla control in C#? Can one override its window procedure? Or maybe you can think of other ways to address my requirements?

Thanks!

Ekopalypse

@BlaxOne

the first thing which came into my mind was, use the modify notification together with the SCI_CHANGEINSERTION method but you can also override the scintilla message queue. Get the window handle and call SetWindowLongPtr to replace it with your proc function.

BlaxOne

Thanks for your hints @Ekopalypse. I considered them with utmost attention. :)

I managed to use SCI_CHANGEINSERTION but apparently checking for the SCN_MODIFIED notification has some inconveniences:
It implies to infer that the Tab key has been pressed, from the text that is to be inserted. I check if this text is made of a tab or 2 or more spaces (letting aside the case where the user has set Tab to be replaced by ONE space in the preferences…), but there are cases where the insertion is not due to the user pressing the tab key: it can be the result of a paste operation, and there I can see no way to know this is the case (modificationType informs about undo/redo but not clipboard operations): when I paste a text made of 2 spaces for example, my code executes whereas it shouldn’t.
And how about checking for a combination of keys, instead of only Tab, like Ctrl+Enter for example, which does not trigger any text modification?
This method seems not reliable nor flexible to me.

So I’m happy to learn it is possible to override the Scintilla message queue in C#. Following your suggestion I tried this (I don’t know if this is a good thing but I set almost everything to static since I need to call SubclassHWnd(PluginBase.GetCurrentScintilla()) from a static method):

[DllImport("user32")]
private static extern IntPtr SetWindowLongPtrW(IntPtr hWnd, int nIndex, Win32WndProc newProc);
[DllImport("user32")]
private static extern int CallWindowProc(IntPtr lpPrevWndFunc, IntPtr hWnd, int Msg, int wParam, int lParam);

// A delegate that matches Win32 WNDPROC:
private delegate int Win32WndProc(IntPtr hWnd, int Msg, int wParam, int lParam);

// from winuser.h:
private const int GWL_WNDPROC = -4;
const int WM_KEYDOWN = 0x100;
const int WM_SYSKEYDOWN = 0x104;

// program variables
static private IntPtr oldWndProc = IntPtr.Zero;
static private Win32WndProc newWndProc = null;

static void SubclassHWnd(IntPtr hWnd)
{
    // delegate for the new wndproc
    newWndProc = new Win32WndProc(MyWndProc);
    // subclass
    oldWndProc = SetWindowLongPtrW(hWnd, GWL_WNDPROC, newWndProc);
}

static private int MyWndProc(IntPtr hWnd, int Msg, int wParam, int lParam)
{
    switch(Msg)
    {
        case WM_KEYDOWN:
        case WM_SYSKEYDOWN:
            Keys keyCode = (Keys)wParam & Keys.KeyCode;
            if (keyCode == Keys.Tab)
            {
                MessageBox.Show("Test");
                //DoStuff();
                return 0;
            }
            break;

        default:
            break;
    }

    return CallWindowProc(oldWndProc, hWnd, Msg, wParam, lParam);
}

But I get an error on the last line:
System.AccessViolationException “Attempted to read or write protected memory. This is often an indication that other memory is corrupt.”
If I enable unmanaged code debugging in VS I get:
“Exception thrown at (…) SciLexer.dll in notepad++.exe. 0xC0000005: Access violation writing location 0x000000001A519AE0.”
I tried with SetWindowLong, SetWindowLongPtr and SetWindowLongPtrW. But surely this is the right track to use that function indeed, there must a be way.
So any clue or working example would be welcome…

Ekopalypse

@BlaxOne

Sorry, don’t have any C# experience but I do the same with python.
By what I understand from your code I’m suspecting your private delegate int Win32WndProc(IntPtr hWnd, int Msg, int wParam, int lParam);

As to msdn it must be LRESULT (I assume IntPtr in C#)

In addition there is either a CallWindowProcA or CallWindowProcW but no CallWindowProc - something you missed?

Are you sure oldWndProc stil exists in this context?

rinku singh

@BlaxOne said:

I managed to use SCI_CHANGEINSERTION but apparently checking for the SCN_MODIFIED notification has some inconveniences:

there SCN_MODIFIED did not works correctly with npp in default mode
link text

BlaxOne

@Ekopalypse
I converted the return types from int to IntPtr as you suggested and changed CallWindowProc to CallWindowProcW but I still get the same error. According to VS, oldWndProc is not null when the last line is to be executed.

Actually I just found there is a dedicated class in C# to subclass a window given its handle. It seems to do the job, while raising other problems but with some luck I should be able to make it through…

Anyway I appreciate your help and I am very grateful to you for that.
Thanks again!

BlaxOne

Skycc Lim

Hi @BlaxOne
have you figure out the solution for your problem ? can you share ?
i had similar problem with yours which i need to filter out certain keystrokes for current scintilla control