RFC / Beta Testers Wanted: pyscFilteredViewer

[PeterJones]

pyscFilteredViewer

I have made my first version of a PythonScript-based helper that takes your active file, runs it through an external filter to produce HTML, and launches the filtered HTML in your default web browser.

I was inspired by the PreviewHTML plugin – I used to use that plugin (with it’s awesome filter ability) to render Markdown files, as well as extracting and rendering the POD (embedded documentation format) in Perl scripts. Since the PreviewHTML developer doesn’t seem to be in a hurry in supporting 64bit NPP, I finally decided to exercise my Python chops by doing a simplified version in PythonScript. (Yes, I do see the irony that my major use case was for Perl, and I solved it by writing a Python script. If I had the embedding and plugin-writing know-how, I would write a Perl-based npp-scripting plugin… but baby steps.)

The helper I’ve come up with does what I think it should, and works similarly enough to the PreviewHTML that I can use it for my Markdown and POD needs. But I don’t know if it’s useful to other people, or whether it’s usable by other people. Hence, I’m putting it out for beta-testing.

Let me know if you have any issues or suggestions; opinions and discussion can be here for now. I think I’ve made enough documentation to get you started – but if not, let me know (especially if I can make anything more clear).

Thank you.

[Eko palypse]

@PeterJones

am I right to assume that Command should be set to the browser in question?
Btw. do you want to prefer to have further discussions here or at github?

[PeterJones]

Discussions going here is fine with me.

The “Command” is the external command that filters from the source to the HTML. I will try to make that more clear in the docs.

[Eko palypse]

@PeterJones

sorry, not being a html programmer nor did or do I use the PreviewHTML plugin.
What is the purpose of the command? What does it mean to filter html?
Is this some kind of preprocessing??

[PeterJones]

So going back a step, PreviewHTML is/was a plugin that primarily took the HTML you were editing in one of NPP’s editing panes, and rendered it (using the IE rendering engine) in a new pane inside NPP. I didn’t much use it for that, because I’m not often working directly with HTML.

However, PreviewHTML also had a filter mode, where it would take your active file (say, a Perl script with embedded POD documentation, or a Markdown text document), it would check the file type (“language”) and/or the file’s extension, compare it to the list in Filters.ini, and if it found a match, run whatever external filter program you had configured for that type of file. This external filter would need to take the name of a saved file of type X as a command line argument, and print an HTML file to the STDOUT. PreviewHTML would take the STDOUT from the filter, and render that HTML in its window.

In my pyscFilteredViewer, I decided against trying to completely reinvent the plugin (especially embedding the render engine and displaying the rendered HTML inside Notepad++), and just focused on what I cared most about (and what I was pretty sure I could figure out how to do in PythonScript): filtering my various input files to convert them from type X to HTML – ie, determining the file type, picking the right filter Command from the .ini file, passing the filename to the filter command, grabbing the resulting HTML out of the filter command --; I then decided to let my default web browser handle rendering the HTML for me, rather than writing my own renderer and trying to add a window to the Notepad++ environment.

[Eko palypse]

I guess I see now - basically what you do is to convert certain text from format X into HTML
and show it in the default browser and the responsible part of doing the conversion is what
is configured as command which would lead to the first suggestion, what about allowing
to have another python script doing that conversion?
In addition, it seems that you created some kind of sphinx.
Well, more or less,

First question about your code.

d = dirname(__file__)
if not d in sys.path:
    sys.path.append(dirname(__file__))

Why not using d to append to sys.path instead of calling dirname again?

[PeterJones]

I don’t know sphinx, but from a brief look, I think my system is more a wrapper inside NPP/PythonScript to call sphinx or similar converters.

Re: your suggestion: added to issues list. I hadn’t been planning on it, because the PreviewHTML methodology was to use existing external converters/filters. But it’s not outside the realm of possibility.

Re: code question: oversight on my part. Based on that coding, that was probably my goal, but I got distracted and didn’t notice I hadn’t finished converting to d. :-)

[Eko palypse]

I’m curious, did you ever encounter a npp crash while using your script?
I’m asking because I see you are using notepad.getLanguageDesc(notepad.getCurrentLang())
and in the past it happened to me that npp crashed randomly.
After some trial and error I found that this was the culprit.

[PeterJones]

@Eko-palypse ,

I think I had seen it crash once, but because it hadn’t come back, I hadn’t investigated it further.

In your experiments, what did you end up doing instead of that construct? Did you determine whether it was the getLanguageDesc() or the getCurrentLang() that caused it, or was it just caused by nesting the two without an intermediate variable (ie, would l=notepad.getCurrentLang(); notepad.getLanguageDesc(l) fix it?)

[Eko palypse]

To be honest, I haven’t done deeper research but I ASSUME that the issue is
that there is no intermediate variable and python already deleted the temporarily
created variable before getLanguageDesc got executed.
I ended up using editor.getLexerLanguage, which was ok for my use cases.
I will try to see if I can setup some tests and will report back.

[PeterJones]

I’ve just run the following a few times:

from Npp import *

console.show()
console.clear()
console.write(__file__ + "::" + __name__ + "\n")
for i in range(10000):
    x = notepad.getLanguageDesc(notepad.getCurrentLang())
    console.write( '#{:-4} => {}\n'.format( i, x ) )

and it hasn’t crashed yet, even when I do things like switch files/tabs, create a new file, and go from NPP to other apps and back while the 10000 loops are running…

I think it might be a combo of that call with something else, if there is an instability.

[Eko palypse]

You are right, this was my first also (using the console), within the console I can’t see that problem
but I can say that an intermediate variable does not solve the problem as I got a crash while
using one. Currently I’m looking into callbacks. Will take some time - I’m coming back later.

[Eko palypse]

I guess I found the culprit.
If uncommenting this # y = notepad.getLanguageDesc(x) then npp crashes.
I assume it is related to what has been reported as an issue as well.

import random
from Npp import editor, notepad, NOTIFICATION, MENUCOMMAND

def on_lang_changed(args):
    x = notepad.getCurrentLang()
    # y = notepad.getLanguageDesc(x)
    y = editor.getLexerLanguage()
    print y

notepad.clearCallbacks([NOTIFICATION.LANGCHANGED])
notepad.callback(on_lang_changed, [NOTIFICATION.LANGCHANGED])

LANGUAGES = [eval('MENUCOMMAND.{}'.format(x)) for x in dir(MENUCOMMAND) if x.startswith('LANG_') and not x.startswith('LANG_USER')]
COUNTS = len(LANGUAGES)
for i in range(1,1001):
    if i % 2 == 0:
        notepad.menuCommand(MENUCOMMAND.LANG_PYTHON)
    else:
        notepad.menuCommand(LANGUAGES[random.randint(0,COUNTS-1)])

[Eko palypse]

it is the function itself as this simplified version crashes npp as well

from Npp import notepad, LANGTYPE
LANGUAGES = LANGTYPE.values.values()
for language in LANGUAGES:
    x = notepad.getLanguageDesc(language)

[Eko palypse]

Final conclusion - it is a python script plugin bug because calling
via SendMessage does not crash (tried it 1000x)

import ctypes
from Npp import notepad, LANGTYPE

LANGUAGES = LANGTYPE.values.values()
SendMessage = ctypes.windll.user32.SendMessageW

# #define NPPM_GETLANGUAGEDESC  (NPPMSG + 84)
NPPM_GETLANGUAGEDESC = 2024+84
# // INT NPPM_GETLANGUAGEDESC(int langType, TCHAR *langDesc)
# // Get programming language short description from the given language type (LangType)
# // Return value is the number of copied character / number of character to copy (\0 is not included)
# // You should call this function 2 times - the first time you pass langDesc as NULL to get the number of characters to copy.
# // You allocate a buffer of the length of (the number of characters + 1) then call NPPM_GETLANGUAGEDESC function the 2nd time
# // by passing allocated buffer as argument langDesc
npp_hwnd = ctypes.windll.user32.FindWindowW(u'Notepad++', None)
for language in LANGUAGES:
    length = ctypes.windll.user32.SendMessageW(npp_hwnd, NPPM_GETLANGUAGEDESC, language, None)
    buffer = ctypes.create_unicode_buffer(length+1)
    return_value = ctypes.windll.user32.SendMessageW(npp_hwnd, NPPM_GETLANGUAGEDESC, language, ctypes.byref(buffer))
    print buffer.value

# crash
# for language in LANGUAGES:
    # print notepad.getLanguageDesc(language)

[PeterJones]

Strange. I could not get a crash, or even weird behavior, using your “simplified version” (2019-01-24T19:06:19.201Z). With the code before that (2019-01-24T18:33:47.183Z), I never got a crash, but I did get one or two times when NPP prompted me that it was about to exit, and did I want to cancel – whether I said YES or NO to canceling, it stayed in NPP and the script continued looping.

Notepad++ v7.6.2   (64-bit)
Build time : Jan  1 2019 - 00:02:38
Path : C:\usr\local\apps\notepad++\notepad++.exe
Admin mode : OFF
Local Conf mode : ON
OS : Windows 10 (64-bit)
Plugins : ComparePlugin.dll DSpellCheck.dll MarkdownViewerPlusPlus.dll mimeTools.dll NppConverter.dll NppExec.dll NppFTP.dll PythonScript.dll 

I guess I’ll keep an eye out for crashes while using pyscFilteredViewer, and I’ll investigate that call as the culprit.

I’m reluctant to just switch to an alternate method, because the getLanguageDesc() method returns slightly different information than getLexerLanguage(), especially on UDL: “User Defined language file - CustomLanguageName”, vs “user”, and I filter at least one UDL regularly. The getLanguageDesc() appears to match what’s normally put in the STATUSBARSECTION.DOCTYPE location. (If there were just a getStatusBar(statusBarSection) to read back what was previously written with setStatusBar(statusBarSection,text), that would get rid of the need for getLanguageDesc() in my utility. Added #3 to explore this more in the future)

[Meta Chuh]

@PeterJones

looks very promising
side note: the first thing i read this morning was “psychicFilteredReader” … i guess my coffee hadn’t kicked in yet 😂

i didn’t have much time to test it enough for much feedback yet (no crash so far), but i’ll probably compile a static, stand alone one file pyscfr command executable (let’s call it phprun.exe or phpysc.exe for now) to render php files to stdout to fiddle around a bit more.

currently i only have suggestions for the guide:

first feedback about the documentation:
i’d simplify it a bit for newer users, it seems more info than necessary, and missing plug and play examples for the .ini at the “getting started” section.

simplified Installation Procedure (on 7.6.2):

• extract pyscFilteredViewer-0.01-beta.zip and copy the containing folder pyscFilteredViewer to %AppData%\Notepad++\plugins\config\PythonScript\scripts

the current pyscfr manual explains a bit much about where to put the files (18 lines instead of 1-2), and it’s doubled as long as it would have been, by giving the user the choice on whether to install it to machine or user scripts, also leaving some users helpless, as they wouldn’t know which location is right for them.

any further explanation, or guide to an alternative script location, can be done at the end, after a simple getting started guide.

pyscFilteredViewer.ini template:
as it is a completely new product, it’s imperative to ensure an immediate, easy start for any new user, eg. by delivering prebuilt, functioning configuration files (pyscFilteredViewer.ini) to attract people to a new application.

a default ini should be fully working, without relying that a new user understands what to do, in order to ensure a high acceptance count.
(unless it’s an already huge product, like apache, where the devs can say: everyone uses it, so learn it or leave it 😉)

a distributed ini example would be:

[PHPFile]									;   => this is the name of the section; it must be unique.  Typically, based on the language
Extension=.php								;   => space-separated list of filename extensions, including the period.
Language=PHP								;   => this is the name of the language (from Notepad++'s perspective): for a UDL, use your "UserName" from the UDL dialog
Command=C:\usr\local\bin\phpysc.exe "%1"	;   => this is the filter command; %1 is the name of the active file/buffer; the command must result in the HTML being dumped to STDOUT

or even better, an ini using a working preview-ini.bat or exe which is already deployed from the pyscFilteredViewer folder in the zip, as it would always be in %AppData%\Notepad++\plugins\config\PythonScript\scripts\ pyscFilteredViewer instead of a non predictable location on each user’s unique setup as example.

btw: nice, i like the emoji based ⇉📺⇉ in the readme.md 😃

[Meta Chuh]

@PeterJones

btw: wasn’t there someone who recently said:

and I just don’t have the py-knowledge to keep up for long

? 😉👍

[Eko palypse]

@PeterJones @Meta-Chuh

huh - that is strange as this is 100% reproducible at my side.
Later this day I will do further tests with clean npp and clean PS startup.py.

[Eko palypse]

Very strange - I download both latest npp and full PythonScript plugin zips.
Create a new minimal test script and boom - crash? 10 tries ten crashes.

Notepad++ v7.6.2   (64-bit)
Build time : Jan  1 2019 - 00:02:38
Path : D:\temp\clean_762_x64\notepad++.exe
Admin mode : OFF
Local Conf mode : ON
OS : Windows 7 (64-bit)
Plugins : DSpellCheck.dll mimeTools.dll NppConverter.dll PythonScript.dll