working use-case for SCI_REPLACETARGETRE

PeterJones

@Ekopalypse said in working use-case for SCI_REPLACETARGETRE:

@PeterJones
Peter you are confusing me :-D

I’m confused. For some reason, I thought the most recent had been another Alan reply, rather than two Eko’s in a row. Sorry.

Too late for me - I go to bed :-)

Maybe I should to, but since the sun is still up and I haven’t had dinner yet, that’s probably not wise. ;-)

PeterJones

@Ekopalypse said

editor.setSearchFlags(FINDOPTION.REGEXP)

So when I add that line, and comment out the sleeps, it goes back and forth between your two output pairs. If I add in the

console.write(".....\n"+editor.getText()+":::::\n")

after the final getTargetText(), it duplicates whatever was printed above.

first

-----
This is line one !!!
Today is a beautiful day !!!
This is line three !!!
=====
target=(32,105)
beautiful target=(64,73)
great target=(64,69)
final target=(64,69)
-----
This is a not selected line !!!
This is line one !!!
Today is a great day !!!
This is line three !!!
This is a not selected line !!!
=====
.....
This is a not selected line !!!
This is line one !!!
Today is a great day !!!
This is line three !!!
This is a not selected line !!!
:::::

second

-----
This is line one !!!
Today is a beautiful day !!!
This is line three !!!
=====
target=(32,105)
beautiful target=(64,73)
great target=(0,5)
final target=(0,5)
-----
great=====
.....
great:::::

If I have the sleep(1) between beautiful and great, it always does the second. But the first is closer to what I would expect (except that the final getTargetText should be great, not the full text)

This is weird.

It’s going to make it really difficult to see if I’ve implemented correctly in Perl. :-)

Ekopalypse

I was using a clean 7.8.4 portable, disabled all Auto* features such as smart highlighting, auto-complete, etc. of npp and even disabled the 3 built-in plugins, and it happened anyway.

Then I used PySnack and ran it ~100 times and it never happened.
So I suppose the reason why PS has this problem is that PS uses threads when executing scripts, which is not the case with PySnack.
And because npp itself changes the target for different purposes, I assume that there is a conflict while PS is still running.

It’s going to make it really difficult to see if I’ve implemented correctly in Perl.

Tell me about it. :-)

Alan Kilborn

@Ekopalypse said in working use-case for SCI_REPLACETARGETRE:

because npp itself changes the target for different purposes,

Methinks you are onto something here.

PS uses threads when executing scripts, which is not the case with PySnack

Is now the time/place to go into how PySnack is going to be better than PythonScript? :-)
A discussion of the different architectures?

I mean, I presume “script” does things with threads because it needs to? And you decided “snack” doesn’t need to?

What’s better in “snack”? What’s “worse”?

How are things like NOTIFICATIONs and synchronous / asynchronous callbacks going to work with “snack”, without threading (maybe making it seem like I know more than I do here, hehehe, maybe one thing has nothing to do with the other).

I’m definitely in favor of the KISS principle, where it works.

Maybe a separate thread for this type of discussion, but maybe you are not even ready for that yet! :-)

BTW, unfortunate that both plugins could be abbreviated as “P.S.”. :-(

Ekopalypse

The biggest and in my opinion the only advantage of threads is that
Npp itself can’t be disturbed by PS.
Even if someone has programmed an endless loop, npp should,
most of the time, still be able to terminate itself cleanly.
The disadvantage of threads can be seen in this problem.
There is the possibility that 2 or more threads work on the same thing
and cause problems. Scintilla, to my knowledge, is not thread-safe.

PySnack does more or less the same thing as PythonScript.
Except for the obvious, i.e. using Python3 and Cython.
Boost::Regex is, unfortunately, also implemented a bit differently - but I’m not 100% sure if I keep it that way.

At first, threading has nothing to do with the notifications but on the
second view it does if we look at the asynchronous callbacks from PS.
Actually, it works like this.

Npp has a NotificationHandler that is called from Scintlla.
Any plugin can register via the beNotified Callback function,
methods which are called then.

This normally runs synchronously.

But if I now know that I need longer to calculate something,
then my plugin, kindly give this task to a separate thread and
thus return control to Npp more quickly, so nothing is blocked.
But now this thread should not make any changes to scintilla but
inform the actual main thread, that something has to be done.
So this is the theory - sometimes easier, mostly more difficult to implement.

What is worse or better with PySnack is like with art in the eye of the beholder.
I think that, at the moment, almost everything that PS has, PySnack can do too and a little bit more.
But to be honest, it is always easier to create something new from an
existing source then to invent it intially.
So without PS there would never have been the idea to develop PySnack.

BTW, unfortunate that both plugins could be abbreviated as “P.S.”. :-(

Maybe not that bad at all, - switching over could be easier :-)
One of the goals of PySnack is to be able to run the scripts of PS
with almost no changes. Won’t be 100%, but I hope 99% are in.

@PeterJones - sorry for kinda high jacking your thread.

PeterJones

@Ekopalypse said in working use-case for SCI_REPLACETARGETRE:

sorry for kinda high jacking your thread.

Not a problem.

Ran some more experiments this morning: I upgraded from PythonScript 1.5.2 to 1.5.3 (didn’t really change the behavior, but I thought it was worth mentioning). I played around with a sleep between when I generate the initial text and when I do the setSearchFlags – either before or after the setTargetRange.

If I have no sleep, then I get unrepeatable results, where sometimes the final getText is only great, and sometimes it’s the whole paragraph, with great rather than beautiful (as I expect). If I put a small sleep (whether before or after setTargetRange), the final getText is much more reliable. If I use 10ms (ie, sleep(0.01)), it was at least 10/10 reliable; if I drop to 1ms, it seemed better than 50% reliable, but it wasn’t every run; if I have no wait, getText shows just great many/most runs.

I cleaned out some of the debug stuff, and added a loop with a counter for how many runs it got right, and tried a few different delays:

# encoding=utf-8
""" https://community.notepad-plus-plus.org/topic/19217/working-use-case-for-sci_replacetargetre

I am working on implementing editor.replaceTargetRE in Win32::Mechanize::NotepadPlusPlus::Editor,
but it's currently not working.

Found https://notepad-plus-plus.org/community/topic/11804/ with LuaScript/PythonScript examples that _should_ work
"""
from Npp import *
from time import sleep

console.show()
#console.clear()

tWait = 0.005
N = 100
right = 0
srctxt = """This is a not selected line !!!
This is line one !!!
Today is a beautiful day !!!
This is line three !!!
This is a not selected line !!!
"""
cmptxt = """This is a not selected line !!!
This is line one !!!
Today is a great day !!!
This is line three !!!
This is a not selected line !!!
"""

for x in range(N):
    notepad.new()
    editor.setText(srctxt)
    sleep(tWait)

    editor.setTargetRange(32,105)

    editor.setSearchFlags(FINDOPTION.REGEXP)
    editor.searchInTarget(r'beautiful')

    editor.replaceTargetRE('great')

    if editor.getText() == cmptxt:
        right = right + 1

    editor.setSavePoint()
    notepad.close()

console.write("Final Score: tWait={} gave {}/{} right!\n".format(tWait,right,N))

which gives, on my various runs:

Final Score: tWait=0.001 gave 30/100 right!
Final Score: tWait=0.01 gave 100/100 right!
Final Score: tWait=0.002 gave 67/100 right!
Final Score: tWait=0.003 gave 95/100 right!
Final Score: tWait=0.004 gave 99/100 right!
Final Score: tWait=0.005 gave 100/100 right!
Final Score: tWait=0.005 gave 99/100 right!

So, somewhere between 5-10ms, it reliably replaces just the expected text, rather than the whole file. At least for me, with PythonScript v1.5.3 (in 64bit Notepad++ v7.8.5), on the machine I’m currently using. Smaller than that, it appears there’s a race condition, where Scintilla gets ahead of itself (tries to do the search before it’s finished writing its buffer?). Given that most of my professional programming involves programming electronics to generate and measure voltages and other signals, I am so used to adding delays in my software to avoid non-deterministic delays on the order of 1-10ms, it seems rather natural to me to add delays to my code – but I can see why someone who doesn’t come from the world of programming physical devices might balk at adding delays to code.

Also, given that the topic from 2016 had to do with oddities in the final Target location after the replacement, I am not going to worry about that. I think this sequence of setText, wait, setTarget, setSearchFlag, searchInTarget, replaceTargetRE seems reliable enough now in PythonScript that I can use it to help debug my PerlScript version.

Thanks for the hints, confirmations, and experiments.

Alan Kilborn

@PeterJones

Since, as @Ekopalypse pointed out, Notepad++ itself uses the “target” concept, and apparently(?) there can only be one of these that Scintilla knows about, isn’t any use of target-related functions (be it in Pythonscript or your Perl-based thing) going to be potentially problematic when a user tries to use it?

I know you are just trying to get thru implementing/testing/documenting of the function under discussion, but…

PeterJones

@Alan-Kilborn said in working use-case for SCI_REPLACETARGETRE:

isn’t any use of target-related functions going to be potentially problematic

Probably true. I think I will add a caveat to the “target” documentation that explains that Notepad++ might interfere with the Scintilla targets, and suggest they code defensively when using targets.

Ekopalypse

@Alan-Kilborn said in working use-case for SCI_REPLACETARGETRE:

isn’t any use of target-related functions problematic

I would say no, if it is called from the main thread or better the thread
which created the scintilla window.

PeterJones

@Ekopalypse said in working use-case for SCI_REPLACETARGETRE:

if it is called from the main thread or better the thread
which created the scintilla window.

PerlScript is an outside-looking-in application, unlike most plugin development (which starts inside). So I won’t have that, which will make it more important for me to warn users (here and elsewhere, really) that Notepad++ is still running, and could still be influencing things separately from PerlScript.

When I get to v2, with a full plugin that uses my perl module from the inside of Notepad++, I might have better control of such things.

dail

Disclaimer: I did not fully read every single line of this thread…but hopefully I can contribute a bit of helpful info.

Accessing Scintilla on a background thread can be very problematic. There are Scintilla calls that have temporal coupling and since you cannot guarantee any two calls made from PythonScript (or outside of Notepad++) are executed without being interrupted by another thread, then weird things can happen. Even doing something simple like:

editor.setTargetRange(0, 100)
x = editor.getTargetText()

could potentially give unexpected results. Notepad++ does its own work (search/replace, function list, auto completion, smart highlighting, URL highlighting, etc) which changes the target quite frequently.