Style Token 1..5 search does not follow default case sensitivity of Find Dialog

Eric Born

When I just started Notepad++, select a word and apply Style Token1 to it, then a case sensitive method is used to find and color the occurrences.
But when I first perform a search (e.g. Ctrl-F3), followed by applying a Style Token, then the Style Token uses the case sensitivity which is defined in the Find-dialog (which is for me always “case-insensitive”).
Why behaves the Style Token method differently in these cases?

Alan Kilborn

@Eric-Born

A guess without checking: The source code is written such that the initial value on this is case-sensitive. When you first do a Find using the find window, the initial value variable gets written with your setting on Find. From then on it follows the Find window setting. Probably considered a bug. I seem to recall an “issue” written on it recently.

Alan Kilborn

Ah…here it is: https://github.com/notepad-plus-plus/notepad-plus-plus/issues/5148

Imgur

“Eric Born”…“ebouden”…both are you?

Eric Born

You’re right 3 times :) :

I agree on your check
I indeed wrote that issue
both are me
Yesterday I couldn’t find my logged issue on the community, and as I did not get any update-emails on that issue anymore, I thought my issue description was perhaps not clear enough, so I logged it here again. But by reading your response, I now realize the issue was logged on github instead of the notepad community. I apologize.

About the issue:

I’ve read that the Style Token 1…5 feature is actually implemented inside Scintilla. Could it perhaps be, that the initial value of the bit in question is located inside Scintilla, and that it gets overwritten with the bit value of the Find-dialog upon doing a real search?

Motivation for a solution of the issue:
I wrote a python script that accepts one or more keywords.
It translates these keywords using a user-defined translation-file and then searches for the translated keyword in a structured text file, writing the results to another text file (in which the top-subject has the most hits (or is most recent)).
This result-file also contains the translated keywords near the top in a piece of structured text.
The script then launches another Notepad++ instance to open that result-file.
I modified “startup.py” (of plugin PythonScript) such that it can detect the structured piece of text in the result-file: it reads the translated keywords from it and performs the Style Token coloring of the first 5 of these keywords.
The speed is good for humans: about 0.2 s when searching for 4 keywords in the text-file containing 6.471 subjects (9 MB), resulting in 4000 subjects, of which the top 20 hits are written to the result-file.
I like the result very much and use it every day (for 25+ years)
If this could be of interest to anyone, I can supply a copy…(don’t know where to put it).
Recently rewrote it from C++ to python, thereby adding some caching due to which the speed of the python version is even higher than the original C++ executable.

It would be really, really nice when this coloring would happen case insensitive (or, preferably, according to the setting in the Find-dialog) w/o having to do a search first.

Possible work-around:
if I can execute a “Crtl-F3” first from the startup.py (Crtl-F3 results in correcting the bit for Style Token searches), then Style Token searches should be ok (==following the case-sensitive-setting of the Find-dialog), but I cannot find how to do a “Crtl-F3” from startup.py.

Alan Kilborn

@Eric-Born said:

it reads the translated keywords from it and performs the Style Token coloring of the first 5 of these keywords

How exactly does the script do this? Does it invoke the menu commands for styling? I ask because, since you are apparently adept at scripting, you could do the styling in a more manual way, and thus avoid limitations about case sensitive/insensitive that you are seeing. Had this not occured to you?

but I cannot find how to do a “Crtl-F3”

What about using the SendKeys Python library?

I think that it is really good that you elaborated on what you are doing/wanting. It gives people the ability to suggest things that normally would not be thought of when someone simply says “feature x doesn’t work right”.

guy038

Hello, @eric-born, @alan-kilborn and All,

Do you know that you do not have to perform a search in order to initialize the style token 1 to 5 with the Whole word only and/or Match case options of classical Search/Replace dialog !

Indeed :

Just open the Find dialog with Ctrl + F shortcut
Verify an/or change the Whole word only and/or Match case options, as desired
Close the Find dialog, with the ESC key
From now on, the Search > Mark All > Using #th Style will highlight, according to these updated settings !

In the case that you’ve just opened N++, without opening the Find dialog yet or performing any search, the default settings used, by a Mark All action, are :

Match whole word only checked
Match case checked
Of course, the implicit options Wrap around and Normal search mode

So Eric, open the Find dialog, verify that the two options are checked and close the dialog with ESC

I agree, that the control of Mark All settings, similar at the Smart HighLighting ( see Settings > Preferences... > Highlighting ), stored in the config.xml file, would be a nice solution ;-))

Best Regards,

guy038

Alan Kilborn

@guy038

I believe your information is correct, but I don’t believe it helps the original poster, as he appears to want to do something under script control right as Notepad++ starts. Or maybe I just misunderstand the relevance.

Alan Kilborn

So here’s an example of the scripting I briefly mentioned before:

import re

def style(x): return 26 - x

def match_found(m):
    editor.setIndicatorCurrent(style(1))
    editor.indicatorFillRange(m.span(0)[0], m.span(0)[1] - m.span(0)[0])

editor.setIndicatorCurrent(style(1))
editor.indicatorClearRange(0, editor.getTextLength())
editor.research(r'\Qhello...there', match_found, re.I)

# hello...there
# Hello...There
# HELLO...THERE

If you run this on its own source code at startup, you’ll get the following:

Imgur

Alan Kilborn

@Eric-Born said:

I’ve read that the Style Token 1…5 feature is actually implemented inside Scintilla. Could it perhaps be, that the initial value of the bit in question is located inside Scintilla

I just noticed this part of your posting.

No, Scintilla just gives its “clients” the ability to set these “styles” (Notepad++ nomenclature) which Scintilla calls “indicators”. In the Pythonscript code just above You can see the direct calls to Scintilla functions where an editor object function is called with “indicator” in its name. Notepad++ itself determines how the styling is done, thus Scintilla is not to blame for any case-sensitive problems.

Eric Born

@alan-kilborn, @guy038

looks very promising! have no time now, will certainly try this, kr

Eric Born

@alan-kilborn said:

How exactly does the script do this? Does it invoke the menu commands for styling? I ask because, since you are apparently adept at scripting, you could do the styling in a more manual way, and thus avoid limitations about case sensitive/insensitive that you are seeing. Had this not occurred to you?

Due to not knowing how to do it (newbie in PythonScript/Scintille), and having found an example that used the MENUCOMMAND.SEARCH_MARKALLEXT1…5, I originally came up with next:
# Code in startup.py
# not shown: some code to read the keywords from the top of the result-file...
# then next (dirty) code performed the coloring, albeit case sensitive:
iColorIndex = 1
for keyword in lstKeywords:
editor.gotoPos(startpos)
editor.setAnchor(startpos+len(keyword))
exec('notepad.menuCommand( MENUCOMMAND.SEARCH_MARKALLEXT%d )' % iColorIndex) # color 1..cnst_MaxColors
iColorIndex += 1
if iColorIndex > cnst_MaxColors:
break
startpos += len(keyword) + 1 # +1 for the space

Your tip about the SendKeys Python library: I definitely will check this out (on my todo-list).

@guy038

I probably mis-wrote my point: as @alan-kilborn said, I liked to have it done w/o any manual intervention.
But thx for your neat comments anyway:)

@alan-kilborn

Didn’t know about the functionality of the editor.XXX_indicator_XXX() Scintilla commands.
So I tested your “example”: it DOES THE JOB perfectly!
In your code, we are not dependent anymore of any settings bit, because “re.I” is specified explicitly, brilliant!

So you supplied the perfect solution and I am very grateful for this!

To explain how my piece of the startup.py code works, here a shot of the first few lines of a “result-file” I mentioned earlier, though I couldn’t achieve the formatting like it is shown in my launched Notepad++:
(note: my apologies for the bad formatting, but I have no skills on this whatsoever, read little in the help, then came up with next)

##LANGUAGE=SERESULTS
Runtime: 0.10 s
SE.py version V01.01
User arguments: script python startup
Used arguments: script python startup
TotalMatches = 631 (displayed = 44)
SkipsFor(+) = 0
SkipsFor(-) = 0
-------------------------------------------------------------------
#Header
#Date 2018/12/14
#Title
#Keywords ORGIIIEGB IIITTIMMERS warning Notepad++ plugins PythonScript PluginManager PythonScript_1.0.8.0.msi specialties features files NiceToKnows problems startup
#Body
2015/09/11:
NEVER install pythonScript using the PluginManager, because that gives problems!
Instead, use the installer PythonScript_1.0.8.0.msi (Stop Notepad++, execute this .msi, then start Notepad++ again))
` Configfile:` ` C:\Users\borne\AppData\Roaming\Notepad++\plugins\config\PythonScriptStartup.cnf` ` startup.py:` ` C:\Program Files (x86)\Notepad++\plugins\PythonScript\scripts\startup.py`
#EndBody
-------------------------------------------------------------------
#Header
#Date 2018/12/21
#Title todo Turning a python script into a website
#Keywords ORGIIIEGB todo Turning a python script into a website
#Body
todo: try this out:
https://blog.pythonanywhere.com/169/
#EndBody
-------------------------------------------------------------------
#Header
#Date 2018/12/20
#Title
#Keywords ORGIIIEGB C:\db\Python_Sources\Notepad++PluginPythonScript\Examples\ BracketHighlighter.py ColumnLexer.py CTags Based Autocompletion.py Disable Virtual Space.py Enable Virtual Space.py EnhancedPythonLexer.py Event Handler Demo.py Formatter.py HideLines.py InsertRuler.py LogfileLexer.py MultiEdit.py Python Regex Replacements.py RegexTester.py Remove and Modify selected lines.py ReplaceDefaultReplaceDialog.py Sorter.py StartWithLocalPython.py Swap2Words.py
#Body
C:\db\Python_Sources\Notepad++PluginPythonScript\Examples\
BracketHighlighter.py
ColumnLexer.py
CTags Based Autocompletion.py

(note that the structured database text file is almost the same:
strip off the part before the first ‘-------------’ and replace all lines ‘-----------’ with empty lines)

And here is the resulting startup.py containing your EXCELLENT solution:

# The lines up to and including sys.stderr should always come first
# Then any errors that occur later get reported to the console
# If you'd prefer to report errors to a file, you can do that instead here.
import sys
from Npp import *
import traceback
import re

# Set the stderr to the normal console as early as possible, in case of early errors
sys.stderr = console

# Define a class for writing to the console in red
class ConsoleError:
	def __init__(self):
		global console
		self._console = console;
		
	def write(self, text):
		self._console.writeError(text);
		
# Set the stderr to write errors in red
sys.stderr = ConsoleError()

# This imports the "normal" functions, including "help"
import site

# This sets the stdout to be the currently active document, so print "hello world", 
# will insert "hello world" at the current cursor position of the current document
sys.stdout = editor

#def switch_language_view(args):
#    notepad.activateBufferID(args["bufferID"])
#    lineone = editor.getLine(0)
#    if '##' in lineone:
#        #lineone = lineone[lineone.rfind('##'):].replace('##', '')
#        language = lineone[lineone.rfind('=')+1:]
#        #lineone = "MENUCOMMAND." + lineone.upper()
#        try:
#            #notepad.menuCommand( eval(lineone) )
#            #notepad.menuCommand(lineone)
#            notepad.runMenuCommand("Language", language)
#        except:
#            pass


def style(x):
  return 26 - x

def match_found(m):
  editor.indicatorFillRange(m.span(0)[0], m.span(0)[1] - m.span(0)[0])

##command to link notification
#notepad.callback(switch_language_view, [NOTIFICATION.FILEOPENED])

#editor.indicatorClearRange(0, editor.getTextLength())  skip this, as nothing is colored yet
try:
  sFirstLinesOfDoc = editor.getTextRange(0,255)
  sTarget1 = '##LANGUAGE=SERESULTS'
  sTarget2 = 'Used arguments: '
  pos = sFirstLinesOfDoc.find(sTarget1)
  if pos == 0:                             # ok, we are probably dealing with a results file, so continue processing
    cnst_FirstStyle = 1
    cnst_MaxStyles = 5   # 5 Token Styles (== Scintilla indicators, thx @Alan-Kilborn)
    pos = sFirstLinesOfDoc.find(sTarget2)
    if pos >= 0:                           # also found this, so way to go
      startpos = pos + len(sTarget2)
      endpos = sFirstLinesOfDoc.find('\r\n', startpos)
      lstKeywords = sFirstLinesOfDoc[startpos:endpos].split()
      currentStyle = cnst_FirstStyle
      for keyword in lstKeywords:
        editor.setIndicatorCurrent(style(currentStyle))
        editor.research(r'\Q' + keyword, match_found, re.I)
        currentStyle += 1
        if currentStyle > cnst_MaxStyles:
          break

except:
  # next error processing code doesn't work:
  # - no error can be seen afterwards, even not in the console when it is instructed to open automatically when an error occurs
  #   (note: the console then opens correctly, but no error appears)
  # - also   editor.insertText()   doesn't work from this exception-block
  strError = r'Exception in file C:\Program Files (x86)\Notepad++\plugins\PythonScript\scripts\startup.py'
  strError += '\n' + traceback.format_exc(100) + '\n'
  editor.insertText(0, strError)           # insert the error at top of file
  print strError

@alan-kilborn said:
…Scintilla is not to blame for any case-sensitive problems…

Of course, you’re right.
I never want to blame anything or anybody, not to mention the fact, that my interpretation often differs from authors’ intention, so I rather play it safe and fair with the gap in between :)

Recap: I am very grateful for your help, gentleman! I am glad that there are people like you, reading problems and react very professionally! (payed services are often worse!)

As a thank you, if you would like to test the searchtool out yourself, let me know how and I’m glad to send you a copy!