Show only lines that contain a search term

Paul Wormer

@Alan-Kilborn said in Show only lines that contain a search term:

LineFilterViaHiding1.py

I downloaded your script and executed it on the Npp license (a .txt file) following your prescriptions. You may be glad to hear that it worked on my PC exactly as you described it.

I’m now in the process of reading your code and trying to learn from it. Your style of writing is a long way removed from the style of the very first program I wrote, which was a translation from a Fortran II to an Algol 60 program. In those days Fortran variables and subroutines had names of max 6 characters that, moreover, were capitals (per some old ASCII standard). Algol gave a little bit more freedom, but I followed as much as possible the Fortran original. Of course, there were no classes, neither in Algol nor in Fortran. Function names as LINE_FILTER_VIA_HIDING1.prompt_for_string_to_match() bring back these old memories, far removed as they are from the standards I started with.

Alan Kilborn

@Paul-Wormer

It’s fine to wax poetic about times gone by, but I’d be interested at any time to your feedback on the script, in a critical sense. There’s many directions future development could take on it, and hopefully the script as is, and as it could possibly be will be a model for future native Notepad++ features involving hidden lines.

My “style of writing” scripts has evolved much in the 7+ years I’ve been doing it. The current class-based approach was brought into play at one point over those many years because I disliked the namespace “pollution” that script execution creates, and the class-based stuff minimizes that.

Now many of the scripts I publish (and of course the others that I write for myself) look very similar in structure. This is because I have a script that I run when I want to create a new script! (of course!). I call that script “NewScriptReplacementWithBoilerplateCodeCreation.py” and, as the name implies it replaces (figuratively) the Plugins > PythonScript > New Script menu command. It prompts me for the name of my new script, and the type (simple or complicated – complicated meaning the script will use callbacks or similar mechanisms), and then generates my stock code.

Paul Wormer

@Alan-Kilborn
As I wrote before, I’m reading your script and what better tool than the use of the very script to understand it? I tried to understand the meaning of the dictionary self.settings_by_viewfile_dict and ran your script on it. The result below shows some lines that do not contain this string.

datatraveller1

I would also like to see a new native Notepad++ command Search - Filter, which would “Search As You Type”.

The LineFilter3 plugin is the best alternative for me, but native support with instant filtering the current document in the same window would be much nicer.

Alan Kilborn

@Paul-Wormer said in Show only lines that contain a search term:

The result below shows some lines that do not contain this string.

Hmmmm, right you are. I can obtain the same result. Curious that I didn’t see this type of thing while working on the script. :-(

Ok, so let’s do this:

There’s a line in the script: self.debug = True if 0 else False. With that line as written, “debug” is off. BTW, I write that line that way so that I can change only a single character to turn debugging on or off. Let’s turn it on now, so change the 0 to a 1.

There are two places in the script where the PythonScript function editor.showLines() is called. In one place it is called to get all lines displayed, so right now we don’t care about that one. The one we care about is where the lines with the search string are displayed after all lines are hidden. That one looks like this in the code: editor.showLines(line_start, line_end). Find that line and put this new line after it:

self.dprint('showing lines: {sp1}-{ep1} \t\t\tvia call: editor.showLines({s}, {e})'.format(s=line_start, e=line_end, sp1=line_start + 1, ep1=line_end + 1))

After restarting N++ to pick up the changes, run the functionality again.

Here’s an example of what I see:

Notice that since we’ve turned debugging on, we now see juicy details about the script in the PythonScript console window. In mine we see the lines the script wants to show: line 49, 58, etc. (Don’t be concerned if my line numbers don’t match yours)

The interesting thing is, in the editor window we can see lines 138 thru 143 that don’t have the search text, but the script only turns on display of line 130 before turning on line 147! (There are some other places in the output where this oddity occurs, but lines 138-143 are the most blatantly obvious)

I currently don’t have an explanation for this :-( but I’m going to keep looking at it…

datatraveller1

@Alan-Kilborn I’ve tried you script now, too and it works well for me. Thank you for that!

Is it possible to do a case insensitive search? ( I suppose not yet).

BTW, I tried to copy all filtered lines to the clipboard but the whole file content gets copied - is this the expected behaviour?

datatraveller1

@Paul-Wormer said in Show only lines that contain a search term:

self.settings_by_viewfile_dict

I think what I see is the correct result:

Alan Kilborn

@datatraveller1 said in Show only lines that contain a search term:

Is it possible to do a case insensitive search? ( I suppose not yet).

Right, let’s “walk before we run” with this… :-)

BTW, I tried to copy all filtered lines to the clipboard but the whole file content gets copied - is this the expected behaviour?

For now, yes. If you select lines (e.g. have a “filtered” view and do a Ctrl+a select-all) that cross visible/hidden boundaries and then do some operation on the selection, you’ll get all real lines in that region – reference my cautions about deleting selected text earlier.

datatraveller1

@datatraveller1 said in Show only lines that contain a search term:

I think what I see is the correct result:

@Alan-Kilborn Do you know why I get the correct result if I filter for self.settings_by_viewfile_dict while Paul and you get the wrong result? :-)

Paul Wormer

@datatraveller1 I run the script three times in a new session:

Hide lines not containing the string
Undo it (show all lines again)
Hide lines not containing the string

Then: step 1 gives too many lines and step 3 is OK.

datatraveller1

@Paul-Wormer I still get always correct results (also after repeating steps 1-3 up to 10 times).

Paul Wormer

@datatraveller1 Also in a new session? That is, after closing Npp and starting it again?

datatraveller1

@Paul-Wormer Yes, always correct results.

Paul Wormer

@Alan-Kilborn
I made a small step forward. After line 170 I modified the code as follows:


            # hide all lines first:
            self.show_all_lines(False)
            
            import time
            print('sleep ..')
            time.sleep(60)
            
            # show only lines with hits from the search based on user criterion:
            for (pos_start, pos_end) in L:
                editor.setIndicatorCurrent(SCE_UNIVERSAL_FOUND_STYLE)
                editor.indicatorFillRange(pos_start, pos_end - pos_start)
                line_start = editor.lineFromPosition(pos_start)
                line_end = editor.lineFromPosition(pos_end)
                editor.showLines(line_start, line_end)
            
            time.sleep(60)
            print('Woke up')

The sleep(60) gave me time to look at what’s happening and I noticed that often not all lines are hidden at the first sleep. These lines stay visible after the second sleep. The lines that stay visible are more or less random, so I get the feeling that it is a matter of timing. When I increase the length of the file by adding comments different lines stay visible. Datatraveller1 may well have a faster PC than Alan and me.

guy038

Hello, @alan-kilborn, @paul-wormer, @datatraveller1 and All,

Well, I finished the traditional sequence of greetings, by phone, SMS and letters. So I just tested your Python script, Alan, and here are the results of my experiments !

Personally, I never saw the case of visible lines not containing the literal string to search for. Good point !
Like you, @alan-kilborn, I used the License.txt file, which I duplicated several times and enter the lower-case word free when running the script And, as I increased the size of the test file, the waiting time to get the line margin highlighted, in salmon color, increased in parallel :

                   Search of the literal lower-case word 'free'

Times "License.txt"             Size                  Lines       Hiding lines time
---------------------------------------------------------------------------------------
license.txt ×    1            35,500 bytes            231 lines          0,4  s
license.txt ×    5           177,500 bytes          1,155 lines          1,95 s
license.txt ×   10           355,000 bytes          2,310 lines          3,5  s
license.txt ×   50         1,775,000 bytes         11,550 lines         17    s
license.txt ×  100         3,550,000,bytes         23,100 lines         34    s
license.txt ×  500        17,750,000 bytes        115,500 lines        165    s
license.txt × 1000        35,500,000,bytes        231,000 lines        335    s

Now, if we choose to look for the literal string COPYING, which occurs once only in the license.txt file ( instead of 24 times for the word free ), the results are very fast, in comparison :

                   Search of the literal upper-case word 'COPYING'

Times "License.txt"             Size                  Lines       Hiding lines time
---------------------------------------------------------------------------------------
license.txt × 1000        35,500,000,bytes        231,000 lines         12,5  s

Of course, if I tried to re-run the script, before getting the highlighted line margin, I received the usual message :
Another script is currently running. Running two scripts at the same time could produce unpredicable results, and is therefore disabled

Note that I tested this script with a portable N++ v8.4.6 version, located on an USB drive

Now, @datatraveller1, in order to ONLY copy the visible lines, here is a work-around :

Run the @alan-kilborn Python script first
Open the Mark dialog ( Crl + M )
Write the same string than in Alan’s Python script, in the Find what zone
Tick the four options Bookmark line, Purge for each search, Match case and wrap around, only
Select the Normal search mode
Click on the Mark All button
Run the Search > Bookmark > Copy Bookmarked Lines option or right-click on the bookmark margin and choose it
Open a new tab ( Ctrl + N )
Paste the copied text ( Ctrl + V )

Finally, @alan-kilborn, @paul-wormer and @datatraveller1, regarding the method, wouldn’t it be better to :

Trigger a usual Mark action, with the Bookmark line option set
Elaborate a Python script which simply would hide ALL non-marked lines, from the Mark results !

3 immediate advantages of this method :

You are not limited, anymore to a literal string
You could choose a sensitive or non-insensitive search
You could choose the search mode : Normal, Extended or Regular expresion

Best Regards,

guy038

Alan Kilborn

@guy038 said in Show only lines that contain a search term:

as I increased the size of the test file, the waiting time to get the line margin highlighted, in salmon color, increased in parallel

I did mention earlier: “Large files may produce performance problems. Scripts aren’t blazingly fast…”

I use a variant of this script on what I consider a large file (in the work that I do). It works acceptably, performance wise.

HOWEVER: The correctness (or rather incorrectness) problem observed by some users needs to be addressed; if it doesn’t show/hide the correct lines, who cares how fast/slow it is?

wouldn’t it be better to :
Trigger a usual Mark action, with the Bookmark line option set
Elaborate a Python script which simply would hide ALL non-marked lines, from the Mark results !

The downsides to that approach are:

It requires multiple steps
Bookmarks become dedicated to that purpose (I like to “filter lines” and then set meaningful bookmarks on only some of the matches, before revealing all lines again and continuing to work with what I’ve bookmarked)

The user interface of the script could evolve to allow things like regex, match-case, etc. But…let’s go for correctness first.

guy038

Hi, @alan-kilborn and All,

Ah…OK. I understand your working method. You should indeed remove these false positive answers, first of all !

By the way, I’m really surprised that I never encountered any visible false lines during my tests, even in the case of a heavy file ! However, my laptop is not a WAR LIGHTNING ! Bought in July 2021, this HP laptop comes with Window 10 Pro 64, a 512 Mb Solid State Drive and 16 Mb DDR4 of RAM.

Moreover, as I have not finished cleaning my old micro and moved all my data yet, I still use a USB key containing the N++ portable version 8.4.6 for all my tests, which must certainly slow down the execution of your Python script ??

BR

guy038

Alan Kilborn

@guy038 said in Show only lines that contain a search term:

I have not finished cleaning my old micro and moved all my data yet,

What is taking so long? Are you ever going to finish with this?? :-)

I still use a USB key containing the N++ portable version 8.4.6 for all my tests, which must certainly slow down the execution of your Python script ??

I don’t see how…everything should be “in memory” for the operations of the script.

I’m really surprised that I never encountered any visible false lines during my tests

Yes that is interesting…

You should indeed remove these false positive answers, first of all !

Indeed. However, at the moment, I’m out of ideas on what the problem might be. :-(

datatraveller1

Hi all,
Just two thoughts

I’m not sure - what does PC speed have to do with line visibility?
Doesn’t “Find All in Current Document” basically do the same thing as the Python script? Actually, the Notepad++ programmer just has to output the same thing in the same window to filter the text, doesn’t he?

Alan Kilborn

@datatraveller1 said in Show only lines that contain a search term:

I’m not sure - what does PC speed have to do with line visibility?

I don’t think it has anything to do with it, but until the root cause is found, everything is “on the table” as a possible problem.

Doesn’t “Find All in Current Document” basically do the same thing as the Python script?

I think the key idea behind the script is you can edit in the filtered view.