Remove duplicate numerical lines

Scott Sumner

You need to publish the code to add the line-ending to the last line, if it’s missing it

Ok, so here it is; I run a similar (but more complicated one for my own needs) from my startup.py so that it is always in place–and thus I never have to deal with files without line-endings on their last lines.

One thing I don’t like, but haven’t found a good method for handling, is that in certain circumstances (e.g. a Save All), after the script does its work, it can leave you sitting in an tab that is different from the tab that was active before. If people are interested in this script and have ideas about solving that particular problem, I’m interested in hearing them.

Here’s the Pythonscript:

from Npp import notepad, editor, NOTIFICATION

def callback_npp_FILEBEFORESAVE(args):
    line_ending = ['\r\n', '\r', '\n'][notepad.getFormatType()]
    doc_size = editor.getTextLength()
    if editor.getTextRange(doc_size - 1, doc_size) != line_ending[-1]:
        # fix Notepad++'s "broken" functionality and add a line-ending at end-of-file
        editor.appendText(line_ending)

notepad.callback(callback_npp_FILEBEFORESAVE, [NOTIFICATION.FILEBEFORESAVE])

Eko palypse

@Scott-Sumner said:

One comment, though: I’m guessing you pretty much exclusively use Windows. I use Windows/Linux about 75%/25%…because of that I have learned to not think that line-endings are always \r\n. So scripts I post here will work (that’s the goal anyway) with either Windows or Linux (or even Mac) files.

Good point and you offered the solution already, even better :-D

from Npp import editor
lastLineContainsEOL = True if len(editor.getLine(editor.getLineCount()-1)) == 0 else False
line_ending = ['\r\n', '\r', '\n'][notepad.getFormatType()]
lines = editor.getText().splitlines()
uniqueLines = set(lines)
newText = '' 
for line in lines:
    if line in uniqueLines or line.strip() == '':
        newText += line + line_ending 
        if line.strip() != '':
            uniqueLines.remove(line)
editor.setText(newText if lastLineContainsEOL else newText[:-2])

Eko

Scott Sumner

@Eko-palypse :

Yes, but you forgot something. :-)

editor.setText(newText if lastLineContainsEOL else newText[:-len(line_ending)])

PeterJones

To continue with the hijack-tangent of this thread… :-)

@Scott-Sumner said,

If people are interested in this script and have ideas about solving that particular problem, I’m interested in hearing them.

Challenge accepted. :-)

My first idea was that you could track the previous bufferID, and make sure you always activate the previous one. While trying to see if that would help, I noticed that with the exact script you had posted, if all open files were missing EOL, it would save all files, but only fix the EOL on the active file.

That gave me the flash for the solution: in the callback, store the currently-active bufferID, activate the buffer for the argument to the callback (ie, the file being saved), make the changes to the now-active file, then re-activate the originally-active buffer. The script below seemed to do it for me:

from Npp import notepad, editor, NOTIFICATION

def callback_npp_FILEBEFORESAVE(args):
    # the editor.appendText will go to the _active_ buffer, whatever
    # file is currently being saved.  So to solve two birds with one
    # stone, save the active buffer ID, then switch to the buffer ID
    # for this instance of the callback -- now the editor has the
    # correct buffer active.
    oldActiveID = notepad.getCurrentBufferID()
    notepad.activateBufferID(args["bufferID"])

    line_ending = ['\r\n', '\r', '\n'][notepad.getFormatType()]
    doc_size = editor.getTextLength()
    if editor.getTextRange(doc_size - 1, doc_size) != line_ending[-1]:
        # fix Notepad++'s "broken" functionality and add a line-ending at end-of-file
        editor.appendText(line_ending)

    # now that you're done editing, go back to the originally-active buffer
    notepad.activateBufferID(oldActiveID)

notepad.callback(callback_npp_FILEBEFORESAVE, [NOTIFICATION.FILEBEFORESAVE])

I tested this with three open files: two in one view, one in other view; I tried various combinations of which ones needed to be saved, and which ones were missing EOL, and which was active, and it seemed to always do what I intended, but it’s possible that other combinations won’t work.

Scott Sumner

@PeterJones said:

if all open files were missing EOL, it would save all files, but only fix the EOL on the active file

Really? I tested with several open files (at least one of the 3 types, Win/Linux/Mac) that needed fixing and when I did a Save All they all got saved after being modified…hmmm, guess I will have another look…

PeterJones

It was that way for me. But the oldID=activeID, activate(args), edit, activate(oldID) should work for your first problem, even if you didn’t have the second problem that I have.

Scott Sumner

@Eko-palypse

So we all learn from each other here. My favorite line from your script is this one:

lastLineContainsEOL = True if len(editor.getLine(editor.getLineCount()-1)) == 0 else False

In my “callback_npp_FILEBEFORESAVE” script I did it differently…but I like your method, too.

Thinking more about it now, you could also do it like this:

lastLineContainsEOL = True if editor.getText()[-1] in '\n\r' else False

but maybe that pulls a lot of text just to look at the last character…

Scott Sumner

@PeterJones said:

the oldID=activeID, activate(args), edit, activate(oldID) should work

Yea, I had something like that (but even more involved) in my original over-complicated version (mentioned yesterday, originally), but I found some cases where even that didn’t always work right…so I cut it out entirely before posting. Maybe I’ll revisit it…

Eko palypse

@Scott-Sumner

yes, that is what I like about open source project in general.
Everyone can learn from others from different point of views or styles etc …

Yes, but you forgot something. :-)

Correct, different size - damn it. :-)

editor.getText()[-1]
but maybe that pulls a lot of text just to look at the last character…

I thought so too but after rechecking scintilla documentation it looks like
SCI_GETCHARACTERPOINTER is what we are looking for because from the document it states
Grant temporary direct read-only access to the memory used by Scintilla to store the document.

So, editor.getCharacterPointer()[-1] shouldn’t allocate any heap memory at all or maybe just a little tiny bit.

@PeterJones nice one - works for me as well :-)

Eko

Scott Sumner

@Eko-palypse

I get nervous about editor.getCharacterPointer() because of my lack of full understanding about multibyte character encodings (as stated previously, I’m an A-Z person). I suppose, though, in this case we are talking about, there could be no issues…

Eko palypse

@Scott-Sumner

not sure I understand your concerns about this.
I assume it is only the pointer to the text buffer returned and pythonscript
plugin has the python buffer protocol implemented so it should be safe always.
But as written, I assume - don’t really know how it is implemented.

Eko

Scott Sumner

@PeterJones

So to revisit this:

if all open files were missing EOL, it would save all files, but only fix the EOL on the active file

I just re-tested after disabling my more-complicated script, restarting N++, and then activating my script as posted above. I created 3 new named files and make sure each had only a single line of text (only a 1 showing in the line-number margin–thus NO line-ending at end-of-buffer, or anywhere in the buffer for that matter). I activated a different file from these 3 and pressed the Save All toolbar button. Checking all 3 files in turn I found that all had a line 2 in the line number margin and this were all appropriately affected by my callback script.

Not sure why you would see different behavior. I’m running PS 1.3.0.0 and N++ 7.2.2, which for the latter, I’m confident you are not… ;-)

Eko palypse

@Scott-Sumner

I’m in the same position as @PeterJones - only the current active file
does get the eol added.
PS 1.3
Npp 7.5.9

Eko

Scott Sumner

@Eko-palypse @PeterJones

I guess I will stop posting scripts now…let others do the heavy lifting. :-)

…unless and until I move to a newer N++…

Eko palypse

@Scott-Sumner

No, no, no … don’t do that … forget all about I wrote :-)
It never happened - everything is good - bright nice day
and bugs are just animals :-)

Eko

Meta Chuh

@PeterJones said:

To continue with the hijack-tangent of this thread… :-)

hahahaha, i like … word of the day: hijack-tangent ;-D 👍

it’s really refreshing, it is different to other bbs’es where this kind of thought exchange is and has to be done via the bbs’ chat and pm systems, often leaving a sterile one post product at the end without any leads how the author(s) got there.

i think this can be very useful to new interested users to get a better grip of how much time you really need to think about a seemingly simple task and/or consult with others to get a good or perfected result within a short period of time.

and i guess no one could come up with a good and plausible reason to mind about this kind of hijacking anyways.
especially because the op is probably happy since yesterday and the rest is a nice brain storming, coding, testing and sharing.

aaaaand … my post (this one) was the real hijacking … lol

PeterJones

I’m on 7.5.8 32-bit, with PythonScript 1.3.0.0…

And now @Scott-Sumner has actually given me a valid reason to downvote him, because if he stops posting PythonScript, it will be up to me… and I just don’t have the py-knowledge to keep up for long. :-)

Scott Sumner

@PeterJones

valid reason to downvote

First labeled a “retard”, then spanked vigorously by Don Ho in perhaps an interesting overreaction…and now downvoted by Peter… I’m super unpopular. :-(

Meta Chuh

@Scott-Sumner:

don’t worry, you’re not … i’m currently googling for a specific helping hack:

nodebb plugin hack that downvotes from specific users triggers a double upvote
(alternative: hack that there are two upvote buttons left and right of the counter for specific user accounts)

@PeterJones

also don’t worry, i’m currently googling for a specific helping hack:

nodebb browser hack that specific users, who refuse to post scripts, are unable to close the np++ browser window ever again
(alternative: they can close it, but as soon as they do, the same page will pop up on their cellular phones)

looool ;-)

PeterJones

I’m super unpopular

… says the one who has 70% higher reputation than anyone else in these forums. :-) Fortunately, downvotes can be transitory: 'Twas done as a joke, referring back to the time that the interface made it look like I had downvoted Scott. But my real downvotes are reserved for egregious behavior – posting spam in the forums, primarily – so I removed this one.

And yes, it does seem to have been rather rough in the forums recently: gang up on Scott this week; last week, I got so frustrated due to a certain conversation that I was within moments of deciding to not bother reading/posting here; and, I’m sure Don hasn’t had the best time, with all the problems that showed up in the transition to 7.6. Hopefully, it will mellow out again soon.

Anyway, I learned a lot through all the code examples and the experiments they prompted. Thanks all.