Community
    • Login

    Remove duplicate numerical lines

    Scheduled Pinned Locked Moved Help wanted · · · – – – · · ·
    48 Posts 8 Posters 14.5k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Eko palypseE
      Eko palypse @Scott Sumner
      last edited by

      @Scott-Sumner

      what about using OrderedDict from collections?
      Preserves the ordering and dict keys are unique per se.

      from Npp import editor
      from collections import OrderedDict
      _dict = OrderedDict.fromkeys(editor.getText().splitlines())
      editor.setText('\r\n'.join(_dict.keys()))
      

      Eko

      Scott SumnerS 1 Reply Last reply Reply Quote 1
      • Scott SumnerS
        Scott Sumner @Eko palypse
        last edited by

        @Eko-palypse said:

        what about…?

        Sure, why not? Only objection might be the empty line case (my experience is that people usually want their blank lines retained as is, and not removed as duplicates).

        Eko palypseE 1 Reply Last reply Reply Quote 1
        • Eko palypseE
          Eko palypse @Scott Sumner
          last edited by

          @Scott-Sumner

          right, this case makes it a little bit more difficulty, agreed.

          Eko

          1 Reply Last reply Reply Quote 0
          • Eko palypseE
            Eko palypse
            last edited by Eko palypse

            @Scott-Sumner

            What about this

            from Npp import editor
            lastLineContainsEOL = True if len(editor.getLine(editor.getLineCount()-1)) == 0 else False
            lines = editor.getText().splitlines()
            uniqueLines = set(lines)
            newText = '' 
            for line in lines:
                if line in uniqueLines or line.strip() == '':
                    newText += line + '\r\n'
                    if line.strip() != '':
                        uniqueLines.remove(line)
            editor.setText(newText if lastLineContainsEOL else newText[:-2])
            
            • generates unique lines only (ignoring empty lines with and without spaces)
            • preserves ordering
            • preserves usage of last EOL

            Eko

            Scott SumnerS 1 Reply Last reply Reply Quote 2
            • Scott SumnerS
              Scott Sumner @Eko palypse
              last edited by

              @Eko-palypse said:

              What about this

              Sure. I say “whatever works”. Much like I don’t get all fancy about shaving a few characters off a regex, I think with scripts it is to each his own. As long as it does the job, it is super. :-)

              1 Reply Last reply Reply Quote 0
              • Scott SumnerS
                Scott Sumner
                last edited by

                @Eko-palypse

                One comment, though: I’m guessing you pretty much exclusively use Windows. I use Windows/Linux about 75%/25%…because of that I have learned to not think that line-endings are always \r\n. So scripts I post here will work (that’s the goal anyway) with either Windows or Linux (or even Mac) files.

                This may be something you want to consider doing as well. But it doesn’t bother me if you don’t because I understand the meaning of it–for someone that just wants to blindly pick up and use a script and doesn’t understand Python, oh and BTW uses Linux files…it could be a problem.

                BTW, good job! I like seeing Pythonscripts besides my own posted here. Not many people are doing it anymore. :-(

                1 Reply Last reply Reply Quote 3
                • Scott SumnerS
                  Scott Sumner @PeterJones
                  last edited by

                  @PeterJones said:

                  You need to publish the code to add the line-ending to the last line, if it’s missing it

                  Ok, so here it is; I run a similar (but more complicated one for my own needs) from my startup.py so that it is always in place–and thus I never have to deal with files without line-endings on their last lines.

                  One thing I don’t like, but haven’t found a good method for handling, is that in certain circumstances (e.g. a Save All), after the script does its work, it can leave you sitting in an tab that is different from the tab that was active before. If people are interested in this script and have ideas about solving that particular problem, I’m interested in hearing them.

                  Here’s the Pythonscript:

                  from Npp import notepad, editor, NOTIFICATION
                  
                  def callback_npp_FILEBEFORESAVE(args):
                      line_ending = ['\r\n', '\r', '\n'][notepad.getFormatType()]
                      doc_size = editor.getTextLength()
                      if editor.getTextRange(doc_size - 1, doc_size) != line_ending[-1]:
                          # fix Notepad++'s "broken" functionality and add a line-ending at end-of-file
                          editor.appendText(line_ending)
                  
                  notepad.callback(callback_npp_FILEBEFORESAVE, [NOTIFICATION.FILEBEFORESAVE])
                  
                  1 Reply Last reply Reply Quote 2
                  • Eko palypseE
                    Eko palypse
                    last edited by

                    @Scott-Sumner said:

                    One comment, though: I’m guessing you pretty much exclusively use Windows. I use Windows/Linux about 75%/25%…because of that I have learned to not think that line-endings are always \r\n. So scripts I post here will work (that’s the goal anyway) with either Windows or Linux (or even Mac) files.

                    Good point and you offered the solution already, even better :-D

                    from Npp import editor
                    lastLineContainsEOL = True if len(editor.getLine(editor.getLineCount()-1)) == 0 else False
                    line_ending = ['\r\n', '\r', '\n'][notepad.getFormatType()]
                    lines = editor.getText().splitlines()
                    uniqueLines = set(lines)
                    newText = '' 
                    for line in lines:
                        if line in uniqueLines or line.strip() == '':
                            newText += line + line_ending 
                            if line.strip() != '':
                                uniqueLines.remove(line)
                    editor.setText(newText if lastLineContainsEOL else newText[:-2])
                    

                    Eko

                    Scott SumnerS 2 Replies Last reply Reply Quote 2
                    • Scott SumnerS
                      Scott Sumner @Eko palypse
                      last edited by Scott Sumner

                      @Eko-palypse :

                      Yes, but you forgot something. :-)

                      editor.setText(newText if lastLineContainsEOL else newText[:-len(line_ending)])

                      1 Reply Last reply Reply Quote 1
                      • PeterJonesP
                        PeterJones
                        last edited by

                        To continue with the hijack-tangent of this thread… :-)

                        @Scott-Sumner said,

                        If people are interested in this script and have ideas about solving that particular problem, I’m interested in hearing them.

                        Challenge accepted. :-)

                        My first idea was that you could track the previous bufferID, and make sure you always activate the previous one. While trying to see if that would help, I noticed that with the exact script you had posted, if all open files were missing EOL, it would save all files, but only fix the EOL on the active file.

                        That gave me the flash for the solution: in the callback, store the currently-active bufferID, activate the buffer for the argument to the callback (ie, the file being saved), make the changes to the now-active file, then re-activate the originally-active buffer. The script below seemed to do it for me:

                        from Npp import notepad, editor, NOTIFICATION
                        
                        def callback_npp_FILEBEFORESAVE(args):
                            # the editor.appendText will go to the _active_ buffer, whatever
                            # file is currently being saved.  So to solve two birds with one
                            # stone, save the active buffer ID, then switch to the buffer ID
                            # for this instance of the callback -- now the editor has the
                            # correct buffer active.
                            oldActiveID = notepad.getCurrentBufferID()
                            notepad.activateBufferID(args["bufferID"])
                        
                            line_ending = ['\r\n', '\r', '\n'][notepad.getFormatType()]
                            doc_size = editor.getTextLength()
                            if editor.getTextRange(doc_size - 1, doc_size) != line_ending[-1]:
                                # fix Notepad++'s "broken" functionality and add a line-ending at end-of-file
                                editor.appendText(line_ending)
                        
                            # now that you're done editing, go back to the originally-active buffer
                            notepad.activateBufferID(oldActiveID)
                        
                        notepad.callback(callback_npp_FILEBEFORESAVE, [NOTIFICATION.FILEBEFORESAVE])
                        

                        I tested this with three open files: two in one view, one in other view; I tried various combinations of which ones needed to be saved, and which ones were missing EOL, and which was active, and it seemed to always do what I intended, but it’s possible that other combinations won’t work.

                        Scott SumnerS 1 Reply Last reply Reply Quote 3
                        • Scott SumnerS
                          Scott Sumner @PeterJones
                          last edited by

                          @PeterJones said:

                          if all open files were missing EOL, it would save all files, but only fix the EOL on the active file

                          Really? I tested with several open files (at least one of the 3 types, Win/Linux/Mac) that needed fixing and when I did a Save All they all got saved after being modified…hmmm, guess I will have another look…

                          1 Reply Last reply Reply Quote 1
                          • PeterJonesP
                            PeterJones
                            last edited by

                            It was that way for me. But the oldID=activeID, activate(args), edit, activate(oldID) should work for your first problem, even if you didn’t have the second problem that I have.

                            Scott SumnerS 2 Replies Last reply Reply Quote 1
                            • Scott SumnerS
                              Scott Sumner @Eko palypse
                              last edited by

                              @Eko-palypse

                              So we all learn from each other here. My favorite line from your script is this one:

                              lastLineContainsEOL = True if len(editor.getLine(editor.getLineCount()-1)) == 0 else False

                              In my “callback_npp_FILEBEFORESAVE” script I did it differently…but I like your method, too.

                              Thinking more about it now, you could also do it like this:

                              lastLineContainsEOL = True if editor.getText()[-1] in '\n\r' else False

                              but maybe that pulls a lot of text just to look at the last character…

                              Eko palypseE 1 Reply Last reply Reply Quote 1
                              • Scott SumnerS
                                Scott Sumner @PeterJones
                                last edited by

                                @PeterJones said:

                                the oldID=activeID, activate(args), edit, activate(oldID) should work

                                Yea, I had something like that (but even more involved) in my original over-complicated version (mentioned yesterday, originally), but I found some cases where even that didn’t always work right…so I cut it out entirely before posting. Maybe I’ll revisit it…

                                1 Reply Last reply Reply Quote 1
                                • Eko palypseE
                                  Eko palypse @Scott Sumner
                                  last edited by

                                  @Scott-Sumner

                                  yes, that is what I like about open source project in general.
                                  Everyone can learn from others from different point of views or styles etc …

                                  Yes, but you forgot something. :-)

                                  Correct, different size - damn it. :-)

                                  editor.getText()[-1]
                                  but maybe that pulls a lot of text just to look at the last character…

                                  I thought so too but after rechecking scintilla documentation it looks like
                                  SCI_GETCHARACTERPOINTER is what we are looking for because from the document it states
                                  Grant temporary direct read-only access to the memory used by Scintilla to store the document.

                                  So, editor.getCharacterPointer()[-1] shouldn’t allocate any heap memory at all or maybe just a little tiny bit.

                                  @PeterJones nice one - works for me as well :-)

                                  Eko

                                  Scott SumnerS 1 Reply Last reply Reply Quote 2
                                  • Scott SumnerS
                                    Scott Sumner @Eko palypse
                                    last edited by

                                    @Eko-palypse

                                    I get nervous about editor.getCharacterPointer() because of my lack of full understanding about multibyte character encodings (as stated previously, I’m an A-Z person). I suppose, though, in this case we are talking about, there could be no issues…

                                    Eko palypseE 1 Reply Last reply Reply Quote 0
                                    • Eko palypseE
                                      Eko palypse @Scott Sumner
                                      last edited by

                                      @Scott-Sumner

                                      not sure I understand your concerns about this.
                                      I assume it is only the pointer to the text buffer returned and pythonscript
                                      plugin has the python buffer protocol implemented so it should be safe always.
                                      But as written, I assume - don’t really know how it is implemented.

                                      Eko

                                      1 Reply Last reply Reply Quote 0
                                      • Scott SumnerS
                                        Scott Sumner @PeterJones
                                        last edited by

                                        @PeterJones

                                        So to revisit this:

                                        if all open files were missing EOL, it would save all files, but only fix the EOL on the active file

                                        I just re-tested after disabling my more-complicated script, restarting N++, and then activating my script as posted above. I created 3 new named files and make sure each had only a single line of text (only a 1 showing in the line-number margin–thus NO line-ending at end-of-buffer, or anywhere in the buffer for that matter). I activated a different file from these 3 and pressed the Save All toolbar button. Checking all 3 files in turn I found that all had a line 2 in the line number margin and this were all appropriately affected by my callback script.

                                        Not sure why you would see different behavior. I’m running PS 1.3.0.0 and N++ 7.2.2, which for the latter, I’m confident you are not… ;-)

                                        Eko palypseE 1 Reply Last reply Reply Quote 1
                                        • Eko palypseE
                                          Eko palypse @Scott Sumner
                                          last edited by

                                          @Scott-Sumner

                                          I’m in the same position as @PeterJones - only the current active file
                                          does get the eol added.
                                          PS 1.3
                                          Npp 7.5.9

                                          Eko

                                          Scott SumnerS 1 Reply Last reply Reply Quote 0
                                          • Scott SumnerS
                                            Scott Sumner @Eko palypse
                                            last edited by

                                            @Eko-palypse @PeterJones

                                            I guess I will stop posting scripts now…let others do the heavy lifting. :-)

                                            …unless and until I move to a newer N++…

                                            Eko palypseE 1 Reply Last reply Reply Quote 1
                                            • First post
                                              Last post
                                            The Community of users of the Notepad++ text editor.
                                            Powered by NodeBB | Contributors