Community
    • Login

    Duplicate lines

    Scheduled Pinned Locked Moved Help wanted · · · – – – · · ·
    4 Posts 3 Posters 10.8k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Dusty PulverD
      Dusty Pulver
      last edited by

      I need to edit a mailing list of over 10,000 users.
      Is there a plug-in that will remove duplicate lines?
      If not, any ideas of how it can be done easily?

      1 Reply Last reply Reply Quote 0
      • Scott SumnerS
        Scott Sumner
        last edited by

        If you don’t care if the lines get sorted, the quickest way is to select all text in the file and use the TextFX menu, choosing TextFX Tools, enable the “Sort outputs only UNIQUE (at column) lines” option, and then choose a “Sort Lines…” action (from the same menu where that option is from).

        1 Reply Last reply Reply Quote 2
        • Dusty PulverD
          Dusty Pulver
          last edited by

          Remove duplicates, remove blank lines and sort data in one operation

          Paste the text into Notepad++ (CTRL+V). ...
          Mark all the text (CTRL+A). ...
          Check +Sort outputs only UNIQUE (at column) lines (if not already checked) 
          Click TextFX → Click TextFX Tools → Click Sort lines case insensitive (at column)
          Duplicates and blank lines have been removed and the data has been sorted alphabetically.
          
          1 Reply Last reply Reply Quote 1
          • guy038G
            guy038
            last edited by

            Hello, Dusty Pulver,

            An other simple possibility, that does not need any plugin, would be :

            • Sort your file, with the native N+++ sort ( menu option Edit - Line Operations - Sort Lines Lexicographically Ascending )

            • Move to the beginning of your file ( CTRL + Origin )

            • Open the Replace dialog ( CTRL + H )

            • Perform the S/R, below, in Regular expression search mode :

            SEARCH : (?-s)^(.*\R)\1+

            REPLACE : \1

            Notes :

            • The (?-s) modifier ensures that the dot will match standard characters, only, even if you previously checked the . matches newline option !

            • The \R stands for any kind of EOL character(s)

            Best Regards,

            guy038

            1 Reply Last reply Reply Quote 0
            • First post
              Last post
            The Community of users of the Notepad++ text editor.
            Powered by NodeBB | Contributors