Community

    • Login
    • Search
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search

    How to delete duplicate lines ?

    Help wanted · · · – – – · · ·
    3
    7
    152
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • bolapedia gg
      bolapedia gg last edited by

      i am wondering how to delete duplicate lines in notepad++

      for example
      1
      1
      2
      3
      3
      4
      4
      4

      result
      2

      Alan Kilborn 1 Reply Last reply Reply Quote 0
      • Alan Kilborn
        Alan Kilborn @bolapedia gg last edited by Alan Kilborn

        @bolapedia-gg

        find: ^(\d+)(\R\1)+\R
        replace: nothing
        search mode: regular expression

        bolapedia gg 1 Reply Last reply Reply Quote 1
        • bolapedia gg
          bolapedia gg @Alan Kilborn last edited by

          @alan-kilborn
          its not worked, if i have many lines, 5 millions data
          i would like to delete duplicate data

          a
          a
          b
          c
          c
          c
          d
          d
          d
          d
          e
          f
          f
          g
          h
          h
          h

          result expected
          b
          e
          g

          Alan Kilborn Neil Schipper 2 Replies Last reply Reply Quote 0
          • Alan Kilborn
            Alan Kilborn @bolapedia gg last edited by

            @bolapedia-gg said in How to delete duplicate lines ?:

            its not worked

            Did you try it on the data you originally supplied?

            1 Reply Last reply Reply Quote 2
            • Neil Schipper
              Neil Schipper @bolapedia gg last edited by

              @bolapedia-gg
              This regex does the transformation: (^.*?\R)\1+

              But first go to the end of the file and make sure there is one blank line at the end. (If this isn’t satisfactory I’m pretty sure there’s a way to fix that).

              Alan Kilborn bolapedia gg 2 Replies Last reply Reply Quote 2
              • Alan Kilborn
                Alan Kilborn @Neil Schipper last edited by

                @neil-schipper said in How to delete duplicate lines ?:

                I’m pretty sure there’s a way to fix that).

                (?-s)^(.+)(?:\R\1)+(\R|\z)

                Use * instead of + if it is desired to remove duplicate contiguous empty lines.

                1 Reply Last reply Reply Quote 4
                • bolapedia gg
                  bolapedia gg @Neil Schipper last edited by

                  This post is deleted!
                  1 Reply Last reply Reply Quote 0
                  • First post
                    Last post
                  Copyright © 2014 NodeBB Forums | Contributors