Community

    • Login
    • Search
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search

    Complex Query (to me) Seeking Help Replacing Text between multiple characters

    General Discussion
    2
    6
    57
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Martijn
      Martijn last edited by

      Hi Professionals,

      Can you help me out on how to replace this text to without comments e.g.

      // Picking up only IOC’s that contain the entities we want\n
      // As there is potentially more than 1 indicator type for matching IP, taking NetworkIP first, then others if that is empty.\n
      // Taking the first non-empty value based on potential IOC match availability\n
      // renaming time column so it is clear the log this came from\n

      The goal is to Replace everything from // to \n

      But as you can see, the characters are attached to each other and my rules are ordered that way.

      If someone knows how to do this, I would be very happy!
      For me it’s a challenge, you are my superhero if you know how.

      Many thanks in advance!

      let dt_lookBack = 1h;\nlet ioc_lookBack = 14d;\nThreatIntelligenceIndicator\n| where TimeGenerated >= ago(ioc_lookBack) and ExpirationDateTime > now()\n| where Active == true\n// Picking up only IOC's that contain the entities we want\n| where isnotempty(NetworkIP) or isnotempty(EmailSourceIpAddress) or isnotempty(NetworkDestinationIP) or isnotempty(NetworkSourceIP)\n// As there is potentially more than 1 indicator type for matching IP, taking NetworkIP first, then others if that is empty.\n// Taking the first non-empty value based on potential IOC match availability\n| extend TI_ipEntity = iff(isnotempty(NetworkIP), NetworkIP, NetworkDestinationIP)\n| extend TI_ipEntity = iff(isempty(TI_ipEntity) and isnotempty(NetworkSourceIP), NetworkSourceIP, TI_ipEntity)\n| extend TI_ipEntity = iff(isempty(TI_ipEntity) and isnotempty(EmailSourceIpAddress), EmailSourceIpAddress, TI_ipEntity)\n| join (\n    DnsEvents | where TimeGenerated >= ago(dt_lookBack)\n    | where SubType =~ \"LookupQuery\" and isnotempty(IPAddresses)\n    | extend SingleIP = split(IPAddresses, \",\")\n    | mvexpand SingleIP\n    | extend SingleIP = tostring(SingleIP)\n    // renaming time column so it is clear the log this came from\n    | extend DNS_TimeGenerated = TimeGenerated\n)\non $left.TI_ipEntity == $right.SingleIP\n| summarize LatestIndicatorTime = arg_max(TimeGenerated, *) by IndicatorId\n| project LatestIndicatorTime, Description, ActivityGroupNames, IndicatorId, ThreatType, Url, ExpirationDateTime, ConfidenceScore, DNS_TimeGenerated,\nTI_ipEntity, Computer, EventId, SubType, ClientIP, Name, IPAddresses, NetworkIP, NetworkDestinationIP, NetworkSourceIP, EmailSourceIpAddress\n| extend timestamp = DNS_TimeGenerated, IPCustomEntity = ClientIP, HostCustomEntity = Computer, URLCustomEntity = Url
      
      PeterJones 1 Reply Last reply Reply Quote 0
      • PeterJones
        PeterJones @Martijn last edited by

        @Martijn ,

        Hopefully I understood what you wanted. I am going to assume that you mean the \n text to mean the actual newline (EOL) character, not the literal text \ followed by the literal text n.

        when I have the text:

        blah blah // comment from here
        // this line is all comment
        blah blah // comment from here
        this line will remain complete
        blah blah // comment from here
        

        FIND = (?-s)//.*$
        * the (?-s) is equivalent to clearing the “. matches newline” checkbox, so that . doesn’t match newlines in the regular expression
        * the //.*$ means "two slashes followed by any characters, up to but not including the newline
        REPLACE = empty
        SEARCH MODE = regular expression

        I end up with

        blah blah 
        
        blah blah 
        this line will remain complete
        blah blah 
        

        If this doesn’t do what you want, then please follow the advice in the italics below, which will help you present the data in a way that we know what you have and what you want it to become. If you do not follow the advice below, I cannot help you any more than I already have.

        ----

        Do you want regex search/replace help? Then please be patient and polite, show some effort, and be willing to learn; answer questions and requests for clarification that are made of you. All example text should be marked as literal text using the </> toolbar button or manual Markdown syntax. To make regex in red (and so they keep their special characters like *), use backticks, like `^.*?blah.*?\z`. Screenshots can be pasted from the clipboard to your post using Ctrl+V to show graphical items, but any text should be included as literal text in your post so we can easily copy/paste your data. Show the data you have and the text you want to get from that data; include examples of things that should match and be transformed, and things that don’t match and should be left alone; show edge cases and make sure you examples are as varied as your real data. Show the regex you already tried, and why you thought it should work; tell us what’s wrong with what you do get. Read the official NPP Searching / Regex docs and the forum’s Regular Expression FAQ. If you follow these guidelines, you’re much more likely to get helpful replies that solve your problem in the shortest number of tries.

        PeterJones 1 Reply Last reply Reply Quote 2
        • PeterJones
          PeterJones @PeterJones last edited by PeterJones

          I guess I will try with the other assumption, as well: that the \n are literally two characters in your text. Such as:

          blah blah // comment from here\n
          // this line is all comment\n
          blah blah // comment from here\n
          this line will remain complete\n
          blah blah // comment from here\n
          this is a line with the literal \ntext multiple times\nwhere you want to remove one comment// this will go away\nand this will stay\n
          

          Using a slightly modified expression, where instead of using the $ (which matches end of line without consuming it) you will use \\n (backslash backslash en), so that it will match the literal \ character followed by the literal n character; you also have to make .* non-greedy by adding ?:
          FIND = (?-s)//.*?\\n
          REPLACE = empty
          SEARCH MODE = regular expression

          becomes

          blah blah 
          
          blah blah 
          this line will remain complete\n
          blah blah 
          this is a line with the literal \ntext multiple times\nwhere you want to remove one commentand this will stay\n
          
          1 Reply Last reply Reply Quote 3
          • Martijn
            Martijn last edited by

            @PeterJones said in Complex Query (to me) Seeking Help Replacing Text between multiple characters:

            (?-s)//.*$

            That’s amazing Peter! Thank you so much.
            I did investigate myself and needed to perform a couple of actions, because the command is one sentence.

            [ ] = spaces within not actually [ ]

            Find what: \n
            Replace with: [ \n ]
            Wrap Around
            SEARCH MODE =Normal

            Then

            Find what: \n
            Replace with: \r\n
            Wrap Around
            SEARCH MODE =Extended

            And then your command:
            FIND = (?-s)//.*$
            REPLACE = empty
            SEARCH MODE = regular expression

            Find what: \r\n
            Replace with: NOTHING
            Wrap Around
            SEARCH MODE =Extended

            Find what: [TAB SPACE]
            Replace with: NOTHING
            Wrap Around
            SEARCH MODE =Normal

            And we’re all good! Thank you so much for your efforts to help me out! You are A wonderful person! You are amazing! May a lot of good vibes come your way! Fantastic!

            Take care my friend!
            Best regards,
            Martijn

            1 Reply Last reply Reply Quote 1
            • Martijn
              Martijn last edited by

              @Martijn said in Complex Query (to me) Seeking Help Replacing Text between multiple characters:

              Find what: \n
              Replace with: \r\n
              Wrap Around
              SEARCH MODE =Extended

              The following command is incomplete (2nd one)

              Find what: \n
              Replace with: \r\n
              Wrap Around
              SEARCH MODE =Extended

              1 Reply Last reply Reply Quote 0
              • Martijn
                Martijn last edited by

                I see

                The 2nd command is replaced with \n and should be attached \ \n together (double backslash).

                1 Reply Last reply Reply Quote 2
                • First post
                  Last post
                Copyright © 2014 NodeBB Forums | Contributors