Feature Request: Sort by IP Address (CIDR Notation)

Avianographer

That is very impressive, guy038! I knew regular expressions were fairly powerful, but I didn’t realize they were that powerful. You’ve inspired me to learn more.

Thank you for the write-up and alternative solution.

Claudia, I’m starting work on modifying your Python script to add in CIDR merging to it. So, for instance, if I have these addresses:

66.137.24.194/32
66.137.24.195/32

The script would know to merge them into a single CIDR range: 66.137.24.194/31. If I succeed, I’ll post the results here in case anyone else could make use of it.

Thanks again, everyone!

guy038

Hi, Dustin Cook and Claudia,

Dustin, from your example of your previous post and with the help of the Wikipedia article, below :

https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing#CIDR_notation

we can deduce, on the same way, that the four CIDR addresses, below :

xxx.xxx.xxx.194/32
xxx.xxx.xxx.195/32
xxx.xxx.xxx.196/32
xxx.xxx.xxx.197/32

can be merged in the unique CIDR range :

xxx.xxx.xxx.194/30

And, that the two CIDR addresses :

xxx.xxx.xxx.194/31
xxx.xxx.xxx.196/31

can, also, be merged in the same CIDR

xxx.xxx.xxx.194/30

Here, we have a good example of the limits of regular expressions :-(( Indeed, these merge actions need some calculus, that cannot be performed by any regex , so that you need to code with a [ script ] language, anyway !

So, Claudia, just be at ease : There are, still, numerous cases, where your loving Python script will be needed :-)))

Cheers,

guy038

Claudia Frank

@Dustin-Cook,
good to see that it is helpful and even better to hear that you will extend its functionality.
As guy038 already said, I’m curios to see the changes ;-))

@guy038,
another great example of the power of regular expression and your devotion for explanation.
Chapeau. I see the time come when an os boots from a regex ;-))))))

Cheers
Claudia

Avianographer

Claudia, are you familiar with Python’s netaddr module? I seem to be having all sorts of trouble getting this to work. Here is what I have so far (including your code at the top).

ipList = [] 												# used to save the ips and do sorting    

def create_ip_list(line_content, line_number, total_lines): # function gets called for each line
    if line_content.find('/') > -1:                         # simple check
        ip, mask = line_content.split('/')                  # first split mask bits from ip
        o1, o2, o3, o4 = [int(x) for x in ip.split('.')]    # split ip to its octets

        if not (o1, o2, o3, o4, int(mask)) in ipList:       # looking for duplicates, check if ip is already in list
            ipList.append((o1, o2, o3, o4, int(mask)))      # not found in list, so append to list

editor.forEachLine(create_ip_list)                          # main function starts here
ipList.sort()                                               # we have all ips, let's sort it

editor.beginUndoAction()                                    # set an undo point, in case of there is a need to undo all
editor.clearAll()                                           # clear editor content
   
for ip in ipList:                                           # iterating over ip list previously saved and
    editor.appendText('{0}.{1}.{2}.{3}/{4}\n'.format(*ip))  # write sorted ips to the editor
    
editor.endUndoAction()                                      # inform editor about action end

ipRange = []
	
def create_range(line_content, line_number, total_lines):   # function gets called for each line
	ipRange.append(IPNetwork(line_content))					# append to list

editor.forEachLine(create_range)							# main function starts here

cidr_merge(ipRange)											# use netaddr cidr_merge to merge CIDR ranges

editor.beginUndoAction()									# set an undo point, in case there is a need to undo all
editor.clearAll()											# clear editor content

for ip in ipRange:											# iterating over ip list previously saved and
	editor.appentText(ip)									# write merged CIDR ranges to the editor

editor.endUndoAction()										# inform editor about action end

I keep getting:

raise AddrFormatError('invalid IPNetwork %s' % addr)
netaddr.core.AddrFormatError: Invalid IPNetwork

It’s like IPNetwork doesn’t realize ‘ip’ is a string in the format it wants for some reason.

Claudia Frank

Hello Dustin,

no, don’t have used it yet but could it be that it is strict about additional eols?
As i converted ip and mask to int, any eol char has been stripped silently.
Maybe give it a try with

ipRange.append(IPNetwork(line_content.strip()))

Which netaddr version do you use?

Cheers
Claudia

Claudia Frank

Hello Dustin,

I’ve downloaded netaddr (0.7.18) and it isn’t strict about the eol.
But it is strict about getting nothing ;-)
I assume you have an empty line, one reason why I used the simple check in my create_ip_list function ;-)

Cheers
Claudia

Avianographer

Alright, here is the final script that does everything I need it to.

from netaddr import *										# import everything from the netaddr module

ipList = [] 												# initialize the array to store the IPNetwork objects

def createIpList(lineContents, lineNumber, totalLines): 	# function used to fill ipList with each line
    if lineContents.find('/') > -1:                         # verify that it is not a blank line by checking for the presence of the "/" in a CIDR range
		ipList.append(IPNetwork(lineContents))				# append to ipList

editor.forEachLine(createIpList)							# main function starts here

result = cidr_merge(ipList)									# use netaddr cidr_merge to merge CIDR ranges. It auto sorts and de-duplicates.

editor.beginUndoAction()									# set an undo point, in case there is a need to undo all
editor.clearAll()											# clear editor content

for ip in result:											# iterating over ipList previously saved and
	editor.appendText(ip)									# write merged CIDR ranges to the editor
editor.appendText("\n")									# add a newline since the IPNetwork object doesn't include one

editor.endUndoAction()										# inform editor about action end

Thanks to everyone, especially Claudia, I now have something that goes beyond my original intentions and fully automates my task. All I have to do is copy/paste/run. So happy!

Also, it turns out cidr_merge sorts and de-duplicates, as well, so I only needed that one function.

Claudia Frank

Hello Dustin,

nice to see that you did it and thank you for pointing me to the netaddr module.
I have played a little with it and I can already see two dns tasks which can take usage
of it.

Cheers
Claudia

Suncatcher

I would greatly appreciate this feature too, and preferably out-of-the box, i.e. without any plugins. Or it’ll be good to built-in this functionality into a plugin.
I noticed that TextFX plugin is dying, so is there any other perspective plugin that can implement this?
guy038, it’s a cool method that was proposed by you, and it works out-of-the-box, which is an advantage over Pythonscript! Thanks for your effort! Brilliant!

Suncatcher

BTW, I noticed that last version of Pythonscript was published in 2014, and it worries me a lot. I wasn’t able to install it from plugin manager (it threw unknown exception) and had to do it manually.
So with the every new version of NPP compatibility issues will be bigger and bigger, and we should search for a replacement anyway.

Scott Sumner

@Suncatcher

I don’t know that you should be “worried, a lot” about the Pythonscript plugin being last published in 2014. Perhaps that just means it is very stable and has few bugs in need of fixing. :-)

What does “search for a replacement” mean??

@Dave-Brotherstone is the author of the Pythonscript plugin, as well as the PluginManager plugin. He has recently been working a lot on updating the PluginManager and finding a new good site for hosting the plugins it manages. I think this work also includes heading toward a build of a 64-bit version…so he is busy, but I’m guessing that when that work is complete he will also strive to achieve a 64-bit build of Pythonscript, as well. So I think it is far from dead, even though it hasn’t been updated since 2014. Just guesses, but maybe educated guesses?

I can think of a few pieces of software that I use that haven’t had new releases since 2007-2009, so to me 2014 is relatively recent!

Suncatcher

@Scott-Sumner said:

Perhaps that just means it is very stable and has few bugs in need of fixing. :-)

Bold claim!)
For me it often means that project is simply abandoned. Abandoned and not maintained for a long time, which means that:

the amount of bugs increased exponentially since then
new features haven’t been implemented for a long time too, whereas technologies are moving forward.
not to mention that in most cases those projects have poor community where it’s impossible to resolve any issue.

But this is a common case and maybe Pythonscipt is another thing.

P.S. Considering that author of Plugin Manager and author of Pythonscript is a same person, and he’s been working a lot on updating the PluginManager, it is especially strange to face issues while installing PS from Manager. I was unable to do it, and not only I.

Scott Sumner

@Suncatcher

It IS rather strange that the Pythonscript plugin has trouble installing via PluginManager, given a common author, I’ll grant you that.

I don’t know the full explanation of the Pythonscript + PluginManager problems, but I haven’t let that discourage me. As you have discovered, installing Pythonscript another way works. I have been using Pythonscript deeply since my very first day of using Notepad++, and I can say I’ve found bugs, but not many. Workarounds are the mainstay of Notepad++ (power) users. Bugs exist in software. Features are lacking. These are just facts.

Suncatcher

@Dustin-Cook BTw, the script from Dustin doesn’t work for me (as well as the first ones), it just cleans the screen with the test data. Maybe I’m doing smth wrong?

Scott Sumner

@Scott-Sumner said:

@Dave-Brotherstone is the author of the Pythonscript plugin, as well as the PluginManager plugin

It appears Dave may be @bruderstein on this forum…maybe if he sees this he could comment on the status of Pythonscript…I’m curious about the state of even the 32-bit version as I just got bit again by the notepad.getFiles() bug. :(

That bug is detailed here: https://github.com/bruderstein/PythonScript/issues/22

I’d fix the bug in a local build as a temporary measure, but thus far I have been unable to build Pythonscript from source (okay, I haven’t worked extensively on getting it to build, but Notepad++ is an easy one to build, and I was hoping for the same from Pythonscript).

Claudia Frank

@Scott-Sumner

Scott, what is the error you get? In the past I compiled it a couple of times
and can’t remember that there was some trouble doing it.

Cheers
Claudia

Scott Sumner

@Claudia-Frank

Since we’re already way off topic, can we take a discussion of building Pythonscript offline to email? If an offline discussion turns up something really interesting, I can post back here (under a new topic) to share with the Community.

Claudia Frank

@Scott-Sumner

sure, @guy038 as far as I remember you already in contact with Scott,
would you be so kind and forward my email to him?

Thanks
Claudia

guy038

Hello Claudia,

No problem : I’ve just sent your e-mail address to Scott !

Good Python discussions, with Scott ;-)

Cheers,

guy038

Claudia Frank

@guy038 - Thank you very much.

Cheers
Claudia