7.7.1 breaks Edit > Line Operations > Sort Lines As Integers Ascending for CIDR IPv4

piranpiran

The unordered ASN data list is REALLY BIG. Whittled down logically with the Sort With Integer option. Manually cutting out all the unwanted stuff below the numbers leaves (prior to 7.7.1) leaves a perfectly ordered/sorted IPv4 CIDR.

Bug filed…
https://github.com/notepad-plus-plus/notepad-plus-plus/issues/5839#issue-461105546

piranpiran

@SinghRajenM
See comment(s) in the bug site… there doesn’t seem to be an asinine timeout dragon thereabouts.

Alan Kilborn

@piranpiran

What’s “asinine” are the spammers…which is what the “timeout dragon” was put in place to counteract. Regardless, you have a reputation of 2+ now, so the dragon shouldn’t bother you further.

@PeterJones said:

The best way to get it fixed: have “before” data*, have after data for v7.7, and the wrong after data for v7.7.1

From looking at the created issue…apparently it was desired to take a sub-optimal route to “fixing”.

;)

Also, you should add a link to this thread from the issue.

PeterJones

@piranpiran said:

unordered ASN data list is REALLY BIG

So what? The subset of data you showed us above (which is a measely 13 lines) does show the bug; that would have been sufficient. You just needed three copies: one in completely unsorted, one in v7.7 sorted, and one in v7.7.1 wrongly-sorted; Alan wished (after the fact) for you to have originally posted all three of those here, which would have made your question easier to understand; and I explicitly suggested you include that in the official bug report, which you apparently ignored.

As it is, without even a link to this thread, there’s no reason for anyone there to believe that there’s a problem, because in the issue page, you don’t show any evidence of the issue, or a data set that will reproduce it, and your description of the issue is not sufficient to reproduce it. It boggles me that someone would go to more effort here to ask for help then when they do the official issue reporting, where something can be done about it.

@Alan-Kilborn:

Oh, wait…you are giving what 7.7.1 outputs, not what you want. I wish you’d given both (I’m too lazy to try an earlier version…for a problem specific to you). :)
Edit: No, that doesn’t make sense either!!!

Once I saw that the posted data was partially sorted – ie, sorted by the first octet, but didn’t sort on the second octet or beyond – and once I confirmed that if I threw that partially-sorted data into my 7.7, it finished sorting by the additional octets (which was the goal, from what I understood), I just assumed that randomly-ordered data in v7.7.1 would sort to the order that was originally posted; I haven’t confirmed that yet myself. (I haven’t yet taken the opportunity to download a copy of v7.7.1… I guess I’ll have to do that at some point.)

PeterJones

@Alan-Kilborn said:

you should add a link to this thread from the issue.

Oh, good, looks like it just happened. Thank you, @piranpiran : I had been boggled by your more recent posts there and here, and was starting to be disappointed, but that disappointment is fading now. Hopefully, you’ll take to heart the comments about showing all three sets of data in the issue, too.

piranpiran

@Alan-Kilborn
Thank you for that helpful consideration.

@PeterJones
The before data is HUGE. Providing it is likely to cause confusion… I do not see npp’s lead programmer being confused with my description …should a fix be on the repair roadmap. Link to the (huge) raw ASN file:
https://www.robtex.com/as/AS38895.html

PeterJones

@piranpiran said:

The before data is HUGE. Providing it is likely to cause confusion…

No, valid before data is a completely-unsorted version of the 13 lines you’ve already pasted – if necessary, just manually cut a few lines and move them around. That’s not huge, that’s 13 lines.

Alan Kilborn

@piranpiran

In computer science, when reporting bugs, it is customary to reduce a large dataset down to a sample that is just big enough to fully illustrate an issue. That’s all that was meant. Ideally, you should take a subset of the huge data and show it the 3 ways requested.

piranpiran

@PeterJones said:

Hopefully, you’ll take to heart the comments about showing all three sets of data in the issue, too.

Programmers tend to like short & sweet - avoids the dreaded TL;DR :-)

chcg

Maybe https://github.com/notepad-plus-plus/notepad-plus-plus/commit/ff20c264df4167943fff6247fec4b0c0ce6227fb#diff-4608be755b00f4ec444233203ee8eafc changed the behaviour as it is a change to the sorting introduced from 7.7 -> 7.7.1.

PeterJones

@piranpiran said:

Programmers tend to like short & sweet - avoids the dreaded TL;DR :-)

In what world is 13 lines of data TL;DR?

Unsorted:

13.228.0.0/15 REACH
15.177.48.0/21 Amazon ROUTE53
18.140.0.0/15 Amazon EC2 SIN prefix source:RADB
35.154.0.0/16 Amazon EC2 BOM prefix source:RADB
43.250.192.0/24 Amazon Asia-Pacific Resources
13.228.0.0/15 Amazon EC2 SIN prefix source:RADB
18.138.0.0/15 Amazon EC2 SIN prefix source:RADB
18.142.0.0/15 Amazon EC2 SIN prefix source:RADB
43.250.193.0/24 Amazon Asia-Pacific Resources
13.250.0.0/15 Amazon EC2 SIN prefix source:RADB
43.250.193.0/24 Amazon SIN Prefix source:RADB
18.136.0.0/16 Amazon EC2 SIN prefix source:RADB
43.250.192.0/24 Amazon SIN prefix source:RADB

Sorted as integers in v7.7 (desired behavior)

13.228.0.0/15 Amazon EC2 SIN prefix source:RADB
13.228.0.0/15 REACH
13.250.0.0/15 Amazon EC2 SIN prefix source:RADB
15.177.48.0/21 Amazon ROUTE53
18.136.0.0/16 Amazon EC2 SIN prefix source:RADB         HERE in v7.7, 136 came before 138, as desired
18.138.0.0/15 Amazon EC2 SIN prefix source:RADB
18.140.0.0/15 Amazon EC2 SIN prefix source:RADB
18.142.0.0/15 Amazon EC2 SIN prefix source:RADB
35.154.0.0/16 Amazon EC2 BOM prefix source:RADB
43.250.192.0/24 Amazon Asia-Pacific Resources
43.250.192.0/24 Amazon SIN prefix source:RADB
43.250.193.0/24 Amazon Asia-Pacific Resources
43.250.193.0/24 Amazon SIN Prefix source:RADB

Sorted as inters in v7.7.1 (unwanted behavior)

13.228.0.0/15 REACH
13.228.0.0/15 Amazon EC2 SIN prefix source:RADB
13.250.0.0/15 Amazon EC2 SIN prefix source:RADB
15.177.48.0/21 Amazon ROUTE53
18.140.0.0/15 Amazon EC2 SIN prefix source:RADB
18.138.0.0/15 Amazon EC2 SIN prefix source:RADB
18.142.0.0/15 Amazon EC2 SIN prefix source:RADB
18.136.0.0/16 Amazon EC2 SIN prefix source:RADB		THIS ONE IS SORTED WRONG IN v7.7.1
35.154.0.0/16 Amazon EC2 BOM prefix source:RADB
43.250.192.0/24 Amazon Asia-Pacific Resources
43.250.193.0/24 Amazon Asia-Pacific Resources
43.250.193.0/24 Amazon SIN Prefix source:RADB
43.250.192.0/24 Amazon SIN prefix source:RADB

See how simple that was? A short set of data that completely shows the bug you are reporting.

I even gave it to you for free. Hopefully, you’ll copy/paste this into the github issue.

piranpiran

@PeterJones

Unsorted para confuses me. It’s not what I do or did or even want to do.
Sorted as integers para is unavailable to me as I don’t have 7.7 any more.
Sorted as inters [sic] para is another way of putting my original premis. Its last line is incorrectly ordered.

I chose to put my difficulty into words within my ‘heads-up’ bug report. Also I pointed out exactly where it’s going wrong too.

Thank you for your opinion …which I fully respect. I need to go away and fix my own server’s anti-spammer dragon which I can see is currently tying up your outfit’s notifications delivery server. That whitelisting activity necessarily takes the server offline for a while so please don’t take offence.

PeterJones

@piranpiran said:

Unsorted para confuses me. It’s not what I do or did or even want to do.

So take a step back here: I believe there are really two activities going on simultaneously, which you need to separate in your mind. 1) You have a complicated, huge set of data which you want to sort. 2) You’ve found a bug in the sorting algorithm, which the 13 lines of data evidence.

For #1: Until the bug is fixed, or unless you revert to v7.7, you are not going to get your actual huge data sorted. If you need that done quickly, go download v7.7 and run using that (if you don’t want to uninstall v7.7.1, then you can download the portable “zip” version, and export to a folder (on your desktop or similar) and run from there to get your task done. Thus, ignore #1 for now

So, on to #2: accurately and helpfully reporting the bug. To do that in an unconfusing manner, you need to supply a small set of unsorted data which will give the different results depending on whether it’s in v7.7 or v7.7.1. The thirteen lines I showed in “unsorted” is just such a set. Of course you don’t want this data in this order: this is just a starting point that will show the bug. At this point, remember, all we’re considering is the bug itself, not whether the data matches your full data set. Of course it’s not properly sorted: you need to start with unsorted data to show a bug in a sorting algorithm. Of course it doesn’t have all the extra data in your original data: it’s a short, self-contained data set that shows all the issues with little extraneous information.

Sorted as integers para is unavailable to me as I don’t have 7.7 any more.

And I’ve kindly given you the result of sorting in v7.7. So you don’t need v7.7… though you could grab as easily as I grabbed v7.7.1 portable/zip to be able to confirm the v7.7.1 sort-order after my last post.

Sorted as inters [sic] para is another way of putting my original premis.

Sorry for the typo; I meant “Sorted as inters in v7.7.1 (unwanted behavior)”, of course. And that paragraph was meant to match your original problem statement. Note that in your github issue, you didn’t even give that much information, which was one of my points to you.

Its last line is incorrectly ordered.

Yep, the unsorted data shows the bug in more than one location. I just highlighted the first one I noticed.

Also I pointed out exactly where it’s going wrong too.

Except you had no data; you didn’t show what you started with, what you expected, or what you were wrongly getting. You gave some words with jargon/abbreviations. Not all programmers deal with “CIDR” or “raw ASN listing”, and so you may be asking them to go look up what those terms are, which seems less helpful than supplying the 13 lines of data that you originally posted here (or the three sets of 13 lines which truly show the whole issue, unsorted, wrongly sorted, and desired sort order); not all software development teams handling bug reports want to come up with their own data set which may or may not evidence the bug being reported, when it would be so much simpler if the person reporting the bug also included data.

I need to go away and fix …

Good luck.

piranpiran

@chcg said:

Maybe https://github.com/notepad-plus-plus/notepad-plus-plus/commit/ff20c264df4167943fff6247fec4b0c0ce6227fb#diff-4608be755b00f4ec444233203ee8eafc changed the behaviour as it is a change to the sorting introduced from 7.7 -> 7.7.1.

Good point.

[18446744073709551615] :: http://www.cplusplus.com/reference/climits/
“ULLONG_MAX - Maximum value for an object of type unsigned long long int - 18446744073709551615 ((2 to the 64) -1) or greater”
OK, a big number …issue may depend on how IPv4 octet max 255.255.255.255 is now being interpreted or parsed?

[7.7.1 change notes] :: “Fix crash while sorting lines with numbers longer than 20 digits” :: I’ve used the ‘previous’ sort mechanism in npp intensively daily for months. No crashes. Sort is reliable & quick. System copes. Something else in 7.7.1 changed or is flawed?

Alan Kilborn

@piranpiran said:

Fix crash while sorting lines with numbers longer than 20 digits

Apparently someone noticed a problem with really long (as in digits) numbers and an attempt was made to fix that. A side effect was that it broke the undocumented behavior you were used to. It remains to be seen if the developers think your desired undocumented behavior is worth bringing back.

piranpiran

Too tired now. The workaround I suggested in the bug site thread did not satisfy at all times. I tried to edit it to no avail. Then I hit its preview delete button and now the whole bug report thread has been deleted or is unavailable (possibly another dragon). If a mod can restore the bug report then good otherwise I’ll have to try and retrace steps after some decent sleep. If anyone’s interested. Seems like a minority vertical issue and it’s just me reporting:-/ g’night

guy038

Hello, @piranpiran, and All,

As I’ve verified that lexicographically sort is NOT broken in the v7.7.1 release, and acts exactly like the prior releases, you could use a regex S/R which would change the IPV4 addresses in such a way that a lexicographically sort would be possible !!

Let’s have a try on your small sample text, below, pasted in a new N++ tab :

13.228.0.0/15 REACH
15.177.48.0/21 Amazon ROUTE53
18.140.0.0/15 Amazon EC2 SIN prefix source:RADB
35.154.0.0/16 Amazon EC2 BOM prefix source:RADB
43.250.192.0/24 Amazon Asia-Pacific Resources
13.228.0.0/15 Amazon EC2 SIN prefix source:RADB
18.138.0.0/15 Amazon EC2 SIN prefix source:RADB
18.142.0.0/15 Amazon EC2 SIN prefix source:RADB
43.250.193.0/24 Amazon Asia-Pacific Resources
13.250.0.0/15 Amazon EC2 SIN prefix source:RADB
43.250.193.0/24 Amazon SIN Prefix source:RADB
18.136.0.0/16 Amazon EC2 SIN prefix source:RADB
43.250.192.0/24 Amazon SIN prefix source:RADB

Open the Replace dialog ( Ctrl + H )

SEARCH \b(((\d)?\d)?\d)(?=\.|/)
REPLACE (?2(?3:\x20):\x20\x20)\1
Tick, preferably, the Wrap around option
Select the Regular expression search mode
Click on the Replace All button

Et voilà ! You should get the following text :

 13.228.  0.  0/15 REACH
 15.177. 48.  0/21 Amazon ROUTE53
 18.140.  0.  0/15 Amazon EC2 SIN prefix source:RADB
 35.154.  0.  0/16 Amazon EC2 BOM prefix source:RADB
 43.250.192.  0/24 Amazon Asia-Pacific Resources
 13.228.  0.  0/15 Amazon EC2 SIN prefix source:RADB
 18.138.  0.  0/15 Amazon EC2 SIN prefix source:RADB
 18.142.  0.  0/15 Amazon EC2 SIN prefix source:RADB
 43.250.193.  0/24 Amazon Asia-Pacific Resources
 13.250.  0.  0/15 Amazon EC2 SIN prefix source:RADB
 43.250.193.  0/24 Amazon SIN Prefix source:RADB
 18.136.  0.  0/16 Amazon EC2 SIN prefix source:RADB
 43.250.192.  0/24 Amazon SIN prefix source:RADB

This regex adds the appropriate number of space character(s) to any number, before the / symbol, containing from 1 to 3 digits, in order to align the four blocks of an IPV4 address ;-))

Now, after using the Edit > Line Operations > Sort lines Lexicographically Ascending menu option, you should be left with your expected sort :

 13.228.  0.  0/15 Amazon EC2 SIN prefix source:RADB
 13.228.  0.  0/15 REACH
 13.250.  0.  0/15 Amazon EC2 SIN prefix source:RADB
 15.177. 48.  0/21 Amazon ROUTE53
 18.136.  0.  0/16 Amazon EC2 SIN prefix source:RADB
 18.138.  0.  0/15 Amazon EC2 SIN prefix source:RADB
 18.140.  0.  0/15 Amazon EC2 SIN prefix source:RADB
 18.142.  0.  0/15 Amazon EC2 SIN prefix source:RADB
 35.154.  0.  0/16 Amazon EC2 BOM prefix source:RADB
 43.250.192.  0/24 Amazon Asia-Pacific Resources
 43.250.192.  0/24 Amazon SIN prefix source:RADB
 43.250.193.  0/24 Amazon Asia-Pacific Resources
 43.250.193.  0/24 Amazon SIN Prefix source:RADB

Finally, to get the initial layout of your IPV4 list, use the simple regex S/R, below :

SEARCH (?-s)\x20+(?=.*/)

REPLACE Leave EMPTY

Here we are !

13.228.0.0/15 Amazon EC2 SIN prefix source:RADB
13.228.0.0/15 REACH
13.250.0.0/15 Amazon EC2 SIN prefix source:RADB
15.177.48.0/21 Amazon ROUTE53
18.136.0.0/16 Amazon EC2 SIN prefix source:RADB
18.138.0.0/15 Amazon EC2 SIN prefix source:RADB
18.140.0.0/15 Amazon EC2 SIN prefix source:RADB
18.142.0.0/15 Amazon EC2 SIN prefix source:RADB
35.154.0.0/16 Amazon EC2 BOM prefix source:RADB
43.250.192.0/24 Amazon Asia-Pacific Resources
43.250.192.0/24 Amazon SIN prefix source:RADB
43.250.193.0/24 Amazon Asia-Pacific Resources
43.250.193.0/24 Amazon SIN Prefix source:RADB

This second regex searches for any non-null amount of space characters, located before the / symbol and delete them, as the replacement zone is Empty

I just hope that these two regex S/R work nice, too, on your VERY BIG unordered ASN data list ;-))

Best Regards,

guy038

AngryGamer

@piranpiran The dragon is slayed. Rest easy. PR#5843 😛

Alan Kilborn

I’d suggest that the OP test the process put forward by @guy038 and, if it validates, record these actions into a macro for future use. As long as the sort algorithm in N++ 7.7.1 sorts true integers correctly, I say there is no bug and the code should be left alone.

The dragon is slayed. Rest easy PR#5843

Not sure there was a dragon to slay, other than the posting dragon for those with reputation < 2. :)

AngryGamer

@Alan-Kilborn False. No matter what there’s a dragon to slay here. Re-check PR#5843 to see why the v7.7.1 setup is a complete mess. I’ve added more information to it.