One line per 1000 items in a macro?

sepodele · Jan 31, 2020, 12:36 PM

I have a macro that separates every line with a comma and ’ ’ around it and put everything on one line.

111
222
333

…becomes ‘111’, ‘222’, ‘333’. I use this with SQL IN to do stuff like “select * from table1 where name in (‘111’,‘222’,'333)”. The problem is that Oracle gives ORA-01795 error if there are more than 1000 items. A workaround is this:

select field1, field2, field3 from table1
where name in (‘value1’, ‘value2’, …, ‘value999’)
or name in (‘value1000’, …, ‘value1999’)
or …;

Is anyone able to modify my macro to separate the result into one line per 1000 items?

    <Macro name="Comma separate lists with '' in front and behind" Ctrl="no" Alt="no" Shift="no" Key="0">
        <Action type="3" message="1700" wParam="0" lParam="0" sParam="" />
        <Action type="3" message="1601" wParam="0" lParam="0" sParam="(.+)" />
        <Action type="3" message="1625" wParam="0" lParam="2" sParam="" />
        <Action type="3" message="1602" wParam="0" lParam="0" sParam="&apos;\1&apos;" />
        <Action type="3" message="1702" wParam="0" lParam="770" sParam="" />
        <Action type="3" message="1701" wParam="0" lParam="1609" sParam="" />
        <Action type="3" message="1700" wParam="0" lParam="0" sParam="" />
        <Action type="3" message="1601" wParam="0" lParam="0" sParam="\r\n" />
        <Action type="3" message="1625" wParam="0" lParam="1" sParam="" />
        <Action type="3" message="1602" wParam="0" lParam="0" sParam="," />
        <Action type="3" message="1702" wParam="0" lParam="770" sParam="" />
        <Action type="3" message="1701" wParam="0" lParam="1609" sParam="" />

PeterJones · Jan 31, 2020, 2:29 PM

@sepodele said in One line per 1000 items in a macro?:

macro to separate the result into one line per 1000 items?

The macro “language” does not have any idea of decisions, counters, indexes, and the like. It is just a list of windows messages / command IDs that are sent to Notepad++.

If you want “decisions” or “counters”, then the Notepad++ macro feature is not what you are looking for. What you’re really looking for is a script; there are multiple scripting plugins available for Notepad++, including PythonScript and LuaScript. Or you could use any programming language (compiled, interpreted, or otherwise) to edit a file that’s not inside Notepad++ at the moment.

base64-encoded external solution in perl; someone with better python intuition than I have could probably come up with similar for use in the PythonScript plugin.

SWYgeW91IGhhdmUgU3RyYXdiZXJyeSBvciBBY3RpdmVTdGF0ZSBwZXJsIG9uIHlvdXIgbWFjaGluZSwgeW91IGNvdWxkIHJ1biBgcGVybCBxdW90ZWNvbW1hMWsucGwgaW5wdXQudHh0YCB1c2luZyB0aGUgYmFzZTY0LWVuY29kZWQgc2NyaXB0IGJlbG93LCB3aGVyZSBpbnB1dC50eHQgaGFzIHlvdXIgaW5kaXZpZHVhbCBsaW5lcyB0aGF0IHlvdSB3YW50IHRvIHF1b3RlLWFuZC1jb21tYSwgYW5kIGl0IHdvdWxkIHByaW50IHRoZSBjb21tYS1zZXBhcmF0ZWQgbGlzdCB0byBTVERPVVQuDQpgYGANCndoaWxlICg8Pikgew0KICAgIGNob21wOw0KICAgICR4IC49ICcsJyBpZiBsZW5ndGgoJHgpPjA7DQogICAgJHggLj0gc3ByaW50ZiBxcSgnJXMnKSwgJF87DQogICAgaWYoMD09JC4lMTAwMCkgew0KICAgICAgICBwcmludCAkeCwgIlxuIjsNCiAgICAgICAgdW5kZWYgJHg7DQogICAgfQ0KfQ0KcHJpbnQgJHggaWYgbGVuZ3RoKCR4KT4wOw0KYGBgDQo

Ekopalypse · Jan 31, 2020, 3:36 PM

A PS solution might be this

ZnJvbSBOcHAgaW1wb3J0IGVkaXRvcg0KbGluZXMgPSBlZGl0b3IuZ2V0VGV4dCgpLnNwbGl0bGluZXMoKQ0KbWF4X2xpbmVzID0gNSAgIyBjaGFuZ2UgdGhpcyB0byAxMDAwIG9yIHdoYXRldmVyIHZhbHVlIHlvdSB3YW50IHRvIGhhdmUNCnNxbF90ZW1wbGF0ZSA9ICIiIg0Kc2VsZWN0IGZpZWxkMSwgZmllbGQyLCBmaWVsZDMgZnJvbSB0YWJsZTENCiIiIg0KDQpmb3IgaSBpbiByYW5nZSgwLCBsZW4obGluZXMpLCBtYXhfbGluZXMpOg0KICAgIGlmIGkgPT0gMDoNCiAgICAgICAgc3FsX3RlbXBsYXRlICs9ICd3aGVyZSBuYW1lIGluICh7fSlcclxuJy5mb3JtYXQoJywnLmpvaW4oWyIne30nIi5mb3JtYXQoeCkgZm9yIHggaW4gbGluZXNbaTppK21heF9saW5lc11dKSkNCiAgICBlbHNlOg0KICAgICAgICBzcWxfdGVtcGxhdGUgKz0gJ29yIG5hbWUgaW4gKHt9KVxyXG4nLmZvcm1hdCgnLCcuam9pbihbIid7fSciLmZvcm1hdCh4KSBmb3IgeCBpbiBsaW5lc1tpOmkrbWF4X2xpbmVzXV0pKQ0KDQplZGl0b3IuYmVnaW5VbmRvQWN0aW9uKCkNCmVkaXRvci5zZXRUZXh0KHNxbF90ZW1wbGF0ZSkNCmVkaXRvci5lbmRVbmRvQWN0aW9uKCk

It would convert a current document

to

select field1, field2, field3 from table1
where name in ('1','2','3','4','5')
or name in ('6','7','8','9','10')
or name in ('11','12','13','14','15')
or name in ('16','17')

Alan Kilborn · Jan 31, 2020, 3:42 PM

LOL, what’s with all the base64?
Make 'em demonstrate some thinking?
Gotta push the lever to get the reward?

Ekopalypse · Jan 31, 2020, 3:53 PM

@Alan-Kilborn

:-D if it is not of interest then we minimized wasting space :-D

PeterJones · Jan 31, 2020, 4:21 PM

@Alan-Kilborn said in One line per 1000 items in a macro?:

LOL, what’s with all the base64?

This time, primarily for space, and because my Perl solution was technically off-topic for the Forum (*), so didn’t want to clutter for others. Often, I will also use base64 when I think the OP hasn’t shown any effort, but since @sepodele is a first-time poster, and showed effort by pasting the existing macro, that wasn’t the case here.

(*: if I’d had the time to polish it up with my nearly-ready Win32::Mechanize::NotepadPlusPlus Perl module commands, and make it work with the files inside Notepad++, rather than just a command-line Perl script, it would’ve been more on-topic. If I can find time today or this weekend to do that, I will post it here and use it as an example of how to use my Perl module.)

PeterJones · Jan 31, 2020, 9:40 PM

This post is deleted!

PeterJones · Jan 31, 2020, 9:47 PM

@PeterJones said in One line per 1000 items in a macro?:

If I can find time today

For Perl users: install Win32::Mechanize:;NotepadPlusPlus into your Perl instance from the github repo (sorry, not quite ready for official CPAN release). (Note that Perl 32-vs-64bit must match Notepad++ 32-vs-64bit)

Run the following script with your list of lines (doesn’t even have to be a named or saved file) in the active Notepad++ editor window. Results are equivalent to @Ekopalypse’s PythonScript.

use warnings;
use strict;

use Win32::Mechanize::NotepadPlusPlus qw/:main/;

my @lines = split /\R|\0/, editor()->getText();

my $max_lines = 5;  # change this to 1000 or whatever value you want to have
my $sql = "select field1, field2, field3 from table1\n";
my $template = "where name in (%s)\n";

my $i = 0;
while ($i <= $#lines) {
    my $j = $i + $max_lines - 1;
    $j = $#lines if $j > $#lines;

    $sql .= sprintf $template, join ',', map {qq('$_')} @lines[$i .. $j];
    $template = "or name in (%s)\n" if $i==0;
    $i += $max_lines;
}
print STDERR $sql;

editor->beginUndoAction();
editor->setText($sql);
editor->endUndoAction();

(sorry for the deleted post; my first copy still had editor1 (which I was using for debug – so it didn’t matter if I left my script editor as the focus, it would always go to the left editor) rather than editor (which will always use the active editor window)

guy038 · Feb 4, 2020, 2:57 PM

Hello, @sepodele, @peterjones, @ekopalypse and All,

Many thanks, @peterjones and @ekopalypse, for finding a solution to the @sepodele problem, in your area of expertise ;-))

Also, I will try to find a search/replacement in my field, as well !

First, let’s build our list of numbers :

Open a new tab ( Ctrl + N )
Hit the Enter key once
Move back to the very beginning
Open the Replace dialog ( Ctrl + H )
SEARCH (\R)
REPLACE $0$0
Select the Regular expression search mode
Hit, repeatedly, on the Replace All button or use several times the Alt + A shortcut

=> Each time, the number of empty lines created increases by a power of 2. That is to say that, after 10 processes, you should get 1024 empty lines ( = 2^10 )

Move back to the very beginning, if necessary
Open the Column Editor ( Edit > Column Editor... ) or ( Alt + C )
Select the Number to insert option
Type in the value 1 in the 3 fields
Do not tick the Leading zeros option ( IMPORTANT )
Select the Dec format
Click on the OK button

=> You get a list of numbers from 1 to 1025

Now, let’s build the suitable regex S/R to mimic the @peterjones and @ekopalypse solutions. Let’s suppose that you want to separate this list in blocks of 99 elements. Then, use the following regex S/R :

SEARCH ((1)|(\d*99)|\d+)\x20*(\R\Z|\z|(\R))

REPLACE (?2select field1, field2, field3 from table1\5where name in $)'\1'(?5(?3$\5or name in $:,):$\r\n)

Tick the Wrap Around option
Select the Regular expresion search mode
Click on the Replace All button

Et voilà, magic, isn’t it !!!

For people using Unix files, simply delete the \r, at the end of the replace expression

To split the list in blocks of 100 elements use the search regex ((1)|(\d*00)|\d+)\x20*(\R\Z|\z|(\R))

To split the list in blocks of 999 elements use the search regex ((1)|(\d*999)|\d+)\x20*(\R\Z|\z|(\R))

To split the list in blocks of 1,000 elements use the search regex ((1)|(\d*000)|\d+)\x20*(\R\Z|\z|(\R))

To split in blocks of 499 elements, use the search regex ((1)|(\d*499|\d*999)|\d+)\x20*(\R\Z|\z|(\R))

To split in blocks of 500 elements, use the search regex ((1)|(\d*500|\d*000)|\d+)\x20*(\R\Z|\z|(\R))

Whatever the search regex used, the replace regex does not change !

Notes :

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Search ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
((1)|(\d*99)|\d+)\x20*(\R\Z|\z|(\R))


Group 1 = ((1)|(\d*99)|\d+) = CURRENT number, without POSSIBLE trailing SPACE characters

    Group 2 = (1)           = The number '1', ONLY

    Group 3 = (\d*99)       = CURRENT number, with the LAST TWO digits = '9'

    \d+                     = CURRENT number, WITHOUT the LAST TWO digits = '9'

\x20*                       = Possible SPACE characters, after CURRENT number

Group 4 = (\R\Z|\z|(\R))    = Any kind of Line ENDING

    \R\Z                    = Any Line-BREAK at END of the list

    \z                      = The VERY END of the file, after the LAST number

    Group 5 = (\R)          = Any Line-BREAK, NOT at END of the list

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Replacement ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

(?2select field1, field2, field3 from table1\5where name in \()'\1'(?5(?3\)\5or name in \(:,):\)\r\n)


(?2                                                         # IF group 2 ( Number 1 )
select field1, field2, field3 from table1\5where name in \( #     WRITE 'select .... table1' + Line BREAK + 'where name in ('
)                                                           # END group 2

'\1'                                                        # Then, in ALL cases, WRITE a SINGLE quote + CURRENT number + SINGLE quote

(?5                                                         # IF group 5 (Line-BREAK, NOT at END of the list )
(?3                                                         #     IF group 3 ( CURRENT number with the LAST TWO digits are '9' )
\)\5or name in \(                                           #         WRITE ')' + Line BREAK + 'or name in ('
:                                                           #     ELSE
,                                                           #        WRITE ','
)                                                           #     END group 3
:                                                           # ELSE
\)\r\n                                                      #     WRITE ')' + Line BREAK
)                                                           # END group 5

Best Regards,

guy038

sepodele · Feb 4, 2020, 2:57 PM

@guy038 I’m quite overwhelmed by the response to my question but I’d like to try your approach first since it was regex I used to get my macro in the first place.

I realize now that I should have specified my wish better in the original post because your solution is made for only digits. What I want/need is for it to comma separate any value on one line. Here are two examples:

512141
123123
129391

ASDKJASD-222
ASDLKJ 222
ASDLKJASD

Should become:

‘512141’, ‘123123’, ‘129391’
‘ASDKJASD-222’, ‘ASDLKJ 222’, ‘ASDLKJASD’

guy038 · Feb 6, 2020, 8:24 AM

Hello, @sepodele and All,

Sorry for my late reply (Yesterday, I accompanied an instructor from the French Ski School with children from 6-7 years old, all beginners… And… it’s not as easy as it seems ;-)) Most of them seem to be a bit in the moonlight, looking everywhere else… except the monitor ! They are also quickly tired, with many falls without gravity, luckily !)

So, here is my second, more general, attempt, with the following rules :

To begin the select .......... section, place the caret, at least, 2 blank lines above the comments line or above the first element of list(s)
You may add some comment lines to identify the different lists. A comment line will always begins a line with 3 sharp symbols. These possible ###.... comments must be preceded with a blank line and may be followed by any number of blank lines, even none
To begin a new or name in(..., ..., ..., ...) line, simply add, at least, 1 blank line between items of current list
The last element, of the last list, must be followed with a final blank line

For instance, let’s imagine the input text, below :



### Your examples :

512141
123123
129391

ASDKJASD-222
ASDLKJ 222
ASDLKJASD


### A TEST :
This is
just a
small test

### French regions :

Auvergne-Rhône-Alpes
Bourgogne-Franche-Comté
Bretagne
Centre-Val de Loire
Corse
Grand Est
Hauts-de-France
Île-de-France
Normandie
Nouvelle-Aquitaine
Occitanie
Pays de la Loire
Provence-Alpes-Côte d'Azur
Guadeloupe
Martinique
Guyane
La Réunion
Mayotte

### List of numbers

37
38
39
40
41
42
43
44
45
46

47
48
49
50
51

52
53
54
55
56
57


### States of America :

Alabama
Alaska
Arizona
Arkansas
California
Colorado
Connecticut
Delaware
Florida
Georgia
Hawaii
Idaho
Illinois
Indiana
Iowa
Kansas
Kentucky
Louisiana
Maine
Maryland
Massachusetts
Michigan
Minnesota
Mississippi
Missouri
Montana
Nebraska
Nevada
New Hampshire
New Jersey
New Mexico
New York
North Carolina
North Dakota
Ohio
Oklahoma
Oregon
Pennsylvania
Rhode Island
South Carolina
South Dakota
Tennessee
Texas
Utah
Vermont
Virginia
Washington
Washington D.C.
West Virginia
Wisconsin
Wyoming

With the following regex S/R :

SEARCH (?-s)(###.*\R+)|\h*(.+?)\h*\R(\R+)?(\z)?|(^\R){2,}

REPLACE (?1:(?5select field1, field2, field3 from table1\r\nwhere name in $:'\2'(?4$\r\n:(?3\)\r\nor name in \(:,))))

It would result the following output text :

select field1, field2, field3 from table1
where name in ('512141','123123','129391')
or name in ('ASDKJASD-222','ASDLKJ 222','ASDLKJASD')
or name in ('This is','just a','small test')
or name in ('Auvergne-Rhône-Alpes','Bourgogne-Franche-Comté','Bretagne','Centre-Val de Loire','Corse','Grand Est','Hauts-de-France','Île-de-France','Normandie','Nouvelle-Aquitaine','Occitanie','Pays de la Loire','Provence-Alpes-Côte d'Azur','Guadeloupe','Martinique','Guyane','La Réunion','Mayotte')
or name in ('37','38','39','40','41','42','43','44','45','46')
or name in ('47','48','49','50','51')
or name in ('52','53','54','55','56','57')
or name in ('Alabama','Alaska','Arizona','Arkansas','California','Colorado','Connecticut','Delaware','Florida','Georgia','Hawaii','Idaho','Illinois','Indiana','Iowa','Kansas','Kentucky','Louisiana','Maine','Maryland','Massachusetts','Michigan','Minnesota','Mississippi','Missouri','Montana','Nebraska','Nevada','New Hampshire','New Jersey','New Mexico','New York','North Carolina','North Dakota','Ohio','Oklahoma','Oregon','Pennsylvania','Rhode Island','South Carolina','South Dakota','Tennessee','Texas','Utah','Vermont','Virginia','Washington','Washington D.C.','West Virginia','Wisconsin','Wyoming')

I hope my example speaks for itself !

Remark : If you want, in my example, to process, only, the last list of the states of America, simply put the caret 2 blank lines, above that list ( so, right under number 57 ) and run the regex S/R. Of course, in that case, the Wrap around option will be un-ticked, to prevent for processing from beginning of file !

Best Regards,

guy038

sepodele · Feb 7, 2020, 10:33 AM

@guy038 Forgive me. I see I’ve failed to explain my wish accurately again. <insert gif of facepalm here>

When I wrote my two examples I didn’t mean that there would be two different lists in one N++ tab. It was just to illustrate that it would sometimes be numbers and sometimes characters. I still need it to separate into a new line for every 1000 items and not after a blank line.

guy038 · Feb 7, 2020, 8:59 PM

Hi, @sepodele and All,

Ah…, Indeed ! I was a bit annoyed to process your two lists, below, simultaneously, as I needed a way to know where to finish one line or name in(..., ...) and begin a new one and I decided, on my own, to do the change on any blank line

512141
123123
129391

ASDKJASD-222
ASDLKJ 222
ASDLKJASD

Now, I understand that you have, only, 1 list, which contains, generally, over 1,000 items, with various characters and that you certainly do not need any comment line at all ! So, everything is much more simple ;-))

We just have to consider a single list, pasted in a new N++ tab, which does not contain any line-break !

Now, let’s suppose, for instance, that you have the list of states of America ( 51 states ) and that you want to make a break every 10 lines

Open a new N++ tab ( Ctrl + N )
Paste your list in this new tab
Add a final line-break ( IMPORTANT )
SEARCH ^(.+\R){1,10}
REPLACE \r\n$0

You should get the text below, with a line break added before each block of 10 lines maximum :


Alabama
Alaska
Arizona
Arkansas
California
Colorado
Connecticut
Delaware
Florida
Georgia

Hawaii
Idaho
Illinois
Indiana
Iowa
Kansas
Kentucky
Louisiana
Maine
Maryland

Massachusetts
Michigan
Minnesota
Mississippi
Missouri
Montana
Nebraska
Nevada
New Hampshire
New Jersey

New Mexico
New York
North Carolina
North Dakota
Ohio
Oklahoma
Oregon
Pennsylvania
Rhode Island
South Carolina

South Dakota
Tennessee
Texas
Utah
Vermont
Virginia
Washington
Washington D.C.
West Virginia
Wisconsin

Wyoming

Remark : If you don’t want to bother about adding a final line-break, prefer the Peter’s solution, in the next post !

Then using this second regex S/R :

SEARCH (?-s)\h*(.+?)\h*\R(\R+)?(\z)?|(^\R)
REPLACE (?4select field1, field2, field3 from table1\r\nwhere name in $:'\1'(?3$\r\n:(?2\)\r\nor name in \(:,)))

You’ll get your expected text :

select field1, field2, field3 from table1
where name in ('Alabama','Alaska','Arizona','Arkansas','California','Colorado','Connecticut','Delaware','Florida','Georgia')
or name in ('Hawaii','Idaho','Illinois','Indiana','Iowa','Kansas','Kentucky','Louisiana','Maine','Maryland')
or name in ('Massachusetts','Michigan','Minnesota','Mississippi','Missouri','Montana','Nebraska','Nevada','New Hampshire','New Jersey')
or name in ('New Mexico','New York','North Carolina','North Dakota','Ohio','Oklahoma','Oregon','Pennsylvania','Rhode Island','South Carolina')
or name in ('South Dakota','Tennessee','Texas','Utah','Vermont','Virginia','Washington','Washington D.C.','West Virginia','Wisconsin')
or name in ('Wyoming')

IMPORTANT : Of course, in your case, the break must occur after 1,000 lines. Then, simply change the first regex S/R as below :

SEARCH ^(.+\R){1,1000}
REPLACE \r\n$0

Note : You may also decide to make the break after 100 or 500 lines. Just change the first regex, accordingly ;-))

If everything works as you like to, I’ll explain, the regexes, next time !

Cheers,

guy038

PeterJones · Feb 7, 2020, 2:45 PM

@guy038 said in One line per 1000 items in a macro?:

Add a final line-break ( IMPORTANT )

Hmm. You just issued a challenge, because it’s not like you to make one of the critical steps be outside of a regex. :-)

SEARCH ^(.+(\R|(\z))){1,10} (or 1000 for the OP requirements)
REPLACE \r\n$0(?3\r\n)

If the last line in the file doesn’t contain a newline, it will still match for the 1-10 (or 1-1000) line groups; if there is the end-of-file instead of newline, it will add a newline at the end as well; and if the final line is an empty line, it won’t bother putting the extra newline at the end.

guy038 · Feb 7, 2020, 9:00 PM

Hi, @sepodele, @peterjones,

Peter, you’re right about that;-)) In fact, after posting, I wondered how the regex would react if the last item, in the list, did not end with a line-break…And, unfortunately, the last closing parenthesis was missing :-((

So, out of laziness, rather than reworking my second S/R, which, obviously, does not cover all cases, I admit that I opted for the easy way out, simply adding the line, below, to my previous post :

Add a final line-break ( IMPORTANT )

But, Peter, you found a nice and correct solution by modifying the first S/R, which is easier to grasp. Thanks !

Cheers,

guy038

Alan Kilborn · Feb 7, 2020, 9:17 PM

@guy038 said in One line per 1000 items in a macro?:

Add a final line-break ( IMPORTANT )

This is the better solution when dealing with files that you “own”. Don’t beat your head against the wall trying to come up with the perfect regex for the end-of-file oddity…

Editorial: …Especially since Notepad++ has no native way to force adding of a line-ending right before end-of-file…grrrr…

But sometimes it (the regex EOF technique) IS worth thinking about when you have to process files that you either don’t “own” (meaning you can’t change them), or you have to deal with files you receive on a continual basis that maybe don’t end “well”.

So this is a good lead-in to a plugin I discovered recently (well, I knew about it for a long time, but just lately had a need for it): “EditorConfig” It has the ability to make it so that there is always a line-ending at end-of-file when you save (as well as some other nice features – “how many spaces is a tab”, etc.), so, at least for your own files, worrying about the right regex for handling an oddball bottom-of-file situation becomes moot. The plugin allows settings on a “project by project” (or file-by-file) basis, so it is much more encompassing than some of Notepad++'s native configuration-by-extension settings.

I quickly decided I love EditorConfig, for many reasons. OT: I even noticed that Visual Studio respects the configuration files I made for EditorConfig, automatically – nice. Except I noticed this when I was hacking Notepad++ source (which uses tab characters) and since my config files (created for other projects) said “no tabs!”, VS was also inserting spaces when I hit the Tab key.

Bottom line: EditorConfig is a good plugin, and can easily give you the “insert-line-ending-before-end-of-file” capability that Notepad++ doesn’t natively provide – in case you don’t always want to think about what happens when doing regex searches/replacements at end-of-file.

sepodele · Feb 10, 2020, 8:41 AM

Great stuff guys. It works like a charm! Thank you! :-)