newbee, macros and basic algorithm with Python Script

Mathias Lujan

Hello

I’m sorry for my bad english.

I used to use perso Notepad++ macros : I just record what I do using keyboard, keyboard shortcuts and the mouse only to clic on ‘close (cross) button’ of find window. It often helps me to save a lot of time in order to modify semi automatically a lot of files ‘song.ini’ (arround 5000).

Unfortunatly, I’m stuck on the impossibility of using basing conditional tests. I understood that PythonScript plugin in Notepad++ allow this conditionnal tests (and far more).
I never used Python and it would take me a lot of time to code my basic algorithm. For a regular, this will probably only take an hour or less.

I wrote my basic algorithm (Algorithme rename.txt) and I hope somebody could code it entirely or help me for the Python part. I have already record a macro (Algorithme rename shortcuts.xml) which is likely what I want without conditionnal tests. I have also two example files (exemple1 song.ini and exemple2 song.ini).

Regards

Algorithme rename :

One tab open in notepad++ : 'buffer.txt'
Desactivate 'back line auto' in 'display'
Desactivate numeric keypad
Open a second tab : 'song.ini'


Start macro

'ctrl' + 'f'; ### open 'find window'
'n' 'a' 'm' 'e' ' ' '=' 'enter'; ### find("name =")

If (find("name =") == true) :
	clic on 'close (cross) button'; ### close find window
	'end' ### go to line's end with key '1' of numeric keypad 
	'shift' + 'home'; ### select all the line with key '7' of numeric keypad...
	'shift' + 'left arrow'; ### ... and select the precedent line break
	'ctrl' + 'x'; ### cut selection
	'ctrl' + 'pg up'; ### go to precedent tab with key '9' of numeric keypad
	'ctrl' + 'v'; ### paste selection
	'ctrl' + 'pg dn'; ### go to next tab with key '3' of numeric keypad
Else :
	clic on 'close (cross) button'; ### close find window
	'ctrl' + 'pg up'; ### go to precedent tab with key '9' of numeric keypad
	'enter' 'n' 'a' 'm' 'e' ' ' '=' ' '; ### create a newline and write "name = "
	'ctrl' + 'pg dn'; ### go to next tab with key '3' of numeric keypad


'ctrl' + 'f'; ### open 'find window'
'a' 'r' 't' 'i' 's' 't' ' ' '=' 'enter'; ### find("artist =")

If (find("artist =") == true) :
	clic on 'close (cross) button'; ### close find window
	'end' ### go to line's end with key '1' of numeric keypad 
	'shift' + 'home'; ### select all the line with key '7' of numeric keypad...
	'shift' + 'left arrow'; ### ... and select the precedent line break
	'ctrl' + 'x'; ### cut selection
	'ctrl' + 'pg up'; ### go to precedent tab with key '9' of numeric keypad
	'ctrl' + 'v'; ### paste selection
	'ctrl' + 'pg dn'; ### go to next tab with key '3' of numeric keypad
Else :
	clic on 'close (cross) button'; ### close find window
	'ctrl' + 'pg up'; ### go to precedent tab with key '9' of numeric keypad
	'enter' 'a' 'r' 't' 'i' 's' 't' ' ' '=' ' '; ### create a newline and write "artist = " 
	'ctrl' + 'pg dn'; ### go to next tab with key '3' of numeric keypad


'ctrl' + 'f'; ### open 'find window'
'a' 'l' 'b' 'u' 'm' ' ' '=' 'enter'; ### find("album =")

If (find("album =") == true) :
	clic on 'close (cross) button'; ### close find window
	'end' ### go to line's end with key '1' of numeric keypad 
	'shift' + 'home'; ### select all the line with key '7' of numeric keypad...
	'shift' + 'left arrow'; ### ... and select the precedent line break
	'ctrl' + 'x'; ### cut selection
	'ctrl' + 'pg up'; ### go to precedent tab with key '9' of numeric keypad
	'ctrl' + 'v'; ### paste selection
	'ctrl' + 'pg dn'; ### go to next tab with key '3' of numeric keypad
Else :
	clic on 'close (cross) button'; ### close find window
	'ctrl' + 'pg up'; ### go to precedent tab with key '9' of numeric keypad
	'enter' 'a' 'l' 'b' 'u' 'm' ' ' '='  ' '; ### create a newline and write "album = " 
	'ctrl' + 'pg dn'; ### go to next tab with key '3' of numeric keypad

### Do the same routine for 'genre = ', 'year = ', 'track = ', 'loading_phrase = ', 'charter = ', 'frets = '
### And the last

'ctrl' + 'f'; ### open 'find window'
's' 'o' 'n' 'g' '_' 'l' 'e' 'n' 'g' 't' 'h' ' ' '=' 'enter'; ### find("song_length =")

If (find("song_length =") == true) :
	clic on 'close (cross) button'; ### close find window
	'end' ### go to line's end with key '1' of numeric keypad 
	'shift' + 'home'; ### select all the line with key '7' of numeric keypad...
	'shift' + 'left arrow'; ### ... and select the precedent line break
	'ctrl' + 'pg up'; ### go to precedent tab with key '9' of numeric keypad
	'ctrl' + 'v'; ### paste selection
Else :
	clic on 'close (cross) button'; ### close find window
	'ctrl' + 'pg up'; ### go to precedent tab with key '9' of numeric keypad
	'enter' 's' 'o' 'n' 'g' '_' 'l' 'e' 'n' 'g' 't' 'h' ' ' '='  ' '; ### create a newline and write "song_length = "

'ctrl' + 'a'; ### select all the lines
'left arrow'; ### go to start of 'buffer.txt'
'delete'; ### delete the empty first line
'ctrl' + 'a'; ### select all the lines
'ctrl' + 'x'; ### cut selection
'ctrl' + 'pg dn'; ### go to next tab with key '3' of numeric keypad
'ctrl' + 'a'; ### select all the lines
'left arrow'; ### go to start of 'song.ini'
'end' ### go to first line's end with key '1' of numeric keypad
'enter'; ### create a newline
'ctrl' + 'v'; ### paste selection with key '3' of numeric keypad
'ctrl' + 'a'; ### select all the lines
'left arrow'; ### go to start of 'song.ini'

End macro

Neil Schipper

@mathias-lujan Hi. If someone comes along and codes the script, they would have no way to test it. I strongly suggest you provide some sample input (at least a few records), and, the output you wish to end up with (in both cases, in text boxes as you did above). It might turn out that a custom script is not needed.

Mathias Lujan

@neil-schipper Very goog idea…

Exemple1 before modification :

[song]
artist = Erasure
name = 04B.103.8e Always (2009 Remix)
album = Pop! Remixed
delay = 3050
diff_guitar = -1
diff_bass = -1
diff_guitar_coop = -1
diff_rhythm = -1
diff_vocals = -1
diff_keys = -1
diff_bass_real = -1
diff_guitar_real = -1
diff_dance = -1
diff_bass_real_22 = -1
diff_guitar_real_22 = -1
diff_vocals_harm = -1
sysex_high_hat_ctrl = True
multiplier_note = 116
pro_drums = True
charter = BobSchneeder45
background = imageWS.jpg
last_play = 2022-01-16-22:36:05
song_length = 245282

Exemple1 after modification :

[song]
name = 04B.103.8e Always (2009 Remix)
artist = Erasure
album = Pop! Remixed
genre = 
year = 
track = 
loading_phrase = 
charter = BobSchneeder45
frets = 
song_length = 245282
delay = 3050
diff_guitar = -1
diff_bass = -1
diff_guitar_coop = -1
diff_rhythm = -1
diff_vocals = -1
diff_keys = -1
diff_bass_real = -1
diff_guitar_real = -1
diff_dance = -1
diff_bass_real_22 = -1
diff_guitar_real_22 = -1
diff_vocals_harm = -1
sysex_high_hat_ctrl = True
multiplier_note = 116
pro_drums = True
background = imageWS.jpg
last_play = 2022-01-16-22:36:05

Exemple2 before modification :

[song]
artist = The Clash
name = Jimmy Jazz
genre = Punk
album = London Calling
year = 1979
icon = rb3dlc
charter = Harmonix
pro_drums = True
five_lane_drums = False
sysex_slider = False
sysex_high_hat_ctrl = True
sysex_rimshot = True
sysex_open_bass = False
diff_band = 4
diff_guitar = 3
diff_vocals = 2
diff_drums = 4
diff_bass = 2
diff_keys = 3
diff_guitar_real = -1
diff_vocals_harm = -1
diff_drums_real = 4
diff_bass_real = -1
diff_keys_real = .3
diff_dance = -1
diff_guitar_coop = -1
diff_rhythm = -1
diff_bass_real_22 = -1
diff_guitar_real_22 = -1
loading_phrase = 
banner_link_a = http://www.rockband.com/
link_name_a = Homepage
banner_link_b = 
link_name_b = 
video = 
video_start_time = -1
preview_start_time = 53000
diff_drums_real_ps = 4
diff_keys_real_ps = .3
delay = 0

Exemple1 after modification :

[song]
name = Jimmy Jazz
artist = The Clash
album = London Calling
genre = Punk
year = 1979
track = 
loading_phrase = 
charter = Harmonix
frets = 
song_length = 
icon = rb3dlc
pro_drums = True
five_lane_drums = False
sysex_slider = False
sysex_high_hat_ctrl = True
sysex_rimshot = True
sysex_open_bass = False
diff_band = 4
diff_guitar = 3
diff_vocals = 2
diff_drums = 4
diff_bass = 2
diff_keys = 3
diff_guitar_real = -1
diff_vocals_harm = -1
diff_drums_real = 4
diff_bass_real = -1
diff_keys_real = .3
diff_dance = -1
diff_guitar_coop = -1
diff_rhythm = -1
diff_bass_real_22 = -1
diff_guitar_real_22 = -1
banner_link_a = http://www.rockband.com/
link_name_a = Homepage
banner_link_b = 
link_name_b = 
video = 
video_start_time = -1
preview_start_time = 53000
diff_drums_real_ps = 4
diff_keys_real_ps = .3
delay = 0

Alan Kilborn

Looking at the first example, it’s really just a reordering of data with some extra “fields” (lines) thrown in:

Mathias Lujan

@alan-kilborn Yes, it’s a basic algorithm with only ‘if/else’ conditionnal tests…
But, as I said, I don’t know how to do ‘if/else’ in macros.

Alan Kilborn

@mathias-lujan said in newbee, macros and basic algorithm with Python Script:

I don’t know how to do ‘if/else’ in macros.

Well…the whole thing is a terrible job for macros…

Alan Kilborn

I was thinking of taking a deeper look at this, so I did the compare on the second example:

I expected to see something very much like the first example, but it is different enough to make me wonder what the true logic is for this data transformation.

Neil Schipper

Hi @mathias-lujan,

I’m pretty sure everything you want done can be accomplished with a series of Find-and-Replace operations. Keep in mind that every F&R is a kind of conditional:

if (text is found matching a condition) then
  replace original text with given replacement text

With Regular Expression F&R, many complex transformations are possible, including reordering of lines.

Your examples suggest a few rules like:

if (artist is present and is not first statement in a [song]) then
  move artist to first statement in [song]

if (genre is absent in a [song]) then
  insert skeleton of genre statement at position 4 in [song]

and so on.

Allow me to make you a believer. Bring up the Replace dialog, choose mode = Regular, leave the box to its right unchecked, and populate these fields:

find: (artist = .*?\R)(name = .*?\R)
repl: $2$1

You will see the next instance (if you issue Replace) or all instances (with Replace All) of consecutive artist & name lines now having the desired order.

Note that I used what may be considered a loose specification: the consecutive lines do not need to be the initial pair of every [song]. More stringent specs are possible but often not needed.

I should mention that a macro could be used for the same transformation in this case: using the same find specification as above, and running it at least once (so the find ‘machinery’ is ‘loaded’), a macro could perform Find Again (F3) to navigate to the next pair, then perform [cursor movement, a select, a cut, more cursor movement, a paste, final cursor movement]. As you know, this can be run once, or N times, or until the end of the file.

Anyway, if you can nail down all pertinent rules, the community will likely be able to provide you with F&R recipes to transform (a safe copy of) your file.

mpheath

@Mathias-Lujan , Hello.

Here is PythonScript to replace the text in the current editor pane.

def ini_reindex():

    # Get text from the editor pane.
    text = editor.getText()

    if text == '':
        return ''

    # Presets ordered as the top-most keys.
    presets = ['name', 'artist', 'album', 'genre', 'year', 'track',
               'loading_phrase', 'charter', 'frets', 'song_length']

    # Create a list of indexed lines.
    lines = [[-1, line.replace('\n', '')] for line in text.splitlines()]

    # Add preset keys if not found.
    for item in presets:
        for line in lines:
            if line[1].startswith(item + ' = '):
                break
        else:
            lines.append([-1, item + ' = '])

    # Index the preset keys.
    index = 0

    for item in presets:
        for line in lines:
            if line[0] == -1:
                if line[1].startswith('[') or line[1].startswith(item + ' = '):
                    line[0] = index
                    index += 1

    # Index the remaining keys.
    for line in lines:
        if line[0] == -1:
            line[0] = index
            index += 1

    # Sort by index.
    lines.sort(key=lambda x: x[0])

    # Create text from lines.
    text = ''

    for line in lines:
        text += line[1] + '\n'

    return text


if __name__ == '__main__':
    text = ini_reindex()

    if text != '':
        editor.setText(text)

This is not full automation as requested. Automating the editor for around 5000 files might be unreliable and expect to be a slow option. Opening and closing 5000 files in the editor would be a huge task, automated or manual. So, I did not try for full automation.

The task might be better suited with a standalone Python script. Tested with Python 3.8.2. Similar code to the above code though it iterates *.ini files without the need of the editor.

import glob

# Presets ordered as the top-most keys.
presets = ['name', 'artist', 'album', 'genre', 'year', 'track',
           'loading_phrase', 'charter', 'frets', 'song_length']

# Start processing each ini file.
print('file:')

for file in glob.iglob('*.ini'):
    print(' ', file)

    # Read the ini file and create a list of indexed lines.
    with open(file) as r:
        lines = [[-1, line.replace('\n', '')] for line in r]

    # Add preset keys if not found.
    for item in presets:
        for line in lines:
            if line[1].startswith(item + ' = '):
                break
        else:
            lines.append([-1, item + ' = '])

    # Index the preset keys.
    index = 0

    for item in presets:
        for line in lines:
            if line[0] == -1:
                if line[1].startswith('[') or line[1].startswith(item + ' = '):
                    line[0] = index
                    index += 1

    # Index the remaining keys.
    for line in lines:
        if line[0] == -1:
            line[0] = index
            index += 1

    # Sort by index.
    lines.sort(key=lambda x: x[0])

    # Save as .txt file.
    with open(file[:-4] + '.txt', 'w') as w:
        for line in lines:
            w.write(line[1] + '\n')

print('done')

This will create .txt files with the same name as the .ini files in the current working directory.

Strongly advise testing in a directory with some ini file copies first to ensure it does what you want it to do. This is newly created code so is not hardened by time.

If you prefer, change the line:

with open(file[:-4] + '.txt', 'w') as w:

to

with open(file, 'w') as w:

to overwrite the original file. This is high risk as no backup file will exist. I would zip up the 5000 original files first so a backup does exist, then changing the line should be a lower risk.

if encoding errors happen, then may need to use encoding='utf_8' added as 3rd parameter of open or another encoding it may require.

Greetings to everyone!
Michael.

Neil Schipper

@mpheath’s contribution made me realize OP’s request might pertain, not to a single file with many [song] records as I had concluded, but rather to many single-song files.

My interpretation was based on OP only naming one file (but I now see the intent might have been that it be treated as a template), and having a start-of-record element in the samples (and in compsci, the word ‘file’ sometimes refers to an abstract record rather than a named disk file).

So if there are actually many files to process, my suggestion that

… everything you want done can be accomplished with a series of Find-and-Replace operations.

is wrong.

OP did not explicitly say every (disk) file contains exactly one record. If there are files containing multiple song records, both of @mpheath’s solutions would need a bit of work to handle per-record processing.

Mathias Lujan

This post is deleted!

mpheath

I cannot edit my last post so the correction is done here.

The PythonScript I posted can read the edit pane with \r\n EOLs. The replaced text may have \n EOLs. If \r\n is wanted, then change this section:

    # Create text from lines.
    text = ''

    for line in lines:
        text += line[1] + '\n'

    return text

to

    # Create text from lines.
    text = ''

    for line in lines:
        text += line[1] + '\r\n'

    return text

This will not be a problem with the standalone script as Python will convert \n to \r\n on write.

Mathias Lujan

This post is deleted!

Mathias Lujan

This post is deleted!

Mathias Lujan

This post is deleted!

Mathias Lujan

This post is deleted!

Mathias Lujan

I’m sorry.
I really have problems with posting and editing in this forum : bots are thinking I produce spam :(

Mathias Lujan

I’m very grateful for the attention you give to my problem.
I will try to precise it.

1 - I have over 5000 song’s folder which contains several files (.ogg, .mid, .bpm, .ini…). In each folder, there is an unique file ‘song.ini’. I used to navigate manually in this 5000 folders : it take me a long long time but I have no alternative for now.
In each folder, I open the ‘song.ini’, operate on it launching with a key shortcut a unique macro to save time on basic manipulations (ideally 1 macro or 2 or 3 if there is no way to do an unique macro due to my incapacity to implement conditional tests), check visually the modifications and IF ‘success’ {save/close (with ‘ctrl’ + ‘w’ and ‘enter’ to validate)} ELSE {close file without saving in order to re-operate it manually correctly following}.

2 - I would love to find a solution with a basic macro which I can set up myself with your help. It would be easier for everybody.
find: (artist = .?\R)(name = .?\R)
repl: $2$1

Your 2 lines “open my mind”…
First you make me realize I can open replace window whith ‘ctrl’ + ‘h’ (it could be useful for me in the future).
Second, I discover ‘.*?\R’ and ‘$2$1’. This expressions are perhaps multiple and powerful. Where can I found information on that kind of expressions ?

I think my problem can’t be solve with REPLACE.
I just have to :

IF (find() == true)
	then MOVE a line;
ELSE
	then CREATE a line;

I re-wrote my algorithm to simplify it :


Open : 'song.ini'
Desactivate 'back line auto' in 'display'
Desactivate numeric keypad


Start macro

# move "name ="
If (find("name =") == true) :
	cut ("name =" line with precedent line break) and paste it after ("[song]" line);
Else :
	create a line after ("[song]" line) and write "name = ";

# move "artist ="
If (find("artist =") == true) :
	cut ("artist =" line with precedent line break) and paste it after ("name =" line);
Else :
	create a line after ("name =" line) and write "artist = ";

# move "album ="
If (find("album =") == true) :
	cut ("album =" line with precedent line break) and paste it after ("artist =" line);
Else :
	create a line after ("artist =" line) and write "album = ";


# Do the same routine to move "genre = ", "year = ", "track = ", "loading_phrase = ", "charter = ", "frets = "
# And the last

# move "song_length ="
If (find("song_length =") == true) :
	cut ("song_length =" line with precedent line break) and paste it after ("frets =" line);
Else :
	create a line after ("frets =" line) and write "song_length = ";

# move cursor to the first line to make easier the visual check before close/save or close/non-save
'ctrl' + 'a'; # select all the lines
'left arrow'; # move cursor to the start of 'song.ini'

End macro

Any proposition ?

3 - Looking at @mpheath’s proposition (which inevitably go away of my algorithm). The second proposition without open editor is not “satisfying” me because I can’t check after modifications and cancel them if necessary. Concerning the first proposition (which is surely the way I have to go), I hope you could take mention of my problem’s details upper to amend this code in consequences.
Otherwise, could you explain more the sense of this lines :
3.1 -

lines = [[-1, line.replace('\n', '')] for line in text.splitlines()]

3.2 -

line[1].startswith(item + ' = '):

3.3 -

lines.append([-1, item + ' = '])

3.4 -

if line[0] == -1:
                if line[1].startswith('[') or line[1].startswith(item + ' = '):
                    line[0] = index
                    index += 1

3.5 -

lines.sort(key=lambda x: x[0])

3.6 -

if __name__ == '__main__':
    text = ini_reindex()

    if text != '':
        editor.setText(text)

Best regards and thanks a lot.

mpheath

Note: The 2nd script can be made recursive with changing:

for file in glob.iglob('*.ini'):

to

for file in glob.iglob(r'**\song.ini', recursive=True):

This will do song.ini in current and subdirectories.

Now your questions answered:

3.1 -

lines = [[-1, line.replace('\n', '')] for line in text.splitlines()]

Creates a list of lists like [[-1, 'Line 1'], [-1, 'Line 2'], [-1, 'Line 3'], ...]

The -1 will be replaced with numbers to order the lines later for sorting.

3.2 -

```line[1].startswith(item + ' = '):```

item is each of presets list of ['name', 'artist', 'album', 'genre', 'year', 'track', 'loading_phrase', 'charter', 'frets', 'song_length']

So 1st one it trys to match is start with name = . Then tries artist = … until all items are tried. If break does not occur, then the else happens and the item key is added to the list.

3.3 -

```lines.append([-1, item + ' = '])```

lines is a list and .append is a method to add to the list.

Like in 3.2, if the else ocurrs for not finding example name = , then [-1, 'name = '] is added to the list.

3.4 -

if line[0] == -1:
     if line[1].startswith('[') or line[1].startswith(item + ' = '):
          line[0] = index
          index += 1

This is lines with a -1 index, not given a line number yet. If starts with [ (section header) or item + ' = ' assign a line number to line[0], which is replace the -1 with a line number. index is the line number so index += 1 increments the number by adding 1.

3.5 -

```lines.sort(key=lambda x: x[0])```

Sorting the list of lists lines by the first element [0]. lambda is like a nameless function. The sort will look at example [[2, 'Line 2'], [1, 'Line 1']] so it sees the 1 and shifts it into 1st place and then 2 for 2nd place. This gets the list into the order that is wanted.

3.6 -

```if __name__ == '__main__':
       text = ini_reindex()

       if text != '':
           editor.setText(text)
```

__name__ is the name of the running script. If named __main__, run the following code.

ini_reindex() is the function call and the returned value is saved in text. If the text returned is not an empty string, then call editor.setText(text) to replace edit pane contents with the value of text.

Mathias Lujan

@mpheath Thanks for the explanations.

I think I’m disturbed by the fact that variables are not declared. For example, I was confused between ‘line’ and ‘lines’.
Moreover, what is ‘line[0]’ ? The ‘line’ number ? The first character of ‘line’ ? The first element of ‘line’ ?

I’m gonna comment your original code with your explanations in order to improve readability and my comprehension too. I will work on it during this next week and I’ll reply as soon as I progress.

Read you soon.