newbee, macros and basic algorithm with Python Script

[Alan Kilborn]

I was thinking of taking a deeper look at this, so I did the compare on the second example:

I expected to see something very much like the first example, but it is different enough to make me wonder what the true logic is for this data transformation.

[Neil Schipper]

Hi @mathias-lujan,

I’m pretty sure everything you want done can be accomplished with a series of Find-and-Replace operations. Keep in mind that every F&R is a kind of conditional:

if (text is found matching a condition) then
  replace original text with given replacement text

With Regular Expression F&R, many complex transformations are possible, including reordering of lines.

Your examples suggest a few rules like:

if (artist is present and is not first statement in a [song]) then
  move artist to first statement in [song]

if (genre is absent in a [song]) then
  insert skeleton of genre statement at position 4 in [song]

and so on.

Allow me to make you a believer. Bring up the Replace dialog, choose mode = Regular, leave the box to its right unchecked, and populate these fields:

find: (artist = .*?\R)(name = .*?\R)
repl: $2$1

You will see the next instance (if you issue Replace) or all instances (with Replace All) of consecutive artist & name lines now having the desired order.

Note that I used what may be considered a loose specification: the consecutive lines do not need to be the initial pair of every [song]. More stringent specs are possible but often not needed.

I should mention that a macro could be used for the same transformation in this case: using the same find specification as above, and running it at least once (so the find ‘machinery’ is ‘loaded’), a macro could perform Find Again (F3) to navigate to the next pair, then perform [cursor movement, a select, a cut, more cursor movement, a paste, final cursor movement]. As you know, this can be run once, or N times, or until the end of the file.

Anyway, if you can nail down all pertinent rules, the community will likely be able to provide you with F&R recipes to transform (a safe copy of) your file.

[mpheath]

@Mathias-Lujan , Hello.

Here is PythonScript to replace the text in the current editor pane.

def ini_reindex():

    # Get text from the editor pane.
    text = editor.getText()

    if text == '':
        return ''

    # Presets ordered as the top-most keys.
    presets = ['name', 'artist', 'album', 'genre', 'year', 'track',
               'loading_phrase', 'charter', 'frets', 'song_length']

    # Create a list of indexed lines.
    lines = [[-1, line.replace('\n', '')] for line in text.splitlines()]

    # Add preset keys if not found.
    for item in presets:
        for line in lines:
            if line[1].startswith(item + ' = '):
                break
        else:
            lines.append([-1, item + ' = '])

    # Index the preset keys.
    index = 0

    for item in presets:
        for line in lines:
            if line[0] == -1:
                if line[1].startswith('[') or line[1].startswith(item + ' = '):
                    line[0] = index
                    index += 1

    # Index the remaining keys.
    for line in lines:
        if line[0] == -1:
            line[0] = index
            index += 1

    # Sort by index.
    lines.sort(key=lambda x: x[0])

    # Create text from lines.
    text = ''

    for line in lines:
        text += line[1] + '\n'

    return text


if __name__ == '__main__':
    text = ini_reindex()

    if text != '':
        editor.setText(text)

This is not full automation as requested. Automating the editor for around 5000 files might be unreliable and expect to be a slow option. Opening and closing 5000 files in the editor would be a huge task, automated or manual. So, I did not try for full automation.

The task might be better suited with a standalone Python script. Tested with Python 3.8.2. Similar code to the above code though it iterates *.ini files without the need of the editor.

import glob

# Presets ordered as the top-most keys.
presets = ['name', 'artist', 'album', 'genre', 'year', 'track',
           'loading_phrase', 'charter', 'frets', 'song_length']

# Start processing each ini file.
print('file:')

for file in glob.iglob('*.ini'):
    print(' ', file)

    # Read the ini file and create a list of indexed lines.
    with open(file) as r:
        lines = [[-1, line.replace('\n', '')] for line in r]

    # Add preset keys if not found.
    for item in presets:
        for line in lines:
            if line[1].startswith(item + ' = '):
                break
        else:
            lines.append([-1, item + ' = '])

    # Index the preset keys.
    index = 0

    for item in presets:
        for line in lines:
            if line[0] == -1:
                if line[1].startswith('[') or line[1].startswith(item + ' = '):
                    line[0] = index
                    index += 1

    # Index the remaining keys.
    for line in lines:
        if line[0] == -1:
            line[0] = index
            index += 1

    # Sort by index.
    lines.sort(key=lambda x: x[0])

    # Save as .txt file.
    with open(file[:-4] + '.txt', 'w') as w:
        for line in lines:
            w.write(line[1] + '\n')

print('done')

This will create .txt files with the same name as the .ini files in the current working directory.

Strongly advise testing in a directory with some ini file copies first to ensure it does what you want it to do. This is newly created code so is not hardened by time.

If you prefer, change the line:

with open(file[:-4] + '.txt', 'w') as w:

to

with open(file, 'w') as w:

to overwrite the original file. This is high risk as no backup file will exist. I would zip up the 5000 original files first so a backup does exist, then changing the line should be a lower risk.

if encoding errors happen, then may need to use encoding='utf_8' added as 3rd parameter of open or another encoding it may require.

Greetings to everyone!
Michael.

[Neil Schipper]

@mpheath’s contribution made me realize OP’s request might pertain, not to a single file with many [song] records as I had concluded, but rather to many single-song files.

My interpretation was based on OP only naming one file (but I now see the intent might have been that it be treated as a template), and having a start-of-record element in the samples (and in compsci, the word ‘file’ sometimes refers to an abstract record rather than a named disk file).

So if there are actually many files to process, my suggestion that

… everything you want done can be accomplished with a series of Find-and-Replace operations.

is wrong.

OP did not explicitly say every (disk) file contains exactly one record. If there are files containing multiple song records, both of @mpheath’s solutions would need a bit of work to handle per-record processing.

[Mathias Lujan]

This post is deleted!

[mpheath]

I cannot edit my last post so the correction is done here.

The PythonScript I posted can read the edit pane with \r\n EOLs. The replaced text may have \n EOLs. If \r\n is wanted, then change this section:

    # Create text from lines.
    text = ''

    for line in lines:
        text += line[1] + '\n'

    return text

to

    # Create text from lines.
    text = ''

    for line in lines:
        text += line[1] + '\r\n'

    return text

This will not be a problem with the standalone script as Python will convert \n to \r\n on write.

[Mathias Lujan]

This post is deleted!

[Mathias Lujan]

This post is deleted!

[Mathias Lujan]

This post is deleted!

[Mathias Lujan]

This post is deleted!

[Mathias Lujan]

I’m sorry.
I really have problems with posting and editing in this forum : bots are thinking I produce spam :(

[Mathias Lujan]

I’m very grateful for the attention you give to my problem.
I will try to precise it.

1 - I have over 5000 song’s folder which contains several files (.ogg, .mid, .bpm, .ini…). In each folder, there is an unique file ‘song.ini’. I used to navigate manually in this 5000 folders : it take me a long long time but I have no alternative for now.
In each folder, I open the ‘song.ini’, operate on it launching with a key shortcut a unique macro to save time on basic manipulations (ideally 1 macro or 2 or 3 if there is no way to do an unique macro due to my incapacity to implement conditional tests), check visually the modifications and IF ‘success’ {save/close (with ‘ctrl’ + ‘w’ and ‘enter’ to validate)} ELSE {close file without saving in order to re-operate it manually correctly following}.

2 - I would love to find a solution with a basic macro which I can set up myself with your help. It would be easier for everybody.
find: (artist = .?\R)(name = .?\R)
repl: $2$1

Your 2 lines “open my mind”…
First you make me realize I can open replace window whith ‘ctrl’ + ‘h’ (it could be useful for me in the future).
Second, I discover ‘.*?\R’ and ‘$2$1’. This expressions are perhaps multiple and powerful. Where can I found information on that kind of expressions ?

I think my problem can’t be solve with REPLACE.
I just have to :

IF (find() == true)
	then MOVE a line;
ELSE
	then CREATE a line;

I re-wrote my algorithm to simplify it :

Open : 'song.ini'
Desactivate 'back line auto' in 'display'
Desactivate numeric keypad


Start macro

# move "name ="
If (find("name =") == true) :
	cut ("name =" line with precedent line break) and paste it after ("[song]" line);
Else :
	create a line after ("[song]" line) and write "name = ";

# move "artist ="
If (find("artist =") == true) :
	cut ("artist =" line with precedent line break) and paste it after ("name =" line);
Else :
	create a line after ("name =" line) and write "artist = ";

# move "album ="
If (find("album =") == true) :
	cut ("album =" line with precedent line break) and paste it after ("artist =" line);
Else :
	create a line after ("artist =" line) and write "album = ";


# Do the same routine to move "genre = ", "year = ", "track = ", "loading_phrase = ", "charter = ", "frets = "
# And the last

# move "song_length ="
If (find("song_length =") == true) :
	cut ("song_length =" line with precedent line break) and paste it after ("frets =" line);
Else :
	create a line after ("frets =" line) and write "song_length = ";

# move cursor to the first line to make easier the visual check before close/save or close/non-save
'ctrl' + 'a'; # select all the lines
'left arrow'; # move cursor to the start of 'song.ini'

End macro

Any proposition ?

3 - Looking at @mpheath’s proposition (which inevitably go away of my algorithm). The second proposition without open editor is not “satisfying” me because I can’t check after modifications and cancel them if necessary. Concerning the first proposition (which is surely the way I have to go), I hope you could take mention of my problem’s details upper to amend this code in consequences.
Otherwise, could you explain more the sense of this lines :
3.1 -

lines = [[-1, line.replace('\n', '')] for line in text.splitlines()]

3.2 -

line[1].startswith(item + ' = '):

3.3 -

lines.append([-1, item + ' = '])

3.4 -

if line[0] == -1:
                if line[1].startswith('[') or line[1].startswith(item + ' = '):
                    line[0] = index
                    index += 1

3.5 -

lines.sort(key=lambda x: x[0])

3.6 -

if __name__ == '__main__':
    text = ini_reindex()

    if text != '':
        editor.setText(text)

Best regards and thanks a lot.

[mpheath]

Note: The 2nd script can be made recursive with changing:

for file in glob.iglob('*.ini'):

to

for file in glob.iglob(r'**\song.ini', recursive=True):

This will do song.ini in current and subdirectories.

Now your questions answered:

3.1 -

lines = [[-1, line.replace('\n', '')] for line in text.splitlines()]

Creates a list of lists like [[-1, 'Line 1'], [-1, 'Line 2'], [-1, 'Line 3'], ...]

The -1 will be replaced with numbers to order the lines later for sorting.

3.2 -

```line[1].startswith(item + ' = '):```

item is each of presets list of ['name', 'artist', 'album', 'genre', 'year', 'track', 'loading_phrase', 'charter', 'frets', 'song_length']

So 1st one it trys to match is start with name = . Then tries artist = … until all items are tried. If break does not occur, then the else happens and the item key is added to the list.

3.3 -

```lines.append([-1, item + ' = '])```

lines is a list and .append is a method to add to the list.

Like in 3.2, if the else ocurrs for not finding example name = , then [-1, 'name = '] is added to the list.

3.4 -

if line[0] == -1:
     if line[1].startswith('[') or line[1].startswith(item + ' = '):
          line[0] = index
          index += 1

This is lines with a -1 index, not given a line number yet. If starts with [ (section header) or item + ' = ' assign a line number to line[0], which is replace the -1 with a line number. index is the line number so index += 1 increments the number by adding 1.

3.5 -

```lines.sort(key=lambda x: x[0])```

Sorting the list of lists lines by the first element [0]. lambda is like a nameless function. The sort will look at example [[2, 'Line 2'], [1, 'Line 1']] so it sees the 1 and shifts it into 1st place and then 2 for 2nd place. This gets the list into the order that is wanted.

3.6 -

```if __name__ == '__main__':
       text = ini_reindex()

       if text != '':
           editor.setText(text)
```

__name__ is the name of the running script. If named __main__, run the following code.

ini_reindex() is the function call and the returned value is saved in text. If the text returned is not an empty string, then call editor.setText(text) to replace edit pane contents with the value of text.

[Mathias Lujan]

@mpheath Thanks for the explanations.

I think I’m disturbed by the fact that variables are not declared. For example, I was confused between ‘line’ and ‘lines’.
Moreover, what is ‘line[0]’ ? The ‘line’ number ? The first character of ‘line’ ? The first element of ‘line’ ?

I’m gonna comment your original code with your explanations in order to improve readability and my comprehension too. I will work on it during this next week and I’ll reply as soon as I progress.

Read you soon.

[Alan Kilborn]

@mpheath said in newbee, macros and basic algorithm with Python Script:

The replaced text may have \n EOLs. If \r\n is wanted, then change this section

A smooth way to handle this would be, using your line of code as a basis:

text += line[1] + ['\r\n', '\r', '\n'][editor.getEOLMode()]

This will put the correct line-ending on according to the current tab’s setup.

[Alan Kilborn]

I don’t know that this is an appropriate forum for teaching someone the ins and outs of programming Python for solving a very specific problem.

While I appreciate that there are some tie-ins to editor objects to get and set the active tab’s text, really this is just a straight Python programming problem.

As such, since @mpheath seems interest in helping @Mathias-Lujan with this, can I suggest they get together using the forum’s direct chat feature and expand/explain the effort in that way?

Use this button to do so:

[Mathias Lujan]

@alan-kilborn I think we are near to the solution. Let me just understand and implement it. I don’t want to learn Python programming, I just want solve my problem.

As I said :
" I would love to find a solution with a basic macro which I can set up myself with your help. It would be easier for everybody.
[…]
I think my problem can’t be solve with REPLACE.
I just have to :
IF (find() == true)
then MOVE a line;
ELSE
then CREATE a line; "

So, macro N++ or Python Script to solve my problem ?

[Alan Kilborn]

@mathias-lujan said in newbee, macros and basic algorithm with Python Script:

I don’t want to learn Python programming, I just want solve my problem.

That’s fine, for you. But this forum doesn’t take kindly to those that don’t want to learn. It means someone else has to (choose to) do your work for you. It also probably means that when you need changes, however small, you’ll come here, expecting more free work done on your behalf.

[Mathias Lujan]

@mathias-lujan said in newbee, macros and basic algorithm with Python Script:

I never used Python and it would take me a lot of time to code my basic algorithm. For a regular, this will probably only take an hour or less.
I wrote my basic algorithm (Algorithme rename.txt) and I hope somebody could code it entirely or help me for the Python part.

When I wrote :

I don’t want to learn Python programming, I just want solve my problem.

I would say : I just want to solve my problem with basic Python programming, not to code in general.

Anyway, I feel that most of the work is already done by @mpheath and I just have to implement and test his code. I will try to do that this week.
Thanks again for all.

[Paul Wormer]

@mathias-lujan
Maybe this is mustard after the meal as we say in Dutch, but here is another solution to your problem:

# -*- coding: utf-8 -*-
from __future__ import print_function # Python 2.7
from Npp import *

def moveLine(stringToBeMoved, targetString):    
    try:
        startW, endW  = editor.findText(FINDOPTION.WHOLEWORD, 0, lenFile, stringToBeMoved)
    except:
        line = stringToBeMoved
        #notepad.messageBox(stringToBeMoved + ' not found')
    else: 
        lineNmbr = editor.lineFromPosition(startW)
        line = editor.getLine(lineNmbr).replace('\r\n', '')
        editor.deleteLine(lineNmbr)
    
    try:
        startW, endW  = editor.findText(FINDOPTION.WHOLEWORD, 0, lenFile,  targetString )
    except:
        notepad.messageBox(targetString + ' not found') 
        return  
    else:
        endLine = editor.getLineEndPosition(editor.lineFromPosition(endW)) 
        editor.gotoPos(endLine)
        editor.addText('\r\n'+line)
	
items = [
    '[song]',
    'name =',
    'artist =',
    'album =',
    'genre =',
    'year =',
    'track =',
    'loading_phrase =',
    'charter =',
    'frets =',
    'song_length =',
    'delay =',
    'diff_guitar =',
    'diff_bass =',
    'diff_guitar_coop =',
    'diff_rhythm =',
    'diff_vocals =',
    'diff_keys =',
    'diff_bass_real =',
    'diff_guitar_real =',
    'diff_dance =',
    'diff_bass_real_22 =',
    'diff_guitar_real_22 =',
    'diff_vocals_harm =',
    'sysex_high_hat_ctrl =',
    'multiplier_note =',
    'pro_drums =',
    'background =',
    'last_play ='
    ]
    
for i in range(0, len(items)-1): 
    stringToBeMoved = items[i+1]
    targetString = items[i]
    moveLine(stringToBeMoved, targetString)