newbee, macros and basic algorithm with Python Script

[Mathias Lujan]

I’m sorry.
I really have problems with posting and editing in this forum : bots are thinking I produce spam :(

[Mathias Lujan]

I’m very grateful for the attention you give to my problem.
I will try to precise it.

1 - I have over 5000 song’s folder which contains several files (.ogg, .mid, .bpm, .ini…). In each folder, there is an unique file ‘song.ini’. I used to navigate manually in this 5000 folders : it take me a long long time but I have no alternative for now.
In each folder, I open the ‘song.ini’, operate on it launching with a key shortcut a unique macro to save time on basic manipulations (ideally 1 macro or 2 or 3 if there is no way to do an unique macro due to my incapacity to implement conditional tests), check visually the modifications and IF ‘success’ {save/close (with ‘ctrl’ + ‘w’ and ‘enter’ to validate)} ELSE {close file without saving in order to re-operate it manually correctly following}.

2 - I would love to find a solution with a basic macro which I can set up myself with your help. It would be easier for everybody.
find: (artist = .?\R)(name = .?\R)
repl: $2$1

Your 2 lines “open my mind”…
First you make me realize I can open replace window whith ‘ctrl’ + ‘h’ (it could be useful for me in the future).
Second, I discover ‘.*?\R’ and ‘$2$1’. This expressions are perhaps multiple and powerful. Where can I found information on that kind of expressions ?

I think my problem can’t be solve with REPLACE.
I just have to :

IF (find() == true)
	then MOVE a line;
ELSE
	then CREATE a line;

I re-wrote my algorithm to simplify it :

Open : 'song.ini'
Desactivate 'back line auto' in 'display'
Desactivate numeric keypad


Start macro

# move "name ="
If (find("name =") == true) :
	cut ("name =" line with precedent line break) and paste it after ("[song]" line);
Else :
	create a line after ("[song]" line) and write "name = ";

# move "artist ="
If (find("artist =") == true) :
	cut ("artist =" line with precedent line break) and paste it after ("name =" line);
Else :
	create a line after ("name =" line) and write "artist = ";

# move "album ="
If (find("album =") == true) :
	cut ("album =" line with precedent line break) and paste it after ("artist =" line);
Else :
	create a line after ("artist =" line) and write "album = ";


# Do the same routine to move "genre = ", "year = ", "track = ", "loading_phrase = ", "charter = ", "frets = "
# And the last

# move "song_length ="
If (find("song_length =") == true) :
	cut ("song_length =" line with precedent line break) and paste it after ("frets =" line);
Else :
	create a line after ("frets =" line) and write "song_length = ";

# move cursor to the first line to make easier the visual check before close/save or close/non-save
'ctrl' + 'a'; # select all the lines
'left arrow'; # move cursor to the start of 'song.ini'

End macro

Any proposition ?

3 - Looking at @mpheath’s proposition (which inevitably go away of my algorithm). The second proposition without open editor is not “satisfying” me because I can’t check after modifications and cancel them if necessary. Concerning the first proposition (which is surely the way I have to go), I hope you could take mention of my problem’s details upper to amend this code in consequences.
Otherwise, could you explain more the sense of this lines :
3.1 -

lines = [[-1, line.replace('\n', '')] for line in text.splitlines()]

3.2 -

line[1].startswith(item + ' = '):

3.3 -

lines.append([-1, item + ' = '])

3.4 -

if line[0] == -1:
                if line[1].startswith('[') or line[1].startswith(item + ' = '):
                    line[0] = index
                    index += 1

3.5 -

lines.sort(key=lambda x: x[0])

3.6 -

if __name__ == '__main__':
    text = ini_reindex()

    if text != '':
        editor.setText(text)

Best regards and thanks a lot.

[mpheath]

Note: The 2nd script can be made recursive with changing:

for file in glob.iglob('*.ini'):

to

for file in glob.iglob(r'**\song.ini', recursive=True):

This will do song.ini in current and subdirectories.

Now your questions answered:

3.1 -

lines = [[-1, line.replace('\n', '')] for line in text.splitlines()]

Creates a list of lists like [[-1, 'Line 1'], [-1, 'Line 2'], [-1, 'Line 3'], ...]

The -1 will be replaced with numbers to order the lines later for sorting.

3.2 -

```line[1].startswith(item + ' = '):```

item is each of presets list of ['name', 'artist', 'album', 'genre', 'year', 'track', 'loading_phrase', 'charter', 'frets', 'song_length']

So 1st one it trys to match is start with name = . Then tries artist = … until all items are tried. If break does not occur, then the else happens and the item key is added to the list.

3.3 -

```lines.append([-1, item + ' = '])```

lines is a list and .append is a method to add to the list.

Like in 3.2, if the else ocurrs for not finding example name = , then [-1, 'name = '] is added to the list.

3.4 -

if line[0] == -1:
     if line[1].startswith('[') or line[1].startswith(item + ' = '):
          line[0] = index
          index += 1

This is lines with a -1 index, not given a line number yet. If starts with [ (section header) or item + ' = ' assign a line number to line[0], which is replace the -1 with a line number. index is the line number so index += 1 increments the number by adding 1.

3.5 -

```lines.sort(key=lambda x: x[0])```

Sorting the list of lists lines by the first element [0]. lambda is like a nameless function. The sort will look at example [[2, 'Line 2'], [1, 'Line 1']] so it sees the 1 and shifts it into 1st place and then 2 for 2nd place. This gets the list into the order that is wanted.

3.6 -

```if __name__ == '__main__':
       text = ini_reindex()

       if text != '':
           editor.setText(text)
```

__name__ is the name of the running script. If named __main__, run the following code.

ini_reindex() is the function call and the returned value is saved in text. If the text returned is not an empty string, then call editor.setText(text) to replace edit pane contents with the value of text.

[Mathias Lujan]

@mpheath Thanks for the explanations.

I think I’m disturbed by the fact that variables are not declared. For example, I was confused between ‘line’ and ‘lines’.
Moreover, what is ‘line[0]’ ? The ‘line’ number ? The first character of ‘line’ ? The first element of ‘line’ ?

I’m gonna comment your original code with your explanations in order to improve readability and my comprehension too. I will work on it during this next week and I’ll reply as soon as I progress.

Read you soon.

[Alan Kilborn]

@mpheath said in newbee, macros and basic algorithm with Python Script:

The replaced text may have \n EOLs. If \r\n is wanted, then change this section

A smooth way to handle this would be, using your line of code as a basis:

text += line[1] + ['\r\n', '\r', '\n'][editor.getEOLMode()]

This will put the correct line-ending on according to the current tab’s setup.

[Alan Kilborn]

I don’t know that this is an appropriate forum for teaching someone the ins and outs of programming Python for solving a very specific problem.

While I appreciate that there are some tie-ins to editor objects to get and set the active tab’s text, really this is just a straight Python programming problem.

As such, since @mpheath seems interest in helping @Mathias-Lujan with this, can I suggest they get together using the forum’s direct chat feature and expand/explain the effort in that way?

Use this button to do so:

[Mathias Lujan]

@alan-kilborn I think we are near to the solution. Let me just understand and implement it. I don’t want to learn Python programming, I just want solve my problem.

As I said :
" I would love to find a solution with a basic macro which I can set up myself with your help. It would be easier for everybody.
[…]
I think my problem can’t be solve with REPLACE.
I just have to :
IF (find() == true)
then MOVE a line;
ELSE
then CREATE a line; "

So, macro N++ or Python Script to solve my problem ?

[Alan Kilborn]

@mathias-lujan said in newbee, macros and basic algorithm with Python Script:

I don’t want to learn Python programming, I just want solve my problem.

That’s fine, for you. But this forum doesn’t take kindly to those that don’t want to learn. It means someone else has to (choose to) do your work for you. It also probably means that when you need changes, however small, you’ll come here, expecting more free work done on your behalf.

[Mathias Lujan]

@mathias-lujan said in newbee, macros and basic algorithm with Python Script:

I never used Python and it would take me a lot of time to code my basic algorithm. For a regular, this will probably only take an hour or less.
I wrote my basic algorithm (Algorithme rename.txt) and I hope somebody could code it entirely or help me for the Python part.

When I wrote :

I don’t want to learn Python programming, I just want solve my problem.

I would say : I just want to solve my problem with basic Python programming, not to code in general.

Anyway, I feel that most of the work is already done by @mpheath and I just have to implement and test his code. I will try to do that this week.
Thanks again for all.

[Paul Wormer]

@mathias-lujan
Maybe this is mustard after the meal as we say in Dutch, but here is another solution to your problem:

# -*- coding: utf-8 -*-
from __future__ import print_function # Python 2.7
from Npp import *

def moveLine(stringToBeMoved, targetString):    
    try:
        startW, endW  = editor.findText(FINDOPTION.WHOLEWORD, 0, lenFile, stringToBeMoved)
    except:
        line = stringToBeMoved
        #notepad.messageBox(stringToBeMoved + ' not found')
    else: 
        lineNmbr = editor.lineFromPosition(startW)
        line = editor.getLine(lineNmbr).replace('\r\n', '')
        editor.deleteLine(lineNmbr)
    
    try:
        startW, endW  = editor.findText(FINDOPTION.WHOLEWORD, 0, lenFile,  targetString )
    except:
        notepad.messageBox(targetString + ' not found') 
        return  
    else:
        endLine = editor.getLineEndPosition(editor.lineFromPosition(endW)) 
        editor.gotoPos(endLine)
        editor.addText('\r\n'+line)
	
items = [
    '[song]',
    'name =',
    'artist =',
    'album =',
    'genre =',
    'year =',
    'track =',
    'loading_phrase =',
    'charter =',
    'frets =',
    'song_length =',
    'delay =',
    'diff_guitar =',
    'diff_bass =',
    'diff_guitar_coop =',
    'diff_rhythm =',
    'diff_vocals =',
    'diff_keys =',
    'diff_bass_real =',
    'diff_guitar_real =',
    'diff_dance =',
    'diff_bass_real_22 =',
    'diff_guitar_real_22 =',
    'diff_vocals_harm =',
    'sysex_high_hat_ctrl =',
    'multiplier_note =',
    'pro_drums =',
    'background =',
    'last_play ='
    ]
    
for i in range(0, len(items)-1): 
    stringToBeMoved = items[i+1]
    targetString = items[i]
    moveLine(stringToBeMoved, targetString)

[Paul Wormer]

@paul-wormer
Bug: add the line

lenFile  = editor.getLength()

as first line to the function moveLine (it is the default of findText, which is why I did not see it)

[Paul Wormer]

@mathias-lujan
I tested my little script on your first example and it worked exactly the way you wanted. Today I tested the script on your second example and it works fine again, except that it fails on many undefined items (keywords).

If you want any script to work you have to define a fixed set of keywords that appear in each and every one of your files. The fact that the sets of keywords change between your files makes me wonder how much coding experience you have. Any programmer understands that automatizing your editing is impossible when the keywords vary between files. When you have to introduce a separate set of keywords for every file, you may as well do the editing by hand.

[Alan Kilborn]

@paul-wormer said in newbee, macros and basic algorithm with Python Script:

If you want any script to work you have to define a fixed set of keywords that appear in each and every one of your files

I don’t know that this is true. A keyword could belong to the set of undefined keywords, it just couldn’t be specially manipulated (reordered into the top grouping the OP seems to want for some keywords). It could certainly “tag along” in the reformatting effort (although apparently not in the script you wrote).

[Paul Wormer]

@alan-kilborn
The OP wasn’t very clear about what he wanted exactly, but I assumed that he wanted a definite order for all keywords appearing in a certain file. If he wanted to distinguish two kinds of keywords - those with a well-defined order and a those with no order (except perhaps that they come after the ordered ones) - he should have stated this explicitly.

[Mathias Lujan]

@paul-wormer It’s true, I have not been very clear.
I had just proposed two very different examples. I were thinking that was enough to illustrate it.
In my 5000 ‘song.ini’, there is all the configurations. That’s why I want to organize that.
I’m sorry for this imprecision which makes you contribute out of the subject.

[Mathias Lujan]

Otherwise, I’m extremely grateful to @mpheath who gave me the solution “clé en main” (ready to use). I have understood it thanks to his explanations. I have implemented both Notepad++ Python Script and Standalone recursive Python Script : both work perfectly.
I might even consider changing more elements very easily (thanks to list ‘presets’), and, amazing, modify the 5000 files without open them in Notepad++ (a lot of time saving). That’s a lot more than I expected.

BIG THANKS to @mpheath and other for contributions.

I share with you the two scripts in final version.
The Notepad++ Python Script :

# coding: utf8
# -*- coding: utf-8 -*-
# Here is PythonScript to replace the text in the current editor pane.

def ini_reindex():

    # Get text from the editor pane.
    text = editor.getText()

    if text == '':
        return ''

    # Presets ordered as the top-most keys.
    presets = ['name', 'artist', 'album', 'genre', 'year', 'track',
               'loading_phrase', 'charter', 'frets', 'song_length']

    # Create a list of indexed lines.
    lines = [[-1, line.replace('\n', '')] for line in text.splitlines()]
	# Creates a list of lists like [[-1, 'Line 1'], [-1, 'Line 2'], [-1, 'Line 3'], ...]
	# The -1 will be replaced with numbers to order the lines later for sorting.

    # Add preset keys if not found.
    for item in presets:
        for line in lines:
            if line[1].startswith(item + ' = '):
			# item is each of presets list of ['name', 'artist', 'album', 'genre', 'year', 'track', 'loading_phrase', 'charter', 'frets', 'song_length']
			# So 1st one it trys to match is start with name = . Then tries artist = … until all items are tried. If break does not occur, then the else happens and the item key is added to the list.
                break
        else:
            lines.append([-1, item + ' = '])
			# lines is a list and .append is a method to add to the list.
			# If the else ocurrs for not finding example 'name = ', then [-1, 'name = '] is added to the list.

    # Index the preset keys.
    index = 0

    for item in presets:
        for line in lines:
            if line[0] == -1:
			# This is lines with a -1 index, not given a line number yet. If starts with [ (section header) or item + ' = ' assign a line number to line[0], which is replace the -1 with a line number. index is the line number so index += 1 increments the number by adding 1.
                if line[1].startswith('[') or line[1].startswith(item + ' = '):
                    line[0] = index
                    index += 1

    # Index the remaining keys.
    for line in lines:
        if line[0] == -1:
            line[0] = index
            index += 1

    # Sort by index.
    # Sorting the list of lists lines by the first element [0]. lambda is like a nameless function. The sort will look at example [[2, 'Line 2'], [1, 'Line 1']] so it sees the 1 and shifts it into 1st place and then 2 for 2nd place. This gets the list into the order that is wanted.
    lines.sort(key=lambda x: x[0])

    # Create text from lines.
    text = ''

    for line in lines:
        text += line[1] + '\n'

    return text

# __name__ is the name of the running script. If named __main__, run the following code.
# ini_reindex() is the function call and the returned value is saved in text. If the text returned is not an empty string, then call editor.setText(text) to replace edit pane contents with the value of text.
if __name__ == '__main__':
    text = ini_reindex()

    if text != '':
        editor.setText(text)

And the Standalone recursive Python Script :

import glob

# Presets ordered as the top-most keys.
presets = ['name', 'artist', 'album', 'genre', 'year', 'track',
           'loading_phrase', 'charter', 'frets', 'song_length']

# Start processing each 'song.ini' file.
print('file:')

# for file in glob.iglob('song.ini'): # This will do 'song.ini' in current directorie.
for file in glob.iglob(r'**\song.ini', recursive=True): # This will do 'song.ini' in current and subdirectories.
    print(' ', file)

    # Read the 'song.ini' file and create a list of indexed lines.
    with open(file, 'r', encoding='utf8') as r:
        lines = [[-1, line.replace('\n', '')] for line in r]

    # Add preset keys if not found.
    for item in presets:
        for line in lines:
            if line[1].startswith(item + ' = '):
                break
        else:
            lines.append([-1, item + ' = '])

    # Index the preset keys.
    index = 0

    for item in presets:
        for line in lines:
            if line[0] == -1:
                if line[1].startswith('[') or line[1].startswith(item + ' = '):
                    line[0] = index
                    index += 1

    # Index the remaining keys.
    for line in lines:
        if line[0] == -1:
            line[0] = index
            index += 1

    # Sort by index.
    lines.sort(key=lambda x: x[0])

    # Save 'song.ini' file.
    with open(file, 'w', encoding='utf8') as w:
        for line in lines:
            w.write(line[1] + '\n')

print('done')

[Alan Kilborn]

See how long a discussion thread can go on when the problem statement is imprecise? I don’t know that it really is precise yet. Here’s how I see the problem:

If lines starting with these items are present, move them to the top, and put them in this exact order:

name =
artist =
album =
genre =
year =
track =
loading_phrase =
charter =
frets =
song_length =
icon =

If they are not present, make them present but be empty, e.g. frets = and then nothing following.

For any other lines, leave them as they are after the icon = line in the output.

[Mathias Lujan]

@alan-kilborn I’m sorry again. I try to do my best…
Your formulation is perfect.
Just a detail, ‘icon =’ is not important for me. So :
For any other lines, leave them as they are after the ‘song_length =’ line in the output.