MacroInspect which comments macros

mpheath

To All, I have updated the gist with revision 3.

Implemented the @Alan-Kilborn idea of single json file saved next to the script.

Please manually remove existing data files from the Config directory:

FindReplaceDlg_rc.h
FindReplaceDlg_rc.h.json
Scintilla.h
Scintilla.h.json

Update MacroInspect.py in the Config\PythonScript\scripts directory. On first run should download the header data and save to Config\PythonScript\scripts\MacroInspect_data.json. No temporary header files as the data will all be processed in memory.

Improved message 1701 comment which is IDC_FRCOMMAND_EXEC will now show it’s lParam related constant.
Some examples:

<!-- IDC_FRCOMMAND_EXEC  [IDCMARKALL] -->
<!-- IDC_FRCOMMAND_EXEC  [IDD_FINDINFILES_FIND_BUTTON] -->
<!-- IDC_FRCOMMAND_EXEC  [IDD_FINDINFILES_FINDINPROJECTS] -->

Alan Kilborn

I ran revision 3 and it seemed to go as desired.

Afterward I looked for the MacroInspect_data.json file and curiously I did not find it anywhere on my system.

After changing the code to add print('json_file:', json_file) I then saw the problem.
I use a script to “run the script in the active tab” so that I can tie it to a keycombo for quickly executing a script under development, over and over again as development proceeds. I think I would go mad if I had to navigate the menus to repeatedly run a script I’m actively working on. (Yes, there are other alternatives…)

Anyway, I found that the data file for your script was being created for me as RunCurrentPyFileAsPythonscript_data.json because my “script runner” script is named RunCurrentPyFileAsPythonscript.py. This results (apparently) from the use of __file__ in your code.

No big deal, I can compensate for this… :-)

Again, great job on a great script.

mpheath

@Alan-Kilborn

Sorry to hear. It’s still a bug even if it is a script running a script. I just tested sys.argv and thought argument 0 might be OK though it returns ['C:\\Programs\\Notepad++\\notepad++.exe']. If you can test something that works for you and everyone else then please share. I may test some alternatives though I am not sure if I can reproduce your environment to be sure of a fix.

Alan Kilborn

@mpheath

In general, a way I’ve found to do it is:

inspect.getframeinfo(inspect.currentframe()).filename

after importing inspect, and then getting the directory part of the path from that.

This worked great in PythonScript2, as the path to the file was returned from the inspect call. In PythonScript3, however, all that call obtains is a filename without the complete path.

You can experiment with that if you like, or I’ll post some more complete code later.

mpheath

@Alan-Kilborn

Just tested in a Python interpreter. __file__ can be set to a value. So perhaps your RunCurrentPyFileAsPythonscript.py can set __file__ to the absolute path of the child script before it runs the subprocess, so then the child process use of __file__ is inherited like as if it is the main process.

Alan Kilborn

@mpheath said:

perhaps your RunCurrentPyFileAsPythonscript.py can set file to the absolute path of the child script before it runs …

A good idea, however it doesn’t work (I still get the calling script’s name/path), at least the way I’m having a script run a script; there are many ways to do it. It’s actually surprisingly difficult to have a script run a script (and have things all turn out like you’d like).

LATER EDIT:
Actually, it did work; what caused it to not work the first time I tried it was programmer error, i.e., me!
I may have to look further into using the __file__ variable technique instead of my inspect-based technique, as it seems like it could simplify things under Python3.

Alan Kilborn

@mpheath

I see you didn’t break down the search “boolean” values in your latest script, so, I took the liberty of pulling that code from my now defunct macro disassembler and inserting it into your script:

elif message == 'IDC_FRCOMMAND_BOOLEANS':
    numeric_data = int(v)
    bit_weights_numeric_value_to_str_dict = {
        1    : 'MATCH_WHOLE_WORD_ONLY',
        2    : 'MATCH_CASE',
        4    : 'PURGE_MARKS_BEFORE_NEW_SEARCH',
        16   : 'BOOKMARK_LINE',
        32   : 'IN_SUBFOLDERS',
        64   : 'IN_HIDDEN_FOLDERS',
        128  : 'IN_SELECTION__OR__PROJECT1',
        256  : 'WRAP_AROUND__OR__PROJECT2',
        512  : 'FORWARD_DIRECTION__OR__PROJECT3',
        1024 : 'DOT_MATCHES_NEWLINE_FOR_REGEX',
        }
    running_single_bitweight = 1
    search_options_str_list = []
    while numeric_data != 0:
        if running_single_bitweight in bit_weights_numeric_value_to_str_dict:
            if (numeric_data & running_single_bitweight) != 0:
                search_options_str_list.append(bit_weights_numeric_value_to_str_dict[running_single_bitweight])
        numeric_data &= ~running_single_bitweight
        running_single_bitweight <<= 1
    value_str = 'NONE' if len(search_options_str_list) == 0 else ' / '.join(search_options_str_list)
    comment += '  [ ' + value_str + ' ]'

mpheath

@Alan-Kilborn

I have already tested IDC_FRCOMMAND_BOOLEANS and ran into issues. If you look at the json data file, you may see lines like:

        "128": "IDF_IN_SELECTION_CHECK IDF_FINDINFILES_PROJECT1_CHECK",
        "256": "IDF_WRAP IDF_FINDINFILES_PROJECT2_CHECK",
        "512": "IDF_WHICH_DIRECTION IDF_FINDINFILES_PROJECT3_CHECK",

which has space delimited pairs of constant names.

as I use this code to fill the dictionary with the constant pairs:

                if k in dic:
                    dic[k] += ' ' + v
                else:
                    dic[k] = v

which are pairs that share the same boolean value in which only 1 can be correct. Though in testing depending on the FindReplaceDLG tab recorded, sometimes none are correct IMO. I am unsure if I can fix the code I have to make it reliable as some incorrect comments may make the user puzzled. I suspect that perhaps some bit flags should not be in the lParams value and perhaps the playback ignores them, though that does not help with making the xml comments.

Edit:

There is also a possibility that Notepad++ is getting the checkbox booleans incorrect. Unchecked/checked is ok until disabling the checkbox, which leads to a 3rd state with a enabled/disabled needing to be validated. IDK yet and so may need investigation. This may explain why I get IDF_FINDINFILES_PROJECT2_CHECK and IDF_FINDINFILES_PROJECT3_CHECK in the comments even though they are unchecked and disabled so cannot be valid in the xml comments.

Alan Kilborn

@mpheath said in MacroInspect which comments macros:

I have already tested IDC_FRCOMMAND_BOOLEANS and ran into issues.

I don’t know…but I’m fairly certain the code I posted works correctly.

which are pairs that share the same boolean value in which only 1 can be correct.

Yep, the “genius” that implemented the project options decided repurposing some bit weighting for that was a great idea.

I suspect that perhaps some bit flags should not be in the lParams value and perhaps the playback ignores them

Yes, there can be items that are not applicable in there.
If one knew in advance what the 1701 value was going to be, one could filter out the n/a items, but alas the 1702 message comes before the 1701.

mpheath

To All, I have updated the gist with revision 4.

Added feature for IDC_FRCOMMAND_BOOLEANS comment to display the options as the related boolean constants.
Added code to only comment Actions that a not in a multi-line comment block which is quite basic code just intended for the existing comments distributed in shortcuts.xml.

This script might be complete unless I have missed something.

@Alan-Kilborn Thanks for sharing the IDC_FRCOMMAND_BOOLEANS code that you use. I went with what I had and decided to keep the code basic and not go deeper into it as the IDC_FRCOMMAND_EXEC actions are many. Two passes of the shortcuts.xml could be done to get the IDC_FRCOMMAND_EXEC actions on the first pass and use a blacklist on the second pass though is it better to try simple IMO before trying complex and probably end up with much uglier code. And as you mention earlier with the idea of a PythonScript example script which IMO should be simple to understand.

@guy038 I’m not a fan of using attributes as comments though this script may help to get the information quicker which from my point of view was the main goal.

@PeterJones Thanks for the lesson of why attributes as comments are used. I was not aware of the exact reason before this topic started.

Alan Kilborn

@mpheath

Getting better all the time…

One minor thing I noticed that I thought a bit odd was that the output file tab had LF line endings rather than the more traditional CRLF, but this is hardly important.

A “second pass” might be worth it for the “booleans”, because you could (a) be more specific about them (e.g. is it “in selection” or is it “project 1”?), and (b) you could filter out or flag irrelevant options (e.g. “search direction” is meaningless for a file-level search operation, but it can be encoded in the booleans).

mpheath

OK, bug squashing time. Just committed revision 5 though let’s works towards revision 6.

@Alan-Kilborn said in MacroInspect which comments macros:

One minor thing I noticed that I thought a bit odd was that the output file tab had LF line endings rather than the more traditional CRLF, but this is hardly important.

Python normalizes \n to the newline sequence though may need to set the new tab to use CRLF.

A “second pass” might be worth it for the “booleans”, because you could (a) be more specific about them (e.g. is it “in selection” or is it “project 1”?), and (b) you could filter out or flag irrelevant options (e.g. “search direction” is meaningless for a file-level search operation, but it can be encoded in the booleans).

a) There is no 1or2 AFAIK. This is the purpose of find_in_files_mode which is the boolean flag to select which one of the pairs of constants to display. It determines this based on a previous action containing FINDINFILES. Perhaps you have a macro that shows where this selection fails?

b) It is a odd constant I agree. IDF_WHICH_DIRECTION is forward direction though not displayed if backwards if I have that correct. Seems poorly named and knowing if backwards seems more important.

Alan Kilborn

@mpheath said:

Python normalizes \n to the newline sequence

Assuming you mean “Python normalizes \n to \r\n on Windows”, yes, this is true but only when writing to a file, which your code is not doing (you are writing to an untitled tab, e.g. new 2).

Perhaps you have a macro that shows where this selection fails?

OK.
I saw this:

            <!-- IDC_FRCOMMAND_BOOLEANS  [IDF_MATCHCASE, IDF_FINDINFILES_RECURSIVE_CHECK, IDF_FINDINFILES_PROJECT3_CHECK] -->
            <Action type="3" message="1702" wParam="0" lParam="546" sParam="" />
            <!-- IDC_FRCOMMAND_EXEC  [IDD_FINDINFILES_FIND_BUTTON] -->
            <Action type="3" message="1701" wParam="0" lParam="1656" sParam="" />

and I saw PROJECT3 in it, and that is irrelevant to a FINDINFILES. But really, I think it is just a boolean (for “direction”) that doesn’t affect the search but got in there as part of the “loose” N++ code handling the booleans when a macro is recorded. Maybe it is another good reason to filter it out or flag it somehow – so dumb users like me don’t think it is relevant. Again, not sure how much further you want to take your script.

Maybe helpful as a reference, here’s what I had in my old “disassembler” code as to what options are relevant to what search actions:

I think you can figure out that “ww” is whole-word and “mc” is match-case. “pp” had me confused for a moment, but I think it is presearch-purge.

Maybe a N++ issue should be raised that sometimes irrelevant search options are being recorded into the booleans? What do you think?

mpheath

@Alan-Kilborn said in MacroInspect which comments macros:

Assuming you mean “Python normalizes \n to \r\n on Windows”, yes, this is true but only when writing to a file, which your code is not doing (you are writing to an untitled tab, e.g. new 2).

Reading and write to files, Python does normalize though the editor methods do not so may need to harmonize EOLs which I may have solved locally and can post a revised update. shortcuts.xml can be read from editor and read from file so both need to be \r\n literally so editor.addText() adds \r\n literally.

… and I saw PROJECT3 in it, and that is irrelevant to a FINDINFILES. But really, I think it is just a boolean (for “direction”) that doesn’t affect the search but got in there as part of the “loose” N++ code handling the booleans when a macro is recorded. Maybe it is another good reason to filter it out or flag it somehow – so dumb users like me don’t think it is relevant. Again, not sure how much further you want to take your script.

Ah, the extra constants that perhaps should not be there in the comments as I mentioned earlier. How far? not more than what is needed is what I hope for.

Maybe helpful as a reference, here’s what I had in my old “disassembler” code as to what options are relevant to what search actions:

An interesting table, this might be useful for filtering with a blacklist. Certainly a great effort into making the table. Well done.

Maybe a N++ issue should be raised that sometimes irrelevant search options are being recorded into the booleans? What do you think?

I consider good to fix the problem at the source. Ideally, the script should just do as is done already without doing excessive filtering. It tires me just to think of having to do more when it should not be needed if the booleans were concise and correct … or perhaps I am not so young anymore.

Will see what can be done with the script within reason. I like doing things based on interest though sometimes it can feel like torture and the filtering implementation reminds me of the latter.

Alan Kilborn

Maybe a N++ issue should be raised that sometimes irrelevant search options are being recorded into the booleans? What do you think?

I consider good to fix the problem at the source. Ideally, the script should just do as is done already without doing excessive filtering. It tires me just to think of having to do more when it should not be needed if the booleans were concise and correct … or perhaps I am not so young anymore.

I will follow-up and put an issue in on this.

mpheath

To All, I have updated the gist with revision 6.

Revision 5:

Fixed root path for portable and installed.

Revision 6:

Fixed EOL to be CRLF.

Alan Kilborn

@Alan-Kilborn said:

I will follow-up and put an issue in on this.

Issue is CREATED.

mpheath

@Alan-Kilborn This table is a whitelist or a blacklist? If a whitelist then perhaps could replace the range() with predefined flags in a list like is in your table. Could you paste the table as text in a codebox so it can be copied as text? Then will try 2 pass reading to get the 16** numbers to know which list of booleans to bitwise and append to the comment. Just need to test this concept and it may work more to users satisfaction.

Alan Kilborn

        'numeric_value_to_relevant_options' : {
        #          ww  mc  pp  bm   sub  hid  sel   wrp   bwd   dot
            1    : 1 | 2 |                          256 | 512 | 1024,             # find_next
            1608 : 1 | 2 |                          256 | 512 | 1024,             # replace
            1609 : 1 | 2 |                    128 | 256 | 512 | 1024,             # replace_all
            1614 : 1 | 2 |                    128 | 256 | 512 | 1024,             # count
            1615 : 1 | 2 | 4 | 16 |           128 | 256 | 512 | 1024,             # mark
            1633 :             16 |           128 | 256 | 512       ,             # clear_marking
            1635 : 1 | 2 |                                      1024,             # replace_in_open_tabs
            1636 : 1 | 2 |                                      1024,             # find_in_open_tabs
            1641 : 1 | 2 |                                      1024,             # find_in_active_tab
            1656 : 1 | 2 |          32 | 64 |                   1024,             # find_in_files
            1660 : 1 | 2 |          32 | 64 |                   1024,             # replace_in_files
        #                                     prj1  prj2  prj3
            1665 : 1 | 2 |                    128 | 256 | 512 | 1024,             # replace_in_projects
            1666 : 1 | 2 |                    128 | 256 | 512 | 1024,             # find_in_projects
        },

I’d use the 1701 msg’s numeric data as the key to looking into this dict to get a relevant_options_val. Then I’d take the previous 1702 msg’s numeric data and AND it with ~relevant_options_val to calculate unneeded_options.

mpheath

To All, I have updated the gist with revision 7.

Added dictionary provided by @Alan-Kilborn to improve IDC_FRCOMMAND_BOOLEANS processing with valid boolean lists.

Let me know of issues.