Config Files Need Updating, Too

dinkumoil

@PeterJones
Thank you for your effort writing this script. It helps to solve a long lasting problem.

However, I tested your script and noticed a flaw that I want to report to you. After the script has processed the XML files of all themes and the langs.xml file, the header line of all these files looks like that:

<?xml version='1.0' encoding='utf-8'?>

Please note the single quotes instead of double quotes around the version number and the encoding name. This prevents the XML Tools plugin from switching to the correct encoding when loading one of these files.

I’ve set the default encoding in Notepad++ to ANSI, so all the files mentioned above are loaded with this encoding. Only after changing the single quotes in the header line to double quotes, the encoding is set accordingly to the encoding info in the file’s header.

I’m even not sure if the XML standard allows single quotes around attribute values, at least I’ve never seen it.

Would you mind to fix your script to prevent it from doing these changes to the processed XML files?

PeterJones

@dinkumoil said in Config Files Need Updating, Too:

I’m even not sure if the XML standard allows single quotes around attribute values, at least I’ve never seen it.

For normal attributes, I was under the impression that either single or double quotes were allowed. w3schools, for example, is explicit about that. Other sites I’ve found when searching for xml quotes single double seem to agree.

However, for an XML prolog, I do see SO questions like this which seem to indicate that there are quite a few tools that don’t like the prolog having single-quoted attributes (and also imply that a different set of tools do default to generate the prolog with the single-quotes).

I should be able to update it to use double-quotes, and will edit this post (or reply) when it’s ready.

Lycan Thrope

@PeterJones, you might want to hold on. I can’t duplicate his results. I ran your script yesterday, and reading this today before you answered him, I couldn’t find the lang.xml file in either the program files directory or the Users\ directory doing that. Unless he’s referring to the .model, I can’t duplicate his findings.
Edit:
My Roaming directory langs.xml shows this:
<?xml version="1.0" encoding="UTF-8" ?>
And my Notepad executing directory shows this:
<?xml version="1.0" encoding="UTF-8" ?>
I can’t find his problem in my setup.
And the langs.model.xml is the same.

PeterJones

@Lycan-Thrope ,

Did you use PythonScript 2 (“PS2”) or PythonScript 3 (“PS3”)?

In PS3, the xml.etree.ElementTree library handles the xml_declaration, and that library is hardcoded to use single-quotes for the xml_declaration/prolog (as discussed here). I used PS3 on my machine, and I agree with @dinkumoil that there are single quotes in the output langs.xml and stylers.xml and themes\*.xml

In PS2, the xml.etree.ElementTree library doesn’t have the xml_declaration option, so I then hardcoded the script to use the double-quote.

Since the PS3 option does the wrong version, I’ll probably just switch it over to have both PS2 and PS3 manually add the prolog with the quotes I want.

Lycan Thrope

@PeterJones,
Let me check.
Edit: Hmm, I have both installed. 2.7 and 3.10.
I have the 2.7 dll installed in the Pythonscript directory. I think I might have had a problem with 3.10 and changed it back…so yeah, you’re right, I have 2.7 so I don’t have the problem he shows.

PeterJones

@dinkumoil ,

I have updated the script to hardcode the double-quotes for xml declaration/prolog.

However, you might want to put in an issue at XML Tools, because if my internet search is any indication (and given that the standard Python xml.etree.ElementTree library does it that way), it would probably be good for XML Tools to be able to handle single-quoted declaration/prolog attributes.

As far as encoding auto-detection – yeah, a lot of such features (probably including the ones that do vimline-style detection for encoding) are likely to have had writers who made the assumption that all declaration/prologs use double-quotes.

PeterJones

ConfigUpdater.py v1.02 is available here (the same link as before – that link will always have the most recent version)

#  Version: 1.02 (2024-Aug-28)  - FEATURE = add "isIntermediateSorted" mode; enable it by editing the bottom of this script to call ConfigUpdater.go(True) instead of ConfigUpdater.go()
#                               - BUGFIX = correctly propagate keywordClass from model to stylers & themes
#                               - BUGFIX = correctly propagate colors to new languages or styles within a language, and more-natural inheritence for default styler from model

Since this is still an early version, I was asked to add a “debug mode” to help make using ComparePlus between the old and the new stylers.xml easier. Since the script changes the order of the languages, it makes it harder to compare your old stylers.xml to the updated version, to make sure nothing was lost. The “isIntermediateSorted” flag will save stylers.xml.orig.sorted and langs.xml.orig.sorted when it is run, so that it will take the original XML, but sort it per the new rules. That way, you can compare that sorted version to the final XML output, and more easily see what was added, and make sure nothing was lost. (I haven’t yet seen anything lost, but I will admit I haven’t tried every edge case yet)

Even with that flag, if you are using PythonScript 2 plugin, you will find more differences than expected between the sorted and the final, because the XML library in PythonScript 2 does not maintain or guarantee the order of attributes in XML elements. The PythonScript 3 library does its best to keep the attribute order the same, so its output files are easier to compare.

There were also a couple of bugfixes.

In v1.00/1.01, the keywordClass wasn’t properly brought from stylers.model.xml to your stylers/themes. v1.02 fixes that – and if you had already run the earlier version, it’s still safe to run the 1.02 script, and it will still make that keywordClass fix for you.
The other bugfix involved how it decides what colors to use for new languages or new styles within an existing language – it didn’t quite do what I intended before, although it accomplished the critical goal of getting all the languages and styles into stylers.xml and the other themes. (This fix also has a secondary feature: when it’s copying from stylers.model.xml into stylers.xml, it will inherit the fg/bg for a given style from the model instead of just using the theme’s default colors, so it’s a bonus feature to make it “more natural behavior” during the copy over from the model file.)

Snabel42

@Lycan-Thrope I’m not sure if it affects this, but XML Tools has a setting for which engine should be used for XML validation (SmipleXML vs QuickXML vs StringXML).

Are you both using the same engine?

dinkumoil

@PeterJones said in Config Files Need Updating, Too:

I have updated the script to hardcode the double-quotes for xml declaration/prolog.

Thank you for the fix, works as expected.

However, you might want to put in an issue at XML Tools …

You are right, I will do that. [EDIT] Done [/EDIT]

@Lycan-Thrope @PeterJones
I use Notepad++ v8.6.9 (64 Bit), PythonScript v3.0.18 and XML Tools v3.1.1.13

@Snabel42
I’m not sure, but it seems like the engine setting you mentioned above only affects the formatting during pretty-printing. However, I use QuickXML.

Lycan Thrope

@Snabel42 ,
Apparently mine is QuickXML. I didn’t set anything, so it looks like it was preset for me.

Lycan Thrope

@dinkumoil said in Config Files Need Updating, Too:

I use Notepad++ v8.6.9 (64 Bit), PythonScript v3.0.18 and XML Tools v3.1.1.13

That would explain the difference. I can’t remember exactly what the problem was or why I went back to 2.7 for the PythonScript plugin, but I just did…it might have been an early version where there were problems and I just stayed with the tried and true, at the time.

PeterJones

@Lycan-Thrope said at various times:

Hmm, I have both installed. 2.7 and 3.10.

and

I went back to 2.7 for the PythonScript plugin

The way you’ve phrased things in this conversation, you seem to have mentally blurred the lines between the plugin version and the version of Python that the plugin uses.

All that matters to the plugin is which version of the PythonScript plugin you use, and which python###.dll the plugin provides. It doesn’t matter which python.exe you might have installed elsewhere on your computer, whether they are Python 2.7 or Python 3.10 or anything else (and, in fact, the plugin ignores other installations unless you have checkmarked the ☑ Prefer installed Python libraries plugin option; and if you have that option checked, the installed version must match the 3.xx that PythonScript was compiled against).

PythonScript plugin v2.0.0.0 uses Python 2.7.
- There is no plugin v2.7.
PythonScript plugin v3.0.xx uses Python 3.yy.
- The plugin hasn’t gotten to v3.10 yet
- It’s currently on PythonScript v3.0.18 using Python 3.12.3
- you might want to give PythonScript v3.0.18 a try, because it seems pretty stable

Both the plugin version and the python version are available in Plugins > PythonScript > About and printed to the console when you first open Plugins > PythonScript > Show Console

Ekopalypse

Hello Peter,
many thanks for the script.

Two comments:

a) Themes with commented LexerTypes cause an exception when sorting.

Not really tested, but you could think about something like

        def sort_key(child):
            child_name = child.get('name')
            return (False, '') if child_name is None else (child_name == 'searchResult', child_name)

        elThemeLexerStyles[:] = sorted(elThemeLexerStyles, key=sort_key)

b) Themes without top-level comments receive the comments from the previous theme. For example, I have a theme called mytheme.xml that now contains the copyright comments from mossylawn.xml.

PeterJones

@Ekopalypse ,

Thanks for those catches. v1.03 is now available at the permalink, and should fix both of those issues

Lycan Thrope

@PeterJones ,
The first is what mine shows.
PythonScript plugin v2.0.0.0 uses Python 2.7.

Python 2.7.18 (v2.7.18:8d21aa21f2, Apr 20 2020, 13:25:05) [MSC v.1500 64 bit (AMD64)]
Copyright (c) 2001-2020 Python Software Foundation.
All Rights Reserved.

I have Python27.dll in the C:\Program Files\Notepad++\plugins\PythonScript - directory.
And, I didn’t have the Prefer option checked, in the Options menu.
Just for confirmation.

PeterJones

v1.06 is available from the permalink:

v1.05 (2024-Sep-05): Add URLs to source code, so you remember where to get an updated copy. (was not announced, since it was not a critical feature/bugfix upgrade)
v1.06 (2024-Sep-14): IMPROVEMENT = error handling for <installed>\themes\ permissions
- The most common error is that you don’t have write permissions for the default themes, which are installed in the installation directory instead of your AppData or other config directory.
- Users have been confused when it doesn’t complete because of write permissions, because I forgot that not everyone has PythonScript set to show the Console if there’s an error, and that error isn’t intuitive to the general audience even if they look in the Console.
- I now trap that error, and I’ve added a MessageBox to inform you of the error, letting you know that if you re-run Notepad++ As Admin, then it should work.
  - Some users have found that when they run As Admin, PythonScript no longer sees their user scripts from the AppData hierarchy. If that’s the case, just save another copy of the script in the Machine Scripts section while you’re running As Admin.
- Even if you get the write error on the <installed>\themes, your AppData or other config-directory stylers and themes and langs.xml should all be properly updated. (If you use Windows Explorer, you should be able to see the last-modified time on the AppData or other config-directory files has been updated after running the script.)

PeterJones

v1.07 is available from the permalink:

fix global styles
1. copy global-styles colors from stylers.model.xml to stylers.xml (similar to what’s already done on stylers.xml’s lexer-styles)
2. when updating/adding global styles in either stylers.xml or themes\*.xml, make sure global-styles don’t have extra attributes that confuse the StyleConfigurator interface

running v1.07 will clean up the Global Styles attributes to match the stylers.model.xml, even if you’ve run a previous version of the script

notdodgeball

@PeterJones, I used version 1.07 of the script and the copyright information on top got removed from the theme files.

They were also unmodified, straight from the install directories, version Portable v8.7.1

PeterJones

@notdodgeball ,

Apparently the xml.etree library behaves differently between Python 2.7 and 3.12 (used by PS2 and PS3 respectively).

I really wish PS3 would move out of alpha and release into Plugins Admin, so that we could just stick with PythonScript 3 for sharing scripts in the forum, and not have to contend with all the headaches of keeping scripts compatible with two major versions of Python, and all the library differences that come with that.

(it also would’ve been nice if Don hadn’t allowed theme authors to submit themes with XML that have invalid top-level comments, which the XML parsing engines don’t like because it’s not valid XML)

v1.08 is available from the permalink, fixing the handling of the invalid XML in PythonScript 2.

However, I highly recommend making the switch to PythonScript 3, despite it claiming to be alpha, unless you are using a lot of ANSI characterset files rather than real unicode-based encodings. I don’t know how much longer I’ll support PythonScript 2 for this script.