Config Files Need Updating, Too

PeterJones

Are you a long-time user of Notepad++?

Have you ever added user-defined keywords in the Style Configurator, or customized your colors in one of the themes or the default stylers.xml?

Do you still want to have the latest-and-greatest list of languages, keywords, and styles for the programming languages available to you, without losing the customizations you’ve already made?

Does it bother you that the “Hello Kitty” theme doesn’t know GDScript syntax highlighting?

Have you ever found yourself manually copying bits of XML from stylers.model.xml or langs.model.xml to your active configuration files, hoping that you’ve gotten it right?

Then have I got a script for you!

The Problem

When you update from a previous version of Notepad++, the installer will not overwrite your existing stylers, themes, and langs XML files, to avoid deleting your customizations. But that means if there have been new languages added to Notepad++ (like Raku and Go), or if a language has a new Style available in the Style Configurator, or if the list of default keywords for a language has been updated, those changes will not be propagated to your active installation. That means you might not have even noticed that Raku can be syntax-highlighted in Notepad++, or your default keyword lists might be out of date without you even noticing.

The Solution

Introducing: ConfigUpdater.py

I wrote the script ConfigUpdater.py for the Python Script plugin (either version 2.0.0 or 3.0.18-or-newer). It will compare your %AppData%\Notepad++\stylers.xml (or stylers.xml in your active config-file location) and all your themes against <install_directory>\stylers.model.xml, and compare your active langs.xml against <install_directory>\langs.model.xml, and it will bring over any new languages, new styles within a given language, and any new or updated keyword lists – and it will do this without losing any of your customizations – it will do this by merging the new XML elements, attributes, and values without overriding any existing values. If new words have been added to the default keywords list, this script will add in the new words, without losing any of the existing words.

Picking Colors

For new languages and new style elements inside a given language, I don’t want to just pick a color and hope that it works with your selected theme, and I don’t want to just copy over the colors from the default light-mode settings from stylers.model.xml, which might completely clash with your dark-inspired theme. Instead, when adding a new language or style element, I assign its foreground and background colors to the default foreground and background for your theme – this means that new languages, and new styles within an existing language, will not look highlighted compared to other text, but they will also not clash with your existing theme.

Since you will want the syntax highlighting to be something different than your theme’s default colors, you will still have to go into the Style Configurator and set reasonable colors. But unlike before, you won’t have to first manually edit the stylers.xml or theme\XXX.xml to even show you the new entries in the Style Configurator. So I hope this somewhat simplifies the theme-upgrade procedure for you.

Instructions

Install PythonScript (you can either install the default v2.0.0 using the Plugins Admin, as described in our PythonScript FAQ, or you can install v3.0.18 or newer from the project’s GitHub releases page)
Download the source code for ConfigUpdater.py and install that per the FAQ instructions
If you have a normal installation of Notepad++, the themes are in the protected C:\Program Files\Notepad++\themes directory, so you will need to run Notepad++ as Administrator (right click on notepad++.exe and Run as administrator). If you have a portable copy, or are using the Cloud Folder or -settingsDir, you can probably get away with not running as Administrator.
After the script has been run, just exit Notepad++ and restart.
After this sequence, all the new Languages and Style-Configurator Styles for a given language will be updated. If you want non-default colors for the new Languages and Styles, you will need to update those entries yourself (as described in Picking Colors, above.)

Whenever you update Notepad++, run this script right after, and any updates from stylers.model.xml or langs.model.xml will be propagated to your active setup.

This is useful to run for v8.6.9, in case you haven’t updated your configuration to include the recent languages like Hollywood, Raku, and Go. And after the next Notepad++ release (v8.7), multiple languages will be getting new Style categories and new keyword lists to go with them, so it will be highly-advantageous to run this script again after updating to v8.7.

dinkumoil

@PeterJones
Thank you for your effort writing this script. It helps to solve a long lasting problem.

However, I tested your script and noticed a flaw that I want to report to you. After the script has processed the XML files of all themes and the langs.xml file, the header line of all these files looks like that:

<?xml version='1.0' encoding='utf-8'?>

Please note the single quotes instead of double quotes around the version number and the encoding name. This prevents the XML Tools plugin from switching to the correct encoding when loading one of these files.

I’ve set the default encoding in Notepad++ to ANSI, so all the files mentioned above are loaded with this encoding. Only after changing the single quotes in the header line to double quotes, the encoding is set accordingly to the encoding info in the file’s header.

I’m even not sure if the XML standard allows single quotes around attribute values, at least I’ve never seen it.

Would you mind to fix your script to prevent it from doing these changes to the processed XML files?

PeterJones

@dinkumoil said in Config Files Need Updating, Too:

I’m even not sure if the XML standard allows single quotes around attribute values, at least I’ve never seen it.

For normal attributes, I was under the impression that either single or double quotes were allowed. w3schools, for example, is explicit about that. Other sites I’ve found when searching for xml quotes single double seem to agree.

However, for an XML prolog, I do see SO questions like this which seem to indicate that there are quite a few tools that don’t like the prolog having single-quoted attributes (and also imply that a different set of tools do default to generate the prolog with the single-quotes).

I should be able to update it to use double-quotes, and will edit this post (or reply) when it’s ready.

Lycan Thrope

@PeterJones, you might want to hold on. I can’t duplicate his results. I ran your script yesterday, and reading this today before you answered him, I couldn’t find the lang.xml file in either the program files directory or the Users\ directory doing that. Unless he’s referring to the .model, I can’t duplicate his findings.
Edit:
My Roaming directory langs.xml shows this:
<?xml version="1.0" encoding="UTF-8" ?>
And my Notepad executing directory shows this:
<?xml version="1.0" encoding="UTF-8" ?>
I can’t find his problem in my setup.
And the langs.model.xml is the same.

PeterJones

@Lycan-Thrope ,

Did you use PythonScript 2 (“PS2”) or PythonScript 3 (“PS3”)?

In PS3, the xml.etree.ElementTree library handles the xml_declaration, and that library is hardcoded to use single-quotes for the xml_declaration/prolog (as discussed here). I used PS3 on my machine, and I agree with @dinkumoil that there are single quotes in the output langs.xml and stylers.xml and themes\*.xml

In PS2, the xml.etree.ElementTree library doesn’t have the xml_declaration option, so I then hardcoded the script to use the double-quote.

Since the PS3 option does the wrong version, I’ll probably just switch it over to have both PS2 and PS3 manually add the prolog with the quotes I want.

Lycan Thrope

@PeterJones,
Let me check.
Edit: Hmm, I have both installed. 2.7 and 3.10.
I have the 2.7 dll installed in the Pythonscript directory. I think I might have had a problem with 3.10 and changed it back…so yeah, you’re right, I have 2.7 so I don’t have the problem he shows.

PeterJones

@dinkumoil ,

I have updated the script to hardcode the double-quotes for xml declaration/prolog.

However, you might want to put in an issue at XML Tools, because if my internet search is any indication (and given that the standard Python xml.etree.ElementTree library does it that way), it would probably be good for XML Tools to be able to handle single-quoted declaration/prolog attributes.

As far as encoding auto-detection – yeah, a lot of such features (probably including the ones that do vimline-style detection for encoding) are likely to have had writers who made the assumption that all declaration/prologs use double-quotes.

PeterJones

ConfigUpdater.py v1.02 is available here (the same link as before – that link will always have the most recent version)

#  Version: 1.02 (2024-Aug-28)  - FEATURE = add "isIntermediateSorted" mode; enable it by editing the bottom of this script to call ConfigUpdater.go(True) instead of ConfigUpdater.go()
#                               - BUGFIX = correctly propagate keywordClass from model to stylers & themes
#                               - BUGFIX = correctly propagate colors to new languages or styles within a language, and more-natural inheritence for default styler from model

Since this is still an early version, I was asked to add a “debug mode” to help make using ComparePlus between the old and the new stylers.xml easier. Since the script changes the order of the languages, it makes it harder to compare your old stylers.xml to the updated version, to make sure nothing was lost. The “isIntermediateSorted” flag will save stylers.xml.orig.sorted and langs.xml.orig.sorted when it is run, so that it will take the original XML, but sort it per the new rules. That way, you can compare that sorted version to the final XML output, and more easily see what was added, and make sure nothing was lost. (I haven’t yet seen anything lost, but I will admit I haven’t tried every edge case yet)

Even with that flag, if you are using PythonScript 2 plugin, you will find more differences than expected between the sorted and the final, because the XML library in PythonScript 2 does not maintain or guarantee the order of attributes in XML elements. The PythonScript 3 library does its best to keep the attribute order the same, so its output files are easier to compare.

There were also a couple of bugfixes.

In v1.00/1.01, the keywordClass wasn’t properly brought from stylers.model.xml to your stylers/themes. v1.02 fixes that – and if you had already run the earlier version, it’s still safe to run the 1.02 script, and it will still make that keywordClass fix for you.
The other bugfix involved how it decides what colors to use for new languages or new styles within an existing language – it didn’t quite do what I intended before, although it accomplished the critical goal of getting all the languages and styles into stylers.xml and the other themes. (This fix also has a secondary feature: when it’s copying from stylers.model.xml into stylers.xml, it will inherit the fg/bg for a given style from the model instead of just using the theme’s default colors, so it’s a bonus feature to make it “more natural behavior” during the copy over from the model file.)

Snabel42

@Lycan-Thrope I’m not sure if it affects this, but XML Tools has a setting for which engine should be used for XML validation (SmipleXML vs QuickXML vs StringXML).

Are you both using the same engine?

dinkumoil

@PeterJones said in Config Files Need Updating, Too:

I have updated the script to hardcode the double-quotes for xml declaration/prolog.

Thank you for the fix, works as expected.

However, you might want to put in an issue at XML Tools …

You are right, I will do that. [EDIT] Done [/EDIT]

@Lycan-Thrope @PeterJones
I use Notepad++ v8.6.9 (64 Bit), PythonScript v3.0.18 and XML Tools v3.1.1.13

@Snabel42
I’m not sure, but it seems like the engine setting you mentioned above only affects the formatting during pretty-printing. However, I use QuickXML.

Lycan Thrope

@Snabel42 ,
Apparently mine is QuickXML. I didn’t set anything, so it looks like it was preset for me.

Lycan Thrope

@dinkumoil said in Config Files Need Updating, Too:

I use Notepad++ v8.6.9 (64 Bit), PythonScript v3.0.18 and XML Tools v3.1.1.13

That would explain the difference. I can’t remember exactly what the problem was or why I went back to 2.7 for the PythonScript plugin, but I just did…it might have been an early version where there were problems and I just stayed with the tried and true, at the time.

PeterJones

@Lycan-Thrope said at various times:

Hmm, I have both installed. 2.7 and 3.10.

and

I went back to 2.7 for the PythonScript plugin

The way you’ve phrased things in this conversation, you seem to have mentally blurred the lines between the plugin version and the version of Python that the plugin uses.

All that matters to the plugin is which version of the PythonScript plugin you use, and which python###.dll the plugin provides. It doesn’t matter which python.exe you might have installed elsewhere on your computer, whether they are Python 2.7 or Python 3.10 or anything else (and, in fact, the plugin ignores other installations unless you have checkmarked the ☑ Prefer installed Python libraries plugin option; and if you have that option checked, the installed version must match the 3.xx that PythonScript was compiled against).

PythonScript plugin v2.0.0.0 uses Python 2.7.
- There is no plugin v2.7.
PythonScript plugin v3.0.xx uses Python 3.yy.
- The plugin hasn’t gotten to v3.10 yet
- It’s currently on PythonScript v3.0.18 using Python 3.12.3
- you might want to give PythonScript v3.0.18 a try, because it seems pretty stable

Both the plugin version and the python version are available in Plugins > PythonScript > About and printed to the console when you first open Plugins > PythonScript > Show Console

Ekopalypse

Hello Peter,
many thanks for the script.

Two comments:

a) Themes with commented LexerTypes cause an exception when sorting.

Not really tested, but you could think about something like

        def sort_key(child):
            child_name = child.get('name')
            return (False, '') if child_name is None else (child_name == 'searchResult', child_name)

        elThemeLexerStyles[:] = sorted(elThemeLexerStyles, key=sort_key)

b) Themes without top-level comments receive the comments from the previous theme. For example, I have a theme called mytheme.xml that now contains the copyright comments from mossylawn.xml.

PeterJones

@Ekopalypse ,

Thanks for those catches. v1.03 is now available at the permalink, and should fix both of those issues

Lycan Thrope

@PeterJones ,
The first is what mine shows.
PythonScript plugin v2.0.0.0 uses Python 2.7.

Python 2.7.18 (v2.7.18:8d21aa21f2, Apr 20 2020, 13:25:05) [MSC v.1500 64 bit (AMD64)]
Copyright (c) 2001-2020 Python Software Foundation.
All Rights Reserved.

I have Python27.dll in the C:\Program Files\Notepad++\plugins\PythonScript - directory.
And, I didn’t have the Prefer option checked, in the Options menu.
Just for confirmation.

PeterJones

v1.06 is available from the permalink:

v1.05 (2024-Sep-05): Add URLs to source code, so you remember where to get an updated copy. (was not announced, since it was not a critical feature/bugfix upgrade)
v1.06 (2024-Sep-14): IMPROVEMENT = error handling for <installed>\themes\ permissions
- The most common error is that you don’t have write permissions for the default themes, which are installed in the installation directory instead of your AppData or other config directory.
- Users have been confused when it doesn’t complete because of write permissions, because I forgot that not everyone has PythonScript set to show the Console if there’s an error, and that error isn’t intuitive to the general audience even if they look in the Console.
- I now trap that error, and I’ve added a MessageBox to inform you of the error, letting you know that if you re-run Notepad++ As Admin, then it should work.
  - Some users have found that when they run As Admin, PythonScript no longer sees their user scripts from the AppData hierarchy. If that’s the case, just save another copy of the script in the Machine Scripts section while you’re running As Admin.
- Even if you get the write error on the <installed>\themes, your AppData or other config-directory stylers and themes and langs.xml should all be properly updated. (If you use Windows Explorer, you should be able to see the last-modified time on the AppData or other config-directory files has been updated after running the script.)

PeterJones

v1.07 is available from the permalink:

fix global styles
1. copy global-styles colors from stylers.model.xml to stylers.xml (similar to what’s already done on stylers.xml’s lexer-styles)
2. when updating/adding global styles in either stylers.xml or themes\*.xml, make sure global-styles don’t have extra attributes that confuse the StyleConfigurator interface

running v1.07 will clean up the Global Styles attributes to match the stylers.model.xml, even if you’ve run a previous version of the script

notdodgeball

@PeterJones, I used version 1.07 of the script and the copyright information on top got removed from the theme files.

They were also unmodified, straight from the install directories, version Portable v8.7.1

PeterJones

@notdodgeball ,

Apparently the xml.etree library behaves differently between Python 2.7 and 3.12 (used by PS2 and PS3 respectively).

I really wish PS3 would move out of alpha and release into Plugins Admin, so that we could just stick with PythonScript 3 for sharing scripts in the forum, and not have to contend with all the headaches of keeping scripts compatible with two major versions of Python, and all the library differences that come with that.

(it also would’ve been nice if Don hadn’t allowed theme authors to submit themes with XML that have invalid top-level comments, which the XML parsing engines don’t like because it’s not valid XML)

v1.08 is available from the permalink, fixing the handling of the invalid XML in PythonScript 2.

However, I highly recommend making the switch to PythonScript 3, despite it claiming to be alpha, unless you are using a lot of ANSI characterset files rather than real unicode-based encodings. I don’t know how much longer I’ll support PythonScript 2 for this script.