Enhance UDL lexer

  • Another version which has code cleanup and comments translated by deepl.com

    I think as long as nobody finds bugs or suggests new features,
    this is the version that can serve as a template for UDL extensions.

    # -*- coding: utf-8 -*-
    from Npp import editor, editor1, editor2, notepad, NOTIFICATION, SCINTILLANOTIFICATION, INDICATORSTYLE
    import ctypes
    import ctypes.wintypes as wintypes
    from collections import OrderedDict
    regexes = OrderedDict()
    # ------------------------------------------------- configuration area ---------------------------------------------------
    # Define the lexer name exactly as it can be found in the Language menu
    lexer_name = 'BR! Source'
    # Definition of colors and regular expressions
    #   Note, the order in which regular expressions will be processed
    #   is determined by its creation, that is, the first definition is processed first, then the 2nd, and so on
    #   The basic structure always looks like this
    #   regexes[(a, b)] = (c, d)
    #   regexes = an ordered dictionary which ensures that the regular expressions are always processed in the same order
    #   a = a unique number - suggestion, start with 0 and always increase by one
    #   b = color in the form of (r,g,b) such as (255,0,0) for the color red
    #   c = raw byte string, describes the regular expression. Example r'\w+'
    #   d = number of the match group to be used
    # Examples:
    #   All found words which may consist of letter, numbers and the underscore,
    #   with the exception of those that begin with fn, are displayed in a blue-like color.
    #   The results from match group 1 should be used for this.
    regexes[(0, (79, 175, 239))] = (r'fn\w+\$|(\w+\$)', 1)
    #   All numbers are to be displayed in an orange-like color, the results from
    #   matchgroup 0, the standard matchgroup, should be used for this.
    regexes[(1, (252, 173, 67))] = (r'\d', 0)
    # Definition of which area should not be styled
    # 1 = comment line style
    # 2 = comment style
    # 16 = delimiter1
    # ...
    # 23 = delimiter8
    excluded_styles = [1, 2, 16, 17, 18, 19, 20, 21, 22, 23]
    # ------------------------------------------------ /configuration area ---------------------------------------------------
    except NameError:
        user32 = wintypes.WinDLL('user32')
        WM_USER = 1024
        NPPMSG = WM_USER+1000
        SC_INDICVALUEBIT = 0x1000000
        class SingletonEnhanceUDLLexer(type):
                Ensures, more or less, that only one
                instance of the main class can be instantiated
            _instance = None
            def __call__(cls, *args, **kwargs):
                if cls._instance is None:
                    cls._instance = super(SingletonEnhanceUDLLexer, cls).__call__(*args, **kwargs)
                return cls._instance
        class EnhanceUDLLexer(object):
                Provides additional color options and should be used in conjunction with the built-in UDL function.
                An indicator is used to avoid style collisions.
                Although the Scintilla documentation states that indicators 0-7 are reserved for the lexers,
                indicator 0 is used because UDL uses none internally.
                Even when using more than one regex, it is not necessary to define more than one indicator
                because the class uses the flag SC_INDICFLAG_VALUEFORE.
                See https://www.scintilla.org/ScintillaDoc.html#Indicators for more information on that topic
            __metaclass__ = SingletonEnhanceUDLLexer
            def __init__(self):
                    Instantiated the class,
                    because of __metaclass__ = ... usage, is called once only.
                editor.callbackSync(self.on_updateui, [SCINTILLANOTIFICATION.UPDATEUI])
                notepad.callback(self.on_langchanged, [NOTIFICATION.LANGCHANGED])
                notepad.callback(self.on_bufferactivated, [NOTIFICATION.BUFFERACTIVATED])
                self.doc_is_of_interest = False
                self.lexer_name = None
                self.npp_hwnd = user32.FindWindowW(u'Notepad++', None)
            def rgb(r, g, b):
                    Helper function
                    Retrieves rgb color triple and converts it
                    into its integer representation
                        r = integer, red color value in range of 0-255
                        g = integer, green color value in range of 0-255
                        b = integer, blue color value in range of 0-255
                return (b << 16) + (g << 8) + r
            def paint_it(color, pos, length):
                    This is where the actual coloring takes place.
                    Color, the position of the first character and
                    the length of the text to be colored must be provided.
                    Coloring occurs only if the position is not within the excluded range.
                        color = integer, expected in range of 0-16777215
                        pos = integer,  denotes the start position
                        length = integer, denotes how many chars need to be colored.
                if pos < 0 or editor.getStyleAt(pos) in excluded_styles:
                editor.indicatorFillRange(pos, length)
            def style(self):
                    Calculates the text area to be searched for in the current document.
                    Calls up the regexes to find the position and
                    calculates the length of the text to be colored.
                    Deletes the old indicators before setting new ones.
                start_line = editor.getFirstVisibleLine()
                end_line = editor.docLineFromVisible(start_line + editor.linesOnScreen())
                start_position = editor.positionFromLine(start_line)
                end_position = editor.getLineEndPosition(end_line)
                editor.indicatorClearRange(0, editor.getTextLength())
                for color, regex in self.regexes.items():
                                    lambda m: self.paint_it(color[1],
                                                            m.span(regex[1])[1] - m.span(regex[1])[0]),
            def configure(self):
                    Define basic indicator settings, the needed regexes as well as the lexer name.
                editor1.indicSetStyle(0, INDICATORSTYLE.TEXTFORE)
                editor1.indicSetFlags(0, SC_INDICFLAG_VALUEFORE)
                editor2.indicSetStyle(0, INDICATORSTYLE.TEXTFORE)
                editor2.indicSetFlags(0, SC_INDICFLAG_VALUEFORE)
                self.regexes = OrderedDict([ ((k[0], self.rgb(*k[1]) | SC_INDICVALUEBIT), v) for k, v in regexes.items() ])
                self.lexer_name = u'User Defined language file - %s' % lexer_name
            def check_lexer(self):
                    Checks if the current document is of interest
                    and sets the flag accordingly
                language = notepad.getLangType()
                length = user32.SendMessageW(self.npp_hwnd, NPPM_GETLANGUAGEDESC, language, None)
                buffer = ctypes.create_unicode_buffer(u' ' * length)
                user32.SendMessageW(self.npp_hwnd, NPPM_GETLANGUAGEDESC, language, ctypes.byref(buffer))
                self.doc_is_of_interest = True if buffer.value == self.lexer_name else False
            def on_bufferactivated(self, args):
                    Callback which gets called every time one switches a document.
                    Triggers the check if the document is of interest.
                        provided by notepad object but none are of interest
            def on_updateui(self, args):
                    Callback which gets called every time scintilla
                    (aka the editor) changed something within the document.
                    Triggers the styling function if the document is of interest.
                        provided by scintilla but none are of interest
                if self.doc_is_of_interest:
            def on_langchanged(self, args):
                    Callback gets called every time one uses the Language menu to set a lexer
                    Triggers the check if the document is of interest
                        provided by notepad object but none are of interest
            def main(self):
                    Main function entry point.
                    Simulates two events to enforce detection of current document
                    and potential styling.

  • Hi, @eko-palypse, and All,

    I noticed something really weird, regarding styling. This happens, whatever your script version !

    If the Word wrap option is ON and that you’re looking at some code of an UDL language, if this code is near the top editor window, one of the new stylings, produced by the script, may disappear if you zoom in, at least, 9 times on Ctrl + + and other stylings, if any, may also disappear if you go on zooming in, till the maximum !

    Note that scrolling this part of code, downwards, makes styling(s) to be present, again !

    Eko, can you reproduce that issue ? May be, it could be because of my weak Win XP configuration ?

    This happens for code, let’s say, on the first or second visible lines and only if the Word wrap is set ? When the Word wrap option is unset, script stylings seem permanent, as expected ;-))



  • Hi @guy038,

    thanks again for testing and yes I can confirm, this happens for me as well.
    The reason why this happens is that editor.getFirstVisibleLine returns
    an incorrect line. Example: with normal zoom the first visible line might be 7.
    But when zooming in and without scrolling the document, at some point
    getFirstVisibleLine returns suddenly 8 and the maximum I got was 141.
    141 got returned when choosing the maximum zoom level and reducing the
    document width so that only the first column was visible.
    I have to check scintilla documentation but I can’t remember that there is
    an method for this purpose available.

    Unfortunately, while investigating this behavior I found another feature.
    When using both views and having a UDL document open in each of them
    then only the document with the focus does get styled. Could be explained
    as the ui update callback is handled on the current active window only but what it
    makes it worse is the following situation.

    Les’s assume you are working in one of the two documents and you click into
    the other and now you decide to scroll the first document without
    activating/clicking into it you won’t see any styling for the new scrolling in text.
    Can also be explained by the current design but isn’t nice …
    I guess it have to go back to the drawing board.

  • I have all files zipped.
    I will now test the last template.

  • @Gerald-Kirchner

    am I correct to say that the line /* block comment in one line, forbidden */
    should be colored?
    If so, then you have to remove style 1 from excluded style list, like
    excluded_styles = [2, 16, 17, 18, 19, 20, 21, 22, 23]

  • I have 2 removed. 1 is line style.

  • Ich schreibe jetzt einfach mal auf Deutsch, da müssen die anderen durch. :-)
    Ich hoffe ich liege dabei auch richtig.

    Die exclusion Liste soll dafür dienen, RegexMatches in Bereichen welche bereits von
    anderen Styles genutzt werden zu ignorieren.
    In Ihrem Fall wollen Sie aber genau hier ansetzen, da diese Art von CommentBlock
    wohl falsch ist, daher muß Style 1 herausgenommen werden damit der Match auch
    farblich dargestellt wird.
    Ich hoffe ich habe mich einigermaßen deutlich ausgedrückt.

  • Dann mal auf Deutsch. Das fällt mir leichter :-)
    Ja, es funktioniert mit weglassen der 1. Auch mit der Vorlage aus /enhance-udl-lexer/7.
    Aber ich begreife nicht warum.
    Definition of which area should not be styled
    übersetze ich mit
    Festlegung, welcher Bereich nicht gestaltet werden soll
    comment line style is ; <-- der soll so bleiben
    comment style is /* */ <-- den will ich ändern

    <Keywords name="Comments">00; 01 02 03/* 04*/</Keywords>

  • Das fällt mir leichter :-)

    Mir ebenfalls und erspart den Umweg über DeepL.com :-)

    Ich verstehe die Verwirrung. Eigentlich ist es genauso gedacht.
    Zum Beispiel: Sie suchen nach Wörter die meinetwegen auf $ enden aber nur dann
    wenn sie nicht in einem Kommentarfeld sind. Dann macht die exclusion Liste diesen Filter. Da das Wort in einem Kommentarfeld gefunden wurde und entweder Style1 der 2
    benutzt würde der Match ignoriert.
    Bei Ihnen ist der Anwendungsfall nun gerade umgekehrt. Sie wollen die Zeilen finden,
    welche den Style 1 benutzt aber fehlerhaft geschrieben wurde, da nur in einer Zeile
    und nicht wie vorgesehen über mehrere Zeilen. Nun darf die exclusion Liste diesen Style nicht mehr enthalten damit dies auch farblich dargestellt wird.

  • Nach der Definition

    Definition of which area should not be styled
    1 = comment line style
    2 = comment style

    will ich aber genau die 2 bearbeiten. Und nicht die 1.
    Kann es sein, das die Definition eventuell mit 0 beginnt und bei mir deshalb die 1 als “comment style” passt?
    In welcher Quelle findet man diese Zuweisung?

  • Der Source Code zu UDL ist hier einzusehen.

    Aber das Problem liegt bei mir.
    Ich habe comment style id und comment line style id verwechselt.
    Es muß genau anders herum sein

    1 = comment style
    2 = comment line style

    Sorry, verdameleite …

  • English, anyone?

    Normally I wouldn’t complain, except Chrome right-click Translate to English failed on this.

  • @Ekopalypse

    I’d say the burden is on posters to post in English, not others to translate later. (AFAIK, the accepted language of this forum in English.)

  • @Alan-Kilborn

    Manager summary :-)

    I failed at the documentation by stating that comment style is using id 2 and
    comment line style is using 1 whereas it is exactly the opposite.

  • @Alan-Kilborn

    (AFAIK, the accepted language of this forum in English.)

    maybe one accepted language as we see more often :-)

  • But thanks, @Alan-Kilborn,

    Chrome used to be my backup browser, but I’ve recently switched to it being my primary, and I didn’t know there was an easy RClick>Translate option in Chrome. When I ran it on the German on this page, it translated okay for me. (Which is good, because my 2 years of highschool German from more than a quarter-century ago just didn’t cut it.)

    But I do agree that the forum’s default language is English, and if people want help here, they should at least provide the translation for us. I understand that some of the posters might be more comfortable (or equally comfortable) in German; and it’s fine for someone fluent in German to answer in German; but also provide a translation to English, so the other people who read this thread – both for those who have started to help, and got lost when it switched language; and for those who come here searching for a solution to a similar problem, and find that the actual answer is buried in another language, when it started in English.

  • In the source code I find the place where the numbers are assigned or do not arise.
    I would like to find that to be able to understand it.
    Then the template would have to be adjusted.

  • Ok, :-D

    the outcome of the discussion is that I need to post another version of the script
    which hopefully adresses @guy038 findings as well,

    So dear future reader,
    if you have read until here - forget about everything you read and hope that you will find
    a new version of the script as another post after this one.

  • @Ekopalypse

    omg … you are german speaking … that’s the last thing i would have expected … your name sounded so apocalyptically greek 😉

Log in to reply