Assign Language by Line



  • @Alan-Kilborn

    she recently posted at pythonscript github so I’m hoping that she might find her
    way back home and of course, I’m always looking for python related stuff :-D



  • @Valley-Moose

    Just in case you wondered what happend, well, just real life or better wife :-)

    I guess I have something you can play with.
    The save option, meaning remembering what was colored where and how, needs to be discussed.

    I see three option

    1. use an additional file with the same name as the current file but with a different unique extension, which contains colors and position
    2. use one file which contains the names of the files which were used and there colors and position
    3. not using any file at all, but a marker in the original file which indicates where a different languages starts and stops.

    Concerning the script itself.
    You have to install python script plugin and, for the time being, avoid updating to npp 7.7 or newer unless PythonScript gets updated to reflect the changes made in npp7.7
    Once PS is installed you have to call it for every document, which by the way needs to be set to normal text, in which you want to see different languages colored once. Then select the text and use the language menu to chose the lexer of your choice. Done.

    Two remarks.
    UDLs are currently supported by a hack. When using an UDL the script opens a new tab, pastes the text, sets the udl lexer, gets the colors and closes the tab, whereas if using a builtin lexer a hidden scintilla component is used.

    Only foreground colors are supported, meaning no font settings like bold or italic. That is because indicators do not support these.
    Using indicators instead of styles has the advantage that switching
    between documents is possible without loosing the styling information.

    The script, which I will post in the next post, itself contains TESTDATA.
    If you want to use it, copy everything from line 291 to 360 and paste it into a new tab.

    Is it bug free? Most likely not.
    In case your finding something weird or in case it doesn’t do what you want it to do, let me know.



  • # -*- coding: utf-8 -*-
    
    from Npp import editor, notepad, INDICATORSTYLE, INDICFLAG, LANGTYPE, NOTIFICATION
    import xml.etree.ElementTree as et
    import os
    import json
    from pprint import pformat
    
    try:
        TextStyler().register_document()
    except NameError:
    
        SC_INDICVALUEBIT = 0x1000000
        INDICATOR_ID = 8
    
    
        class SingletonTextStyler(type):
            '''
                Ensures, more or less, that only one
                instance of the main class can be instantiated
            '''
            _instance = None
            def __call__(cls, *args, **kwargs):
                if cls._instance is None:
                    cls._instance = super(SingletonTextStyler, cls).__call__(*args, **kwargs)
                return cls._instance
    
    
        class TextStyler(object):
    
            __metaclass__ = SingletonTextStyler
    
            def __init__(self):
                self.hidden_scintilla = notepad.createScintilla()
                editor1.indicSetStyle(INDICATOR_ID, INDICATORSTYLE.TEXTFORE)
                editor1.indicSetFlags(INDICATOR_ID, INDICFLAG.VALUEFORE)
                editor2.indicSetStyle(INDICATOR_ID, INDICATORSTYLE.TEXTFORE)
                editor2.indicSetFlags(INDICATOR_ID, INDICFLAG.VALUEFORE)
                self.npp_config_dir = notepad.getNppDir()
                self.styler_dict = dict()
                self.active_theme = et.parse(self.get_active_styler_xml())
                # self.udls = self.create_udls_xml()
                self.lexer_keywords = self.get_langs_xml()
                self.list_of_document_ids = []
                self.current_lexer = 'null'
                self.default_id = 0
                self.memory_file = os.path.join(notepad.getPluginConfigDir(), 'TextStylerMemory.txt')
                self.remember_styles = None
                notepad.callback(self.on_langchanged, [NOTIFICATION.LANGCHANGED])
                notepad.callback(self.on_fileclosed, [NOTIFICATION.FILECLOSED])
                notepad.callback(self.on_filesaved, [NOTIFICATION.FILESAVED])
                notepad.callback(self.on_shutdown, [NOTIFICATION.SHUTDOWN])
    
    
            # there is an peformance issue when calling paint_it directly,
            # therefore we are hacking around langchanged notification
            def on_langchanged(self, args):
                ''' Gets called by npp when language change is deteced
                    Resets to null lexer and calls main method
                '''
                if (args['bufferID'] in self.list_of_document_ids):
                    if notepad.getLangType() == LANGTYPE.TXT:
                        if self.current_lexer != 'null':
                            self.main(self.current_lexer)
                        else:
                            notepad.messageBox('Expected a lexer but null found', 'Error',0)
                    else:
                        self.current_lexer = editor.getLexerLanguage()
                        if self.current_lexer == 'user':
                            self.current_lexer = notepad.getLanguageName(notepad.getLangType())
                        notepad.setLangType(LANGTYPE.TXT)
    
    
            def on_shutdown(self, args):
                ''' Gets called by npp when shutting down
                    TODO: store knwon files and their colors which
                    then would be reused on fileopen/bufferactived callback
                '''
                notepad.destroyScintilla(self.hidden_scintilla)
    
    
            def on_filesaved(self, args):
                ''' Gets called by npp when a buffer is saved
                    TODO:
                        check if doc is of interest
                          if so, get current filename
                          check if doc is empty
                            if so delete from memory
                            if not store current colors and their position
                '''
                if (args['bufferID'] in self.list_of_document_ids):
                    current_file = notepad.getCurrentFilename()
                    from pprint import pformat
                    values = [editor.indicatorValueAt(INDICATOR_ID,x) for x in range(editor.getTextLength())]
                    prev_value = 0
                    fill_range = 1
                    sorted_values = []
                    for offset, value in enumerate(values):
                        if value == 0:
                            continue
                        if prev_value == value:
                            fill_range += 1
                        else:
                            sorted_values.append((value, offset-fill_range, fill_range))
                            prev_value = value
                            fill_range = 1
    
                    print(pformat(sorted_values))
    
    
            def on_fileclosed(self, args):
                ''' Gets called by npp when a buffer gets closed
                    Removes unneeded buffer ids from internal list
                '''
                if (args['bufferID'] in self.list_of_document_ids):
                    self.list_of_document_ids.remove(args['bufferID'])
    
    
            def get_active_styler_xml(self):
                ''' returns the xml path of the current theme '''
                xml_file = os.path.join(self.npp_config_dir, 'config.xml')
                xml_doc = et.parse(xml_file)
                return xml_doc.find('GUIConfigs/GUIConfig[@name="stylerTheme"]').get('path')
    
    
            # def create_udls_xml(self):
                # ''' merge all udls from userDefineLangs into one big xml '''
                # udls_dir = os.path.join(self.npp_config_dir, 'userDefineLangs')
                # udl_files = os.listdir(udls_dir)
                # root = et.Element('NotepadPlus')
                # for udl_file in udl_files:
                    # xml_doc = et.parse(os.path.join(udls_dir, udl_file))
                    # user_lang = xml_doc.find('UserLang')
                    # root.append(user_lang)
                # return root
    
    
            def get_lexer_styles(self, lexername):
                ''' Creates the styling dictionary by reading the styler xml
                '''
                if lexername.startswith('udf - '):
                    return  # as long as hidden scintilla doesn't do the styling we are done here
                    # tag = 'UserLang[@name="%s"]/Styles/WordsStyle' % lexername[6:]
                    # lexer_styles = self.udls.findall(tag)
                    # self.default_id = 0
                    # for id, result in enumerate(lexer_styles):
                        # fgColor = result.attrib.get('fgColor', None)
                        # if not fgColor:
                            # notepad.messageBox('Received unexpected value\nid: {}\nfgColor: {}'.format(id, fgColor), 'Error',0)
                            # return
    
                        # red, green, blue = bytearray.fromhex(fgColor)
                        # color = (blue<<16) + (green<<8)  + red
                        # self.styler_dict[int(id)] = color
    
                tag = 'LexerStyles/LexerType[@name="%s"]/WordsStyle' % lexername
                lexer_styles = self.active_theme.findall(tag)
                for result in lexer_styles:
                    if result.attrib.get('name', None) == 'DEFAULT':
                        id = result.attrib.get('styleID', None)
                        self.default_id = int(id)
                    else:
                        id = result.attrib.get('styleID', None)
                    fgColor = result.attrib.get('fgColor', None)
                    if not fgColor:
                        notepad.messageBox('Received unexpected value\nid: {}\nfgColor: {}'.format(id, fgColor), 'Error',0)
                        return
    
                    red, green, blue = bytearray.fromhex(fgColor)
                    color = (blue<<16) + (green<<8)  + red
    
                    self.styler_dict[int(id)] = color
    
    
            def get_langs_xml(self):
                ''' returns a dictionary which contains all keywords
                    for all defined builtin languages
                '''
                xml_file = os.path.join(self.npp_config_dir, 'langs.xml')
                xml_doc = et.parse(xml_file)
                lexer_keywords = dict()
                for lang in xml_doc.findall('Languages/Language'):
                    lexer_keywords[lang.attrib['name']] = lang.findall('Keywords')
                return lexer_keywords
    
    
            def paint_it(self, color, start, length):
                ''' Does the coloring by using an indicator '''
                editor.setIndicatorCurrent(INDICATOR_ID)
                editor.setIndicatorValue(color | SC_INDICVALUEBIT)
                editor.indicatorFillRange(start, length)
    
    
            def main(self, lexer):
                ''' Gathers the needed information for the current selected lexer
                    and sets these in the hidden scintilla.
                    Calls hidden scintillas colourise method and retrieves the
                    needed colors and their position.
                    Finally sets it in the active document
                '''
                text = editor.getSelText()
                text_length = len(text)
                if not text_length:
                    notepad.messageBox('Nothing selected!', 'Error',0)
                    return
    
                if not lexer:
                    notepad.messageBox('lexer expected, received: {}'.format(lexer), 'Error',0)
                    return
    
                offset = min(editor.getSelectionStart(), editor.getSelectionEnd())
    
                # unfortunately udls behave different than builtin lexers
                # there for this hack until the secrets get revealed
                if lexer.startswith('udf - '):
                    active_buffer = notepad.getCurrentBufferID()
                    notepad.new()
                    editor.setText(text)
                    notepad.runMenuCommand('Language', lexer[6:])
                    styles = [(pos, editor.styleGetFore(editor.getStyleAt(pos))) for pos in range(text_length)]
                    editor.undo()
                    notepad.close()
                    notepad.activateBufferID(active_buffer)
    
                    for position, color in styles:
                        self.paint_it(color[0] + (color[1] << 8) + (color[2] << 16),
                                      offset+position,
                                      1)
                    return  # we are done here
    
                # else part - aka builtin lexer
                self.get_lexer_styles(lexer)
                keywords = self.lexer_keywords.get(lexer, None)
    
                self.hidden_scintilla.styleClearAll()
                self.hidden_scintilla.setLexerLanguage(lexer)
                if keywords:
                    # not all lexers do have keywords, e.g. xml lexer
                    for i, _keywords in enumerate(keywords):
                        self.hidden_scintilla.setKeyWords(i, _keywords.text)
    
                self.hidden_scintilla.setText(text)
                self.hidden_scintilla.colourise(0, text_length)
                styles = [(pos, self.hidden_scintilla.getStyleAt(pos)) for pos in range(text_length)]
    
                color = self.styler_dict.get(self.default_id, None)
                if color is None:
                    return
    
                self.paint_it(color, offset, text_length)  # TODO: can default color different
    
                previous_style = []
                sorted_styles = dict()
                for pos, style in styles:
                    if previous_style:
                        if previous_style[0] == style:
                            previous_style[2] += 1
                        else:
                            if previous_style[0] != 0:
                                if previous_style[0] in sorted_styles:
                                    sorted_styles[previous_style[0]].append((previous_style[1],previous_style[2]))
                                else:
                                    sorted_styles[previous_style[0]] = [(previous_style[1],previous_style[2])]
    
                            previous_style = [style, pos, 1]
                    else:
                        previous_style = [style, pos, 1]
    
                for k, v in sorted_styles.items():
                    color = self.styler_dict.get(k, None)
                    if not color:
                        continue
                    for _v in v:
                        self.paint_it(color, offset+_v[0], _v[1])
    
    
            def register_document(self):
                ''' store current buffer id in internal list '''
                id = notepad.getCurrentBufferID()
                if id not in self.list_of_document_ids:
                    self.list_of_document_ids.append(id)
    
        TextStyler().register_document()
    
    
    print('done')
    
    # --------------------------------------------------------------------------
    # TESTDATA
    TESTDATA = '''
    Start with an perl example
    
    use strict;  # comment
    use warnings;
    print "Hello, World!\n";
    
    ------------------------------------------------------------------------------
    here we have an cpp example
    
    #include <iostream>
    using namespace std; // line comment
    
    int main()
    {
        /* multi line
        comment */
        cout << "Hello, World!";
        return 0;
    }
    
    ------------------------------------------------------------------------------
    followed by an rust example
    
    fn main() {
        // Print text to the console
        println!("Hello World!");
    }
    
    ------------------------------------------------------------------------------
    an xml example
    
    <tags>
        <tag id="0" value="text" />
    </tags>
    
    ------------------------------------------------------------------------------
    an python example
    
    class test(object):
        def __init__(self):
            self.greet = 'Hello World!'  # comment
    
        def do_greet(self):
           print(self.greet)
    
    ------------------------------------------------------------------------------
    and finally udl markdown
    
    \1-2-3 *multiple italic*, **multi bold** and ***multi bold italic***
    
    I have  *a pen **I have an apple**  uuh*  Apple pen.
    I have **a pen *I have a pineapple* uuh** Pineapple pen!
    
    I have  *a pen __I have an apple__  uuh*  Apple pen.
    I have **a pen _I have a pineapple_ uuh** Pineapple pen!
    
    \* apple \* pen \* <!-- escape test \* t \* t \* t -->
    
    ```js
    var a = 1 * 1
    ```
    ****** <!-- hr -->
    
    ## Bullet point test
    
    - normal text
    
    * This should be normal text
    * mess up *some thing* (\* suppose \* it's \* normal)
    * This should be normal text
    '''
    # --------------------------------------------------------------------------
    

Log in to reply