Community
    • Login

    Trouble making a functionList parser for MATLAB

    Scheduled Pinned Locked Moved Help wanted · · · – – – · · ·
    29 Posts 4 Posters 18.6k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Per IsaksonP
      Per Isakson
      last edited by

      @MAPJe71

      The “block comment problem” disappeared as a result of changes of the <parser>, which have no obvious coupling to comments. Strange!

      MATLAB syntax allows continuation lines. Could you give an example?

      function ...
          WeirdUseOfContinuationLine ( ...
          str )
          disp( str )
          % This runs as expexted.
      end
      

      I guess this could be handled with regex, but I will not try.

      1 Reply Last reply Reply Quote 0
      • MAPJe71M
        MAPJe71
        last edited by

        @Per-Isakson

        The “block comment problem” disappeared as a result of changes of the <parser>

        Strange indeed.
        I presume you applied the changes, could you post the updated parser (even if it’s not the final one)?

        I guess this could be handled with regex

        Possibly.

        but I will not try.

        Wise, keep it as challenge for an other revision :-)

        1 Reply Last reply Reply Quote 0
        • Per IsaksonP
          Per Isakson
          last edited by

          Hello @guy038 and @MAPJe71

          Thank you for your answers, especially on Nested Pairs and recursion. Now I have a <parser>, which doesn’t rely on the indentation of the Matlab code and produces correct results for my first set of test files. However, keywords embedded in comments and in string values make the <parser> fail.

          A new <parser> and some test files

          <parser 
              id          = "matlab" 
              displayName = "Matlab Node" 
              commentExpr =  "(?x)                    # Utilize inline comments (see `RegEx - Pattern Modifiers`)
                              (?s:%\{.*%\})           # Multi Line Comment
                              |(?m-s:\h*%.*$)         # Single Line Comment
                              "
          >    
              <classRange
                  mainExpr = "(?x)(?s)                                    # dot matches new line
                              (?-i)                                       # case sensitive
                              (                                           # --- open 1st group 
                                  \b                                      # word boundary  
                                  (                                       # --- open 2nd group
                                          classdef                        # keywords that open a 
                                      |   e(?:numeration|vents)           # Balanced Construct
                                      |   f(?:or|unction)                 # that is closed by 
                                      |   if                              # 'end'.
                                      |   methods                         #
                                      |   p(?:arfor|roperties)            #
                                      |   switch                          #
                                      |   try                             #
                                      |   while                           #
                                  )                                       # --- close 2nd group
                                  \b                                      # word boundary 
                              )                                           # --- close 1st group
                                                  
                              (                                           # open 3rd capturing group    
                                  (?:                                     # open non-capturing group
                                      (?!                                 # negative look-ahead; if not a keyword
                                          \b                              # word boundary  
                                          (                               # --- open 4th group
                                                  classdef                # keywords, of the 2nd group + 'end',
                                              |   e(?:numeration|vents)   # which must NOT occur,
                                              |   f(?:or|unction)         # at ANY position, of the
                                              |   if                      # present sub-block, till
                                              |   methods                 # its associated 'end' closing word
                                              |   p(?:arfor|roperties)    #
                                              |   switch                  #
                                              |   try                     #
                                              |   while                   #
                                              |   end                     #
                                          )                               # --- close 4th group
                                          \b                              # word boundary 
                                      ).                                  # then one character
                                  )+                                      # repeat 
                                  |                                       # until keyword found
                                  (?R)                                    # recurse RE from start
                              )+                                          # repeat 3rd group
                              \bend\b                                     #      
                              "
              >
                  <className>
                      <nameExpr 
                          expr = "(?x)                    # ok, 10:07 2017-03-25
                                  ^\h*                    # optional whitespace
                                  classdef                # start-of-class indicator
                                  \h+                     # whitespace
                                  (                       #
                                      \([^\)]*?\)         # (ClassAttributes)
                                      \h+                 # whitespace
                                  )?                      #
                                  \K                      # discarge what's matched so far 
                                  [A-Za-z]\w*             # valid identifier, i.e. class name
                                  "
                      />
                  </className>
                  <function
                      mainExpr = "(?x)                    # ok, 10:22 2017-03-25, [2]
                                  (?m)                    # ^ and $ match at line breaks
                                  (   
                                      (^\h*)              # optional whitespace starting line
                                      |                   # or
                                      ([,;]\h*)           # separator before 'function'
                                  )       
                                  function                # start-of-function indicator
                                  \h+                     # whitespace
                                  \K                      # discard what's matched so far 
                                  (                       # open optional left hand side, LHS
                                      (                   #
                                          (\w+)           # single variable name
                                          |               # or
                                          (\[[\w,\h]*\])  # one or more variable names in brackets 
                                      )                   #
                                      \h*=\h*             # 
                                  )?                      # close optional left hand side
                                  [A-Za-z]\w*\b           # function name
                                  \h*                     #      
                                  (                       # open optional argument list
                                      (\([\h,\w~]*\))     # pair of parentheses with optional arguments 
                                      |                   # or
                                      (                   #  
                                          \([\h,\w~]*?    # opening parenthisis and optional arguments
                                          (?=\h*\.{3})    # and closed by continuation line
                                      )                   #
                                  )?                      # close optional argument list
                                  "
                  >
                      <functionName>
                          <funcNameExpr 
                              expr = "(?x)                # ok, 12:07 2017-03-25
                                      (?m)                # ^ and $ match at line breaks
                                      (?-s)               # single-line  
                                      (                   # open optional left hand side, LHS
                                      ((\w+)|             # single variable name
                                      (\[[\w,\h]*\]))     # one or more variable names in brackets 
                                      \h*=\h*             # 
                                      )?                  # close optional left hand side
                                      [A-Za-z]\w*\b       # function name
                                      \h*                 #      
                                      (                   # open optional argument list
                                          (\([\h,\w~]*\)) # pair of parenthisis with optional arguments
                                          |               # or
                                          (\([\h,\w~]*?   # lazy to avoid trailing whitespace 
                                          (?=\h*\.{3}))   # continuation line
                                      )?                  # close optional argument list
                              "
                          />
                      </functionName>
                  </function>
              </classRange>
              <function
                  mainExpr = "(?x)            # [3]
                              ^\h*            # optional whitespace
                              function        # start-of-function indicator 
                              \h+             # whitespaces
                              (?-s:.*)        # the rest of the line
                              "
              >
                  <functionName>
                      <nameExpr 
                          expr = "(?x)                    #
                                  function                # start-of-function indicator 
                                  \h+                     # whitespace        
                                  \K                      # discard what's matched so far
                                  (                       # open optional left hand side, LHS
                                      (                   #
                                          (\w+)           # single variable name
                                          |               # or
                                          \[[\w,\h]*\]    # one or more variable names in brackets 
                                      )                   #
                                      \h*=\h*             # 
                                  )?                      # close optional left hand side
                                  [A-Za-z]\w*             # function name
                                  \b\h*                   #      
                                  (                       # open optional input argument list
                                      \([\h,\w~]*\)       # pair of parenthisis with optional arguments or
                                      |                   # or     
                                      (                   #
                                          \([\h,\w]*?     # lazy to avoid trailing whitespace 
                                          (?=\h*\.{3})    # 
                                      )                   #
                                  )?                      # close optional argument list
                                  "
                      />
                  </functionName>
              </function>
          </parser>
          

          function_with_keyword_in_comment.m. The <parser> fails with this test file. Only the file name is displayed in Function List.

          function    [ out, val ] = function_with_keyword_in_comment( val )
              out = 17;
              % comment with keyword: while
          end
          

          class_with_keyword_in_block_comment.m. The <parser> fails with this test file. Only the file name is displayed in Function List.

          classdef    class_with_keyword_in_block_comment
              methods
                  function    no_return( this )
                      %{
                          comment with keyword: while
                      %}
                      a = 17;
                  end        
              end
          end
          

          class_with_keyword_in_string.m. The <parser> fails with this test file. Only the file name is displayed in Function List.

          classdef    class_with_keyword_in_string
              methods
                  function    no_return( this )
                      str = 'string with keyword: while'; 
                      a = 17;
                  end        
              end
          end
          

          OneLineClasses.m. The <parser> displays the expected result in Function List

          classdef OneLineClasses,methods,function OneLineMethod(~,str),disp(str),end,function [q]=OneLineMethod(str),disp(str),end,end,end
          

          Update the editor window first and then update the Function List

          Problem: “Browsing” through open files, Next Taband Previous Tab, is halted for seconds by files that the <parser> fails to process.
          Proposal: Let Next Taband Previous Tab interrupt the <parser> (if working), update the editor window, and then update Function List. While the <parser> is working the update icon could rotate to indicate ongoing work.

          More than one Matlab <parser>

          In some use cases I would like to show the full function signature in the Function List and in other only the function name. It’s straight forward to modify the current <parser> to only display function names. However, I cannot think of a good way to switch between the two variants.

          Keywords embedded in comments and in string values

          The problem with keywords in comments and string values makes the current <parser> nearly worthless for practical work. I have many “for” and “end” in my comments and string values.

          There is a simple solution, replace parser.classRange.mainExpr by something simple that requires that files honor the recommended format, e.g. (?m)^classdef\h+(?s:.*?)^end\b. That’s good enough for me. And I have done a useful exercise with RE.

          1 Reply Last reply Reply Quote 0
          • MAPJe71M
            MAPJe71
            last edited by

            Update the editor window first and then update the Function List

            I suspect the recursive part of classRange.mainExpr is giving the RE engine a hard time :-)

            More than one Matlab <parser>

            Based on current Function List implementation I can think of three solutions: define a <parser> for each use-case and switch between them by …

            1. changing the association; this requires a restart of NP++ to take effect;
            2. swapping their parser id, this also requires a restart of NP++ to take effect;
            3. utilizing file extension association and changing the extension of the file.

            Keywords embedded in comments and in string values

            The trick here is to handle literate strings as comment regions.
            e.g.

            commentExpr="(?x)                                    # free spacing
            				(?s:(?s:%\{.*%\})                    # Multi Line Comment
            			|	(?m-s:%.*$)                          # Single Line Comment
            			|	(?s:\x22(?:[^\x22\x5C]|\x5C.)*\x22)  # String Literal - Double Quoted
            			|	(?s:\x27(?:[^\x27\x5C]|\x5C.)*\x27)  # String Literal - Single Quoted
            			"
            
            1 Reply Last reply Reply Quote 0
            • Per IsaksonP
              Per Isakson
              last edited by

              @MAPJe71, thanks for your answer.

              The trick here is to handle literate strings as comment regions.

              Yes, but I fail to make it work. I copy&pasted your commentExpr value into the <parser> that I posted yesterday. None of the “keyword in comment and string” test files give the expected result in the Function List, i.e. the same result as yesterday.

              Speculation: commentExpr has no effect in my <parser>.

              1. The “block-comment-problem” is back, i.e. function_no_return_C2 is displayed
              2. With classRange.mainExpr using recursion skipping strings values and comments is needed.
              1 Reply Last reply Reply Quote 0
              • MAPJe71M
                MAPJe71
                last edited by

                @Per-Isakson

                There is definitely something weird going with the comment expression. The single line comment and both double and single quoted string literals are detected just perfectly but the multi line comment keeps failing.

                Oh just noticed that the comment expression I posted has a superfluous (?s: at the start of the “Multi Line Comment”-part.

                1 Reply Last reply Reply Quote 0
                • Per IsaksonP
                  Per Isakson
                  last edited by

                  @MAPJe71

                  The single line comment and both double and single quoted string literals are detected just perfectly …

                  That is not how I interpret what I see om my side. I have no debugging means to see how NPP uses the value of commentExpr. (Or I missed it.) Thus, I resort to indirect tests.

                  With

                  • your new value of commentExpr (corrected),
                  • classRange.mainExpr using recursion,
                  • the Matlab files attached and
                  • Notepad++ v7.3.3 (32-bit), Build time : Mar 8 2017 - 03:37:37, Admin mode : OFF, Local Conf mode : OFF, OS : Windows 7 (64-bit), Plugins : …

                  Function List displays a row of dots, a document icon and the file name. The evaluation, which obviously fails, takes a few seconds.

                  With the keyword, while, replaced by _while the Function List displays the expected results without delay.

                  Do you see a different behavior?

                  I’m out of ideas, are there other test I could do?

                  function    [ out, val ] = function_with_keyword_in_string( val )
                      out = 17;
                      str = 'string with keyword: while'; 
                  end
                  
                  function    [ out, val ] = function_with_keyword_in_comment( val )
                      out = 17;
                      % comment with keyword: while
                  end
                  
                  classdef    class_with_keyword_in_comment
                      methods
                          function    no_return( this )
                              % comment with keyword: while
                              a = 17;
                          end        
                      end
                  end
                  
                  classdef    class_with_keyword_in_string
                      methods
                          function    no_return( this )
                              str = 'string with keyword: while'; 
                              a = 17;
                          end        
                      end
                  end
                  
                  1 Reply Last reply Reply Quote 0
                  • MAPJe71M
                    MAPJe71
                    last edited by

                    This is what I get …

                    with the following <parser> …

                    <parser
                    	displayName="MATLAB - MATrix LABoratory"
                    	id         ="matlab_syntax"
                    	commentExpr="(?x)                                    # free-spacing (see `RegEx - Pattern Modifiers`)
                    					(?'MLC':                             # Multi Line Comment
                    						%\{                              # ...start-of-comment indicator
                    						(?:                              # ...followed by zero or more characters
                    							[^%]                         # ...not start of start-of-comment indicator
                    						|	%(?![%{}])                   # ...not being an SLC or a start- or end-of-comment indicator
                    						)*?
                    						%\}                              # ...end-of-comment indicator
                    					)
                    				|	(?m-s:\x25.*$)                       # Single Line Comment (SLC)
                    				|	(?s:\x22(?:[^\x22\x5C]|\x5C.)*\x22)  # String Literal - Double Quoted
                    				|	(?s:\x27(?:[^\x27\x5C]|\x5C.)*\x27)  # String Literal - Single Quoted
                    				"
                    >
                    	<classRange
                    		mainExpr    ="(?x)                               # free-spacing (see `RegEx - Pattern Modifiers`)
                    				(?s-i)                                   # dot matches at line breaks, case-sensitive
                    				\bclassdef\b                             # start-of-class indicator
                    				.*?                                      # whatever, until...
                    				\bmethods\b                              # ...start-of-class-body indicator
                    			"
                    		openSymbole ="(?x)                               # free-spacing (see `RegEx - Pattern Modifiers`)
                    				\b                                       # ensure leading word boundary for start-of-block indicator
                    				(?-i:                                    # case-sensitive start-of-block indicators
                    					e(?:numeration|vents)
                    				|	f(?:or|unction)
                    				|	if
                    				|	methods
                    				|	p(?:arfor|roperties)
                    				|	switch
                    				|	try
                    				|	while
                    				)
                    				\b
                    			"
                    		closeSymbole="(?x)                               # free-spacing (see `RegEx - Pattern Modifiers`)
                    				\b                                       # ensure leading word boundary for end-of-block indicator
                    				(?-i:                                    # case-sensitive
                    					end                                  # end-of-block indicator
                    				)
                    				\b                                       # ensure trailing word boundary for end-of-block indicator
                    			"
                    	>
                    		<className>
                    			<nameExpr expr="(?x)                         # free-spacing (see `RegEx - Pattern Modifiers`)
                    					\b(?-i:classdef)                     # case-sensitive start-of-class indicator
                    					\h+                                  # required whitespace
                    					(?:\([^)]*?\)\h+)?                   # optional class-attributes
                    					\K                                   # discard text matched so far
                    					[A-Za-z]\w*                          # valid character combination for identifiers i.e. class name
                    				"
                    			/>
                    		</className>
                    		<function
                    			mainExpr="(?x)                               # free-spacing (see `RegEx - Pattern Modifiers`)
                    					(?ms-i)                              # ^, $ and dot match at line breaks, case-sensitive
                    					(?:	^                                # a function can be found at start-of-line
                    					|	[,;]                             # ...or after a separator
                    					)\h*                                 # ...optionally followed by whitespace
                    					\K                                   # discard text matched so far
                    					\bfunction                           # ensure word boundary for start-of-function indicator
                    					\s+                                  # required whitespace separator
                    					.*?                                  # whatever, until...
                    					\bend\b                              # ...the first end-of-block indicator
                    				"
                    		>
                    			<functionName>
                    				<funcNameExpr expr="(?x)                 # free-spacing (see `RegEx - Pattern Modifiers`)
                    						function\s+                      # start-of-function indicator
                    						(?:\.{3}\s+)?                    # optional continuation-line indicator
                    						\K                               # discard text matched so far
                    						(?:                              # optional return value(s)
                    							(?:	\w+                      # ...single variable name
                    							|	\[[\h,\w]*\]             # ...or one or more variable names in brackets
                    							)\h*=\h*                     # ...followed by a separator with optional whitespace
                    						)?
                    						[A-Za-z]\w*                      # valid character combination for identifiers i.e. function name
                    						\b                               # ensure word boundary for name
                    						(?:                              # optional parameter list
                    							\s*                          # ...optional leading whitespace
                    							\(                           # ...start-of-parameter-list indicator
                    							[\h,\w~]*                    # ...with optional parameters
                    							(?:	\)                       # ...until end-of-parameter-list indicator
                    							|	\.{3}                    # ...or continuation-line indicator
                    							)
                    						)?
                    					"
                    				/>
                    				<!-- comment out the following node to display the method with its parameters -->
                    <!--				<funcNameExpr expr="(?:(?:\w+|\[[\h,\w]*\])\h*=\h*)?[A-Za-z]\w*" /> -->
                    			</functionName>
                    		</function>
                    	</classRange>
                    	<function
                    		mainExpr="(?x)                                   # free-spacing (see `RegEx - Pattern Modifiers`)
                    				(?ms-i)                                  # ^, $ and dot match at line breaks, case-sensitive
                    				^\h*                                     # optional leading whitespace at start-of-line
                    				function\b                               # start-of-function indicator
                    				.*?                                      # whatever, until...
                    				\bend\b                                  # ...the first end-of-block indicator
                    			"
                    	>
                    		<functionName>
                    			<nameExpr expr="(?x)                         # free-spacing (see `RegEx - Pattern Modifiers`)
                    					function\s+                          # start-of-function indicator
                    					(?:\.{3}\s+)?                        # optional continuation-line indicator
                    					\K                                   # discard text matched so far
                    					(?:                                  # optional return value(s)
                    						(?:	\w+                          # ...single variable name
                    						|	\[[\h,\w]*\]                 # ...or one or more variable names in brackets
                    						)\h*=\h*                         # ...followed by a separator with optional whitespace
                    					)?
                    					[A-Za-z]\w*                          # valid character combination for identifiers i.e. function name
                    					\b                                   # ensure word boundary for name
                    					(?:                                  # optional parameter list
                    						\s*                              # ...optional leading whitespace
                    						\(                               # ...start-of-parameter-list indicator
                    						[\h,\w~]*                        # ...with optional parameters
                    						(?:	\)                           # ...until end-of-parameter-list indicator
                    						|	\.{3}                        # ...or continuation-line indicator
                    						)
                    					)?
                    				"
                    			/>
                    			<!-- comment out the following node to display the method with its parameters -->
                    <!--			<nameExpr expr="(?:(?:\w+|\[[\h,\w]*\])\h*=\h*)?[A-Za-z]\w*" /> -->
                    		</functionName>
                    	</function>
                    </parser>
                    
                    1 Reply Last reply Reply Quote 1
                    • Per IsaksonP
                      Per Isakson
                      last edited by

                      @MAPJe71 ,
                      Your new parser works like a charm. It even handles WeirdUseOfContinuationLine.m. Thank you very much!

                      1 Reply Last reply Reply Quote 0
                      • Per IsaksonP
                        Per Isakson
                        last edited by

                        @MAPJe71

                        Sorry, I’m back with more problems. However, the result so far is good enough for me who know the limitations. Other Matlab users may think differently.

                        My goal is that the <parser> produces the intended result for syntactically correct Matlab files. Exceptions are acceptable for rare and smelly cases. It’s good if something is shown for syntactically incorrect Matlab files - the more the better.

                        The Matlab class syntax, [Class Syntax Guide], includes more than I have “communicated” with the test files in this thread.

                        Problem: A Matlab file allows one classdef...end block, which in turn may contain zero or more methods...end blocks. classdef shall be the first keyword in the file. The current <parser> displays the functions of the first methods...end block as belonging to the class, i.e. being methods, and erroneously the functions of subsequent methods...end blocks as functions. This is illustrated by the attached m-file.

                        I failed to modify your <parser> to show all function inside the classdef...end block as methods belonging to the class.

                        classdef (Sealed) aCompleteClassWithFunction < handle
                           properties (Access = private)
                              Prop1 = datenum(date)
                           end
                           properties
                              Prop2
                           end
                           events (ListenAccess = protected)
                              StateChanged
                           end
                           methods
                              function obj = aCompleteClassWithFunction(x)
                                 obj.Prop2 = x;
                              end
                           end
                           methods (Access = {?MyOtherClass})
                              function d = myMethod(obj)
                                 d = obj.Prop1 + x;
                              end
                           end
                        end
                        function myUtilityFcn
                            A = 17;
                        end
                        
                        1 Reply Last reply Reply Quote 0
                        • MAPJe71M
                          MAPJe71
                          last edited by

                          @Per-Isakson

                          Sigh, it was to good to be true ;-)
                          I’ll will look at it tomorrow.

                          1 Reply Last reply Reply Quote 0
                          • Per IsaksonP
                            Per Isakson
                            last edited by

                            @MAPJe71

                            I’m obviously not capable to contribute effectively to the code of this <parser>:-(. I lack both regarding knowledge on regular expressions, and understanding of how the <parser>-code produces the function list.

                            How important is a good Matlab-<parser> to Notepad++ ? To me Notepad++ is a valuable complement to Matlab’s editor, which is sluggish and doesn’t support regular expressions, not even wildcards. I use Notepad++ daily. Notepad++ is superior when it comes to large files and browsing large code-bases. Google returns 0.18 million results for “notepad++ matlab”.

                            I send this post rather to help you decide than insisting on specific things being included in the <parser>. What we achieved so far is good enough for me!

                            Is there a mean to make the user aware of limitations in the capability of a specific <parser>. Documented limitations are more acceptable.

                            MLC commentExpr

                            The colon after ‘MLC’ is that a typo or non-capture?

                            Proposal: replace the MLC-code by

                            (?xms)
                            (?'OMLC' (?>^\h*%\{\h*$))
                            .*?
                            (?'CMLC' (?>^\h*%\}\h*$))
                            

                            Justification: Again Matlab’s syntax [Add Comments to Programs] is permissive

                            • nested multi-line-comments (candidate for documented limitation)
                            • single-line-comments inside MLC
                            • I often comments out code with both SLC and MLC, because Matlab has a feature, Find in Files, which doesn’t honor MLC (bug). MLC allows me to code-fold the comment-out code.

                            This should be matched by MLC

                            %{
                            some code
                            %% Section 
                            %  some other code
                            %}
                            

                            This expression is affected by the block-comment-problem.

                            Nested functions

                            Matlab have nested functions. My previous <parser> showed functions nested in functions as ordinary functions and functions nested in methods broke the function list (showed only file name). Your current <parser> ignores nested functions altogether, which is better. Candidate for documented limitation.

                            function    out = function_scalar_return_nested( val )
                                out = 17;
                                function nested
                                end
                            end
                            
                            1 Reply Last reply Reply Quote 0
                            • Per IsaksonP
                              Per Isakson
                              last edited by

                              @MAPJe71

                              There is another Matlab feature, which is a strong candidate for documented limitation. In files defining functions and subfunctions, but no nested functions, end is optional in the function ... end block. It’s a legacy from long time ago.

                              1 Reply Last reply Reply Quote 0
                              • MAPJe71M
                                MAPJe71
                                last edited by

                                @Per-Isakson

                                Yep, the colon in the MLC commentExpr is a typo.
                                FYI: failing to detect multi-line/block comments when the parser has a classRange node appears to be a bug.

                                The only way I can think of right now to document limitations of parsers is to add it as comment in functionList.xml. I will do so for this parser.

                                Although I haven’t found a solution for multiple ‘methods’ sections within a class definition yet, I think the parser should support it presuming it’s used a lot in OO programming.

                                1 Reply Last reply Reply Quote 0
                                • MAPJe71M
                                  MAPJe71
                                  last edited by

                                  @Per-Isakson

                                  The latest:

                                  <!--
                                  |   Note(s):
                                  |   1)  One-line class definitions are not supported;
                                  |   2)  Nested functions are not supported;
                                  |   3)  All functions require an 'end' statement;
                                  |   4)  In some situations where a multi-line/block comment contains function definition(s),
                                  |       the function(s) is/are still displayed as global function(s).
                                  \-->
                                  <parser
                                  	displayName="MATLAB - MATrix LABoratory"
                                  	id         ="matlab_syntax"
                                  	commentExpr="(?x)                                    # free-spacing (see `RegEx - Pattern Modifiers`)
                                  					(?'MLC'                              # Multi Line Comment
                                  						(?m)                             # ...^ and $ match at line breaks
                                  						^%\{$                            # ...start-of-comment indicator
                                  						(?:                              # ...followed by zero or more characters
                                  							[^%]                         # ...not start of start-of-comment indicator
                                  						|	%(?![%{}])                   # ...not being an SLC or a start- or end-of-comment indicator
                                  						|	(?&amp;MLC)
                                  						)*?
                                  						^%\}$                            # ...end-of-comment indicator
                                  					)
                                  				|	(?m-s:%.*$)                          # Single Line Comment (SLC)
                                  				|	(?s:\x22(?:[^\x22\x5C]|\x5C.)*\x22)  # String Literal - Double Quoted
                                  				|	(?s:\x27(?:[^\x27\x5C]|\x5C.)*\x27)  # String Literal - Single Quoted
                                  				"
                                  >
                                  	<classRange
                                  		mainExpr    ="(?x)                               # free-spacing (see `RegEx - Pattern Modifiers`)
                                  				(?s-i)                                   # dot matches at line breaks, case-sensitive
                                  				\bclassdef\b                             # start-of-class indicator
                                  				.*?                                      # whatever, up till...
                                  				(?=                                      # ...start-of-class-body indicator
                                  					\b
                                  					(?:e(?:numeration|vents)|methods|properties)
                                  					\b
                                  				)
                                  			"
                                  		openSymbole ="(?x)                               # free-spacing (see `RegEx - Pattern Modifiers`)
                                  				\b                                       # ensure leading word boundary for start-of-block indicator
                                  				(?-i:                                    # case-sensitive start-of-block indicators
                                  					e(?:numeration|vents)
                                  				|	f(?:or|unction)
                                  				|	if
                                  				|	methods
                                  				|	p(?:arfor|roperties)
                                  				|	switch
                                  				|	try
                                  				|	while
                                  				)
                                  				\b
                                  			"
                                  		closeSymbole="(?x)                               # free-spacing (see `RegEx - Pattern Modifiers`)
                                  				\b                                       # ensure leading word boundary for end-of-block indicator
                                  				(?-i:                                    # case-sensitive
                                  					end                                  # end-of-block indicator
                                  				)
                                  				\b                                       # ensure trailing word boundary for end-of-block indicator
                                  			"
                                  	>
                                  		<className>
                                  			<nameExpr expr="(?x)                         # free-spacing (see `RegEx - Pattern Modifiers`)
                                  					\b(?-i:classdef)                     # case-sensitive start-of-class indicator
                                  					\h+                                  # required whitespace
                                  					(?:\([^)]*?\)\h+)?                   # optional class-attributes
                                  					\K                                   # discard text matched so far
                                  					[A-Za-z]\w*                          # valid character combination for identifiers i.e. class name
                                  				"
                                  			/>
                                  		</className>
                                  		<function
                                  			mainExpr="(?x)                               # free-spacing (see `RegEx - Pattern Modifiers`)
                                  					(?ms-i)                              # ^, $ and dot match at line breaks, case-sensitive
                                  					(?:	^                                # a function can be found at start-of-line
                                  					|	[,;]                             # ...or after a separator
                                  					)\h*                                 # ...optionally followed by whitespace
                                  					\K                                   # discard text matched so far
                                  					\bfunction                           # ensure word boundary for start-of-function indicator
                                  					\s+                                  # required whitespace separator
                                  					.*?                                  # whatever, until...
                                  					\bend\b                              # ...the first end-of-block indicator
                                  				"
                                  		>
                                  			<functionName>
                                  				<funcNameExpr expr="(?x)                 # free-spacing (see `RegEx - Pattern Modifiers`)
                                  						function\s+                      # start-of-function indicator
                                  						(?:\.{3}\s+)?                    # optional continuation-line indicator
                                  						\K                               # discard text matched so far
                                  						(?:                              # optional return value(s)
                                  							(?:	\w+                      # ...single variable name
                                  							|	\[[\h,\w]*\]             # ...or one or more variable names in brackets
                                  							)\h*=\h*                     # ...followed by a separator with optional whitespace
                                  						)?
                                  						[A-Za-z]\w*                      # valid character combination for identifiers i.e. function name
                                  						\b                               # ensure word boundary for name
                                  						(?:                              # optional parameter list
                                  							\s*                          # ...optional leading whitespace
                                  							\(                           # ...start-of-parameter-list indicator
                                  							[\h,\w~]*                    # ...with optional parameters
                                  							(?:	\)                       # ...until end-of-parameter-list indicator
                                  							|	\.{3}                    # ...or continuation-line indicator
                                  							)
                                  						)?
                                  					"
                                  				/>
                                  				<!-- comment out the following node to display the method with its parameters -->
                                  <!--				<funcNameExpr expr="(?:(?:\w+|\[[\h,\w]*\])\h*=\h*)?[A-Za-z]\w*" /> -->
                                  			</functionName>
                                  		</function>
                                  	</classRange>
                                  	<function
                                  		mainExpr="(?x)                                   # free-spacing (see `RegEx - Pattern Modifiers`)
                                  				(?ms-i)                                  # ^, $ and dot match at line breaks, case-sensitive
                                  				^\h*                                     # optional leading whitespace at start-of-line
                                  				function\b                               # start-of-function indicator
                                  				.*?                                      # whatever, until...
                                  				\bend\b                                  # ...the first end-of-block indicator
                                  			"
                                  	>
                                  		<functionName>
                                  			<nameExpr expr="(?x)                         # free-spacing (see `RegEx - Pattern Modifiers`)
                                  					function\s+                          # start-of-function indicator
                                  					(?:\.{3}\s+)?                        # optional continuation-line indicator
                                  					\K                                   # discard text matched so far
                                  					(?:                                  # optional return value(s)
                                  						(?:	\w+                          # ...single variable name
                                  						|	\[[\h,\w]*\]                 # ...or one or more variable names in brackets
                                  						)\h*=\h*                         # ...followed by a separator with optional whitespace
                                  					)?
                                  					[A-Za-z]\w*                          # valid character combination for identifiers i.e. function name
                                  					\b                                   # ensure word boundary for name
                                  					(?:                                  # optional parameter list
                                  						\s*                              # ...optional leading whitespace
                                  						\(                               # ...start-of-parameter-list indicator
                                  						[\h,\w~]*                        # ...with optional parameters
                                  						(?:	\)                           # ...until end-of-parameter-list indicator
                                  						|	\.{3}                        # ...or continuation-line indicator
                                  						)
                                  					)?
                                  				"
                                  			/>
                                  			<!-- comment out the following node to display the method with its parameters -->
                                  <!--			<nameExpr expr="(?:(?:\w+|\[[\h,\w]*\])\h*=\h*)?[A-Za-z]\w*" /> -->
                                  		</functionName>
                                  	</function>
                                  </parser>
                                  
                                  1 Reply Last reply Reply Quote 0
                                  • Per IsaksonP
                                    Per Isakson
                                    last edited by

                                    @MAPJe71

                                    Thank you for your latest parser.

                                    I’ve tested your latest parser and the parser I attach below with the files in a couple of real Matlab programs.

                                    Your latest parser handles “multiple ‘methods’ sections” well in most cases. I failed to spot what characterizes the few cases, which show methods as functions.

                                    One lesson I learned is that I must be more careful so that the test cases covers all relevant parts of the syntax of the target language. I previously missed that

                                    • the keyword, end, is used as index, e.g A(end) is the last element of the vector A.
                                    • apostrophe, ', is shorthand for the function, transpose, e.g. A' is equal to transpose(A)

                                    My latest parser works as expected, i.e. I haven’t seen it fail yet. However, it puts higher requirements on the target Matlab code; it will fail with more files.

                                    I have included two parsers in functionList.xml, one for the UDL matlab2. I switch between the two
                                    by the menu entries, Language|matlab2 and Language|M|Matlab, respectively. The obviously overtakes the extension, .m2.

                                    I plan to upload my parsers to Matlab File Exchange to see it I get any interest.

                                    <!-- P09, KISS, full function signature 
                                    Limitations and requirements
                                    1.1 The keyword, 'classdef', shall start a new line; no indentions
                                    1.2 The keyword, 'end', which match 'classdef', shall start a new line
                                    1.3 No keyword, 'end', shall start a new line inside the classdef block
                                    2.1 The keyword, 'function', optionally indented, shall start a new line 
                                    3.1 The line of the last keyword, 'end', shall be ended by a new line character
                                    4.1 Nested functions are displayed as methods and sub-functions, respectively.
                                    5.1 Block comments must not be nested; no block comment inside a block comment
                                    6.1 Function blocks shall be ended by the keyword, 'end'
                                    7.1 The file shall contain at least one method or function
                                    /-->
                                                <parser
                                                    displayName="MATLAB - MATrix LABoratory"
                                                    id         ="matlab2_syntax"
                                                    commentExpr=   "(?x)                        # free-spacing
                                                                    (?m)                        # ^ and $ match at line breaks
                                                                    (?-s)                       # dot does NOT match new line
                                                                    (?:                         # Multi Line Comment, (MLC)
                                                                        (?:^\h*\x25\x7B\h*$)    # '%{' with optional space
                                                                        (?s:.*?)                # lazy: any character up till
                                                                        (?:^\h*\x25\x7D\h*$)    # '%}' with optional space
                                                                    )                           #
                                                                    |                           # or
                                                                    (?:                         # Single Line Comment (SLC)
                                                                        (?:                     #
                                                                            \x25                # '%'
                                                                            |                   # or
                                                                            \x2D{3}             # '...' 
                                                                        ).*?$                   # and up till end of line
                                                                    )                           #
                                                                    |                           # or
                                                                    (?:\x27(?:.*?)*\x27)        # String Literal - Single Quoted - on one line
                                                                "
                                                >
                                                     <classRange
                                                        mainExpr = "(?x)                        # free-spacing
                                                                    (?m)                        # ^ and $ match at line breaks
                                                                    (?s)                        # dot matches new line
                                                                    (?-i)                       # case-sensitive
                                                                    ^classdef\h+                # start-of-class indicator at start-of-line
                                                                    (?:.*?)                     # whatever, until...
                                                                    ^end                        # end-of-class indicator at start-of-line
                                                                    "
                                                    >
                                                        <className>
                                                            <nameExpr expr="(?x)                # free-spacing
                                                                    (?-i)                       # case-sensitive
                                                                    \bclassdef                  # case-sensitive start-of-class indicator
                                                                    \h+                         # required whitespace
                                                                    (?:\([^)]*?\)\h+)?          # optional class-attributes
                                                                    \K                          # discard text matched so far
                                                                    [A-Za-z]\w*                 # valid Matlab identifiers; class name
                                                                "
                                                            />
                                                        </className>
                                                        <function
                                                            mainExpr="(?x)                      # free-spacing
                                                                    (?m)                        # ^ and $ match at line breaks
                                                                    (?s)                        # dot matches new line
                                                                    (?-i)                       # case-sensitive
                                                                    ^\h+                        # required intendation
                                                                    function                    # start-of-function indicator
                                                                    \h+                         # required whitespace separator
                                                                    .*?$                        # whatever, up till the end of line
                                                                "
                                                        >
                                                            <functionName>
                                                                <funcNameExpr expr="(?x)        # free-spacing
                                                                        function\h+             # start-of-function indicator
                                                                        \K                      # discard text matched so far
                                                                        (?:                     # optional return value(s)
                                                                            (?:                 #
                                                                                \w+             # ...single variable name
                                                                                |               # or
                                                                                \[[\h,\w]*\]    # ...one or more names in square brackets
                                                                            )                   #
                                                                            \h*=\h*             # ...a separator with optional whitespace
                                                                        )?                      #
                                                                        \b                      # ensure word boundary
                                                                        [A-Za-z]\w*             # valid Matlab identifiers; method name
                                                                        \b                      # ensure word boundary for name
                                                                        (?:                     # optional parameter list
                                                                            \h*                 # ...optional leading whitespace
                                                                            \(                  # ...start-of-parameter-list indicator
                                                                            [\h,\w~]*           # ...with optional parameters
                                                                            (?:                 #
                                                                                \)              # ...up till end-of-parameter-list indicator
                                                                                |               # or
                                                                                \.{3}           # ...continuation-line indicator
                                                                            )
                                                                        )?
                                                                    "
                                                                />
                                                            </functionName>
                                                        </function>
                                                    </classRange>
                                                    <function
                                                        mainExpr="(?x)                          # free-spacing
                                                                    (?m)                        # ^ and $ match at line breaks
                                                                    (?s)                        # dot matches new line
                                                                    (?-i)                       # case-sensitive
                                                                    ^\h*                        # optional intendation
                                                                    function                    # start-of-function indicator
                                                                    \h+                         # required whitespace separator
                                                                    .*?$                        # whatever, until end of line
                                                            "
                                                    >
                                                        <functionName>
                                                            <nameExpr expr="(?x)                # free-spacing (see `RegEx - Pattern Modifiers`)
                                                                    function\h+                 # start-of-function indicator
                                                                    (?:\.{3}.*$)?               # optional continuation-line indicator
                                                                    \K                          # discard text matched so far
                                                                    (?:                         # optional return value(s)
                                                                        (?:                     #
                                                                            \w+                 # ...single variable name
                                                                            |                   # or
                                                                            \[[\h,\w]*\]        # ...one or more variable names in brackets
                                                                        )                       #
                                                                        \h*=\h*                 # ...followed by a separator with optional whitespace
                                                                    )?                          #
                                                                    [A-Za-z]\w*                 # valid Matlab identifiers; function name
                                                                    \b                          # ensure word boundary for name
                                                                    (?:                         # optional parameter list
                                                                        \h*                     # ...optional leading whitespace
                                                                        \(                      # ...start-of-parameter-list indicator
                                                                        [\h,\w~]*               # ...with optional parameters
                                                                        (?:                     #
                                                                            \)                  # ...until end-of-parameter-list indicator
                                                                            |                   # or
                                                                            \.{3}               # ...or continuation-line indicator
                                                                        )
                                                                    )?
                                                                "
                                                            />
                                                        </functionName>
                                                    </function>
                                                </parser>
                                    

                                    Alternate <functionName> block to show method and function name (without input and output arguments)

                                                        <functionName>
                                                            <nameExpr expr="(?x)                # free-spacing (see `RegEx - Pattern Modifiers`)
                                                                    function\h+                 # start-of-function indicator
                                                                    (?:\.{3}.*$)?               # optional continuation-line indicator
                                                                    (?:                         # optional return value(s)
                                                                        (?:                     #
                                                                            \w+                 # ...single variable name
                                                                            |                   # or
                                                                            \[[\h,\w]*\]        # ...one or more variable names in brackets
                                                                        )                       #
                                                                        \h*=\h*                 # ...followed by a separator with optional whitespace
                                                                    )?                          #
                                                                    \K                          # discard text matched so far
                                                                    [A-Za-z]\w*                 # valid Matlab identifiers; function name
                                                                    \b                          # ensure word boundary for name
                                                                "
                                                            />
                                                        </functionName>
                                    1 Reply Last reply Reply Quote 0
                                    • MAPJe71M
                                      MAPJe71
                                      last edited by

                                      @Per-Isakson

                                      *indentation ;-)

                                      Glad you found a parser that meets your needs!
                                      I propose to wait for feedback from other Matlab users (I’m not one of them) before including it in an official N++ release.

                                      1 Reply Last reply Reply Quote 0
                                      • Per IsaksonP
                                        Per Isakson
                                        last edited by

                                        @MAPJe71

                                        I’ve finally uploaded some <parser>-elements to the Matlab File Exchange.

                                        1 Reply Last reply Reply Quote 1
                                        • First post
                                          Last post
                                        The Community of users of the Notepad++ text editor.
                                        Powered by NodeBB | Contributors