UDL for DXL (C LIke) Number processing issues.

swegmike

I have v8.4.6 x64 bit and all plugins that it comes with installed on Windows 8.1. I’m making my own UDL language for DOORS DXL but am having various issues in the Number recognizer so the styler is applied correctly.

My setup below for comment and numbers seup is below. My docs show 0x and 0X for hex numbers,
0b for binary and 0(zero) for octal #'s. These were added to prefix 1 and 2 and apply the styles colors as shown in the second pic below. I’ve also included your examples for your user docs on B O D; however, D will not apply the color. If I reverse the order for B O D to i.e. D O B, then the B wont get colorized, and likewise if the last character in prefix 1 in O instead.

Moreover, the real #'s will not colorize for the period, but will for comma if I check each respectively. If both is checked #'s after the period don’t colorize, but comma does. I’m not sure how to get the floating point # to fully colorize too.

Moreover, if I setup delimiter 1 to colorize strings as shown, it will work in most cases. I add more info for the issue after posting this. However, if I want to do something similar for char string as shown in delimiter #2 using apostrophes, it will work (see below); however, this causes problems later when the suffix 'H is specified per user docs since it will colorize it and everything else after it looking for the closing apostrophe which it never finds. I even tried using ((EOL)) as the closing char instead hoping it sees the EOL after 'H to stop colorizing too - didn’t work.

if delimiter #2 is deleted for the apostrophes, similar colorizing problems occur for suffix #2.

Moreover, the next post to this will be for comment issues. Perhaps ((EOL)) functionality would work here but so far nada…

pic #1 UDL number setups
pic #2 UDL delimiter setups with test code colorizing shown next to popup as the int, real, char etc.

notepad++_UDL_Number.png

notepad++_UDL_Delimiter_filecolor_prob.png

swegmike

Here is the complete list of operator 1 content.

; , ? : = ( + * [ ) ] & - ! ~ / % << >> <> < > <= >= == != ^ | && || ^^ += -= *= /= %= <<= >>= &= |= ^= <- := => … . -> :: \

Also you’ll notice that the UDL popups for comments, delimiters etc aren’t scrollable, thus the bottom parts of each show below the scrollbar and are hard to edit on my Dell laptop. Maybe notpad can change this to that they are scrollable.

Also, It would be nice if notepad++ allowed one to enter an alias name thats associated with each keyword group. I.e.
I’d enter as an alias for keyword 1 group “Statements”, and for keyword 2 group the alias “DXL Types”, and keyword 3 group “DXL BUILT-IN FUNCTIONS”. All of these I’d apply a different color code style; however, for my user defined functions I write for my program, I’m not sure how they can be colorized. I don’t want to add them to a the DXL UDL xml keyword list since they could change for the next program thats written with the help of the DXL UDL and notepad++

swegmike

I’ve narrowed down the problem in notepad for UDL numbering.

a). If prefix1 is set to i.e… D O B or one of these letters at a time, it will work for the case of all these letters in any combination for the first two letters; however, for the last letter specified it will not work regardless which one is specified if there is no space after it such as "B " (but without the quotes.) Moreover if one defines one letter at a time it will work only if there is a space after it.

For suffix 2 it will not work regardless of the D O B letter ordering for the letters D and B, but works for O only. Thus, if each of these letters is specified one at a time, only the O will work.

So for both these cases notepad has a software bug.

b). If suffix 1 specifies 'H and one wants to colorize char codes i.e. ‘A’ as mentioned above earlier, it will colorize everything to include 'H trying to find the closing quote. So notepad UDL appears to have a bug since it will not treat the end of the line as the terminator for this special case nor entering ((EOL)) in suffix 1 work.

c). As for coloring real #'s with exponents, I’ve discovered that adding E e to the suffix 1 field will work and even colorize the number between the decimal point and the E or e.

swegmike

I ran into another problem that is inconsistent across UDL setups.

1). For any keyword entry 1-8 I can separate them with a space or a carriage return. The carriage return is nice when ones list is long about 100 items since I could sort them in excel and copy and paste them into to a particular keyword number; however, for operator 1 & 2, I can’t sort the list and separate each operator by a carriage return. It only allows a space between them. Hence if ones list is long, having a long run-on list separated by spaces is hard to maintain. Again if NPP can be changed to allow a carriage return too.

If a carriage return is used, then the operators and keywords aren’t colorized if there is no space in the actual code between the keyword and operator. I.E. a function call

add(int a, int c)
None if this will be colorized for the function name, operators ( , and keyword int unless I put a space between the function name add and the operator (. I must also do the same between the operator , and the keyword type int for cases including any that don’t specify a function name.

PeterJones

@swegmike ,

Again if NPP can be changed

The members of this forum are fellow users of Notepad++, and cannot change the application codebase for you. And if you go to the official issues tracker, you will find that UDL feature requests are essentially ignored: basically, UDL is as it is, even if there are valid requests for improvements.

add and the operator (

I would suggest not making parentheses as operators, but rather as a delimiter pair: that’s how every UDL I’ve seen or created does it.

Back to older comments (sorry, I wasn’t near a Notepad++ instance when I first read your post last week, and forgot to come back to it):

If prefix1 is set to i.e… D O B or one of these letters at a time, it will work for the case of all these letters in any combination for the first two letters; however, for the last letter specified it will not work regardless which one is specified if there is no space after it such as "B "

My experience seems different to yours:

With the order B O D in prefix 1, I get all three colorized, eve with 0b added. With 0x in Prefix2 and a b c d e f A B C D E F in the Extras 1, I get all hex and binary colorized (and octal has no problem, because the 0 prefix is a number in Notepad++'s mind already, so no need to add that to prefix 1). I do seem to see a problem with suffix 2 – but that might just be because trying to make the same character mean too many different things confuses the parser.

Presumably, the reason that “real numbers” aren’t working for you is because you defined the . as an operator that doesn’t require a space between, so maybe it’s interpreting that character as an operator rather than the decimal point. Once again, the fewer things that you make “conflict” between one setting and another, the easier it is for the UDL parser to actually do the parsing.

But yes, UDL is not perfect. The intention was to use UDL for simple things, and then write a custom lexer-plugin if your needs are too complicated for UDL. As an intermediate step, in general, any time the UDL parser isn’t sufficient for your needs, I suggest adding in the EnhanceAnyLexer parser, which allows you to set up a list of regular expressions to change foreground colors base on matches, with different regex for each language (whether builtin or UDL).

Sean F

@swegmike

Hi Mike

Do you have a syntax highlighting file that works for editing IBM DOORS DXL script files that you are willing to share?

I have one that used to work with the old text editor UltraEdit but I am not sure how to convert it to be used with NPP

Thanks
Sean

swegmike

@Sean-F Not yet. I’m still working thru the DXL reference manual for v9.7. As you can see, the various issues I’m having. I’m still in the learning phase of NPP. If I get this working, I may move from UDL to the full blown DXL language support like the other languages included in NPP so that function lists can be shown on the sidebar.

I’m not sure if DOORS Next has a newer version of DXL. Google doesn’t show one specific to DOORS Next and I don’t have this particular product version.

Which version of DOORS does the copy you have support?

See below for IBM’s partner. One will have to play with the different versions they have free/licensed versions to see how they work.

SodiusWillert has Editor Pro.
Their old version is Smart DXL and is supposed to be Eclipse compatible.

http://www.smartdxl.com/content/?p=483

PeterJones

@swegmike said in UDL for DXL (C LIke) Number processing issues.:

I may move from UDL to the full blown DXL language support like the other languages included in NPP so that function lists can be shown on the sidebar.

FYI: while it may be a good idea to write a Syntax-Highlighter Plugin for DXL, you do not need to wait for doing that before you have function lists. As described in the online User Manual, you define the function list for a UDL in the same way that you define one for a builtin syntax highlighter or plugin syntax highlighter:

There is even a section in the User Defined Languages Collection for also including the functionList XML file for that UDL. So once you get the UDL and Function List working for DXL, make sure to submit it there.

swegmike

@PeterJones Thanks. I’ll need to look more into it. Since the parsers are more regular expression, I’m not sure if there is a typical structural layout for any language. That’s why I posted this below to see if that tool would create the parser/regular expression in XML so that all I would do is copy and paste the answer based on the tools user friendly input (???)
Still working on the UDL right now.

https://community.notepad-plus-plus.org/topic/23612/xml-regular-express-editor-for-notepad-function-list

swegmike

@swegmike How can one configure the UDL such that it will colorize differently if the same string name for a keyword is used to denote a built-in function while the same name is used for a property and a value? The DXL scripting language does this alot.

I’ve tried to specify ie. keyword 1 for abc be the builtin function name, and then keyword 2 be abc or .abc for the property, and keyword 3 be abc for a constant and the same for a value. The resulting color is based on the matching the first keyword symbol table created by npp+ and then npp+ stops searching the rest of the keywords that have the same keyword. For the case of the property where it may be a function name or a value such as a or a.description() or a.description, the colorization is based on the first match found in the keyword list 1-8. Luckily, the resuse of the name as a type for this language doesn’t occur. I have builtin types also assigned to a keyword #.

PeterJones

@swegmike said in UDL for DXL (C LIke) Number processing issues.:

How can one configure the UDL such that it will colorize differently if the same string name for a keyword is used to denote a built-in function while the same name is used for a property and a value?

That is not a feature of UDL.

UDL is meant as a simple interface for simple keyword, operator, and delimiter highlighting of langauges that Notepad++ does not offer; it is not meant to be as powerful as a custom-coded lexer for a given language (though it’s usually sufficient for most people’s actual needs). If the power of UDL is not sufficient for your needs, then you need to write a custom lexer plugin.

I’m not sure if there is a typical structural layout for any language

Every language is unique. As far as I know, no one has written a tool to help Notepad++ users write regex specifically for function lists definitions. There are plenty of regex-helper tools out there; you seem to have already linked to one in that other thread; if that, or another existing tool, is useful to you in the development of your function-list definition regex, great, use it! But if not, then you are going to have to just dig in and create the regex yourself. Start with a simple regex that doesn’t cover every edge case, but just gives simple functions and then work your way toward something more complicated, making sure it’s working at every step along the way – that’s how I’ve done it.

swegmike

@PeterJones I was looking thru the notepad online manual for the valid delimiter escape codes allowed.
This also applies to the close field too. In my case, I’d like to specify a non-printable character such as a tab along with a printable character such as space. Thus the open delimiter would be a period and when it parses to the end of the string and sees a “space” or “tab” after the string the escape or close part of the delimiter would stop the coloring and continue on to the end of the line. I realize there is a special ((EOL)) which isn’t mentioned n the user manual delimiter section. Are there any other undocumented special codes?
I have tried both the \t \r \n along with the hexadecimal codes ie. \0x020 for the space etc and none of these will work either.

Also the npp+ keywords screen has the prefix checkbox for each one, it would be nice to have a postfix/suffix checkbox whereby one can check either one or both. I think this would eliminate the escape characters stated above but in the end both the escape characters and the check boxes would be great to have…

PeterJones

@swegmike said in UDL for DXL (C LIke) Number processing issues.:

@PeterJones I was looking thru the notepad online manual for the valid delimiter escape codes allowed.

There aren’t any, that I’m aware of.

This also applies to the close field too. In my case, I’d like to specify a non-printable character such as a tab along with a printable character such as space. Thus the open delimiter would be a period and when it parses to the end of the string and sees a “space” or “tab” after the string the escape or close part of the delimiter would stop the coloring and continue on to the end of the line.

As far as I know, that’s not possible.

I realize there is a special ((EOL)) which isn’t mentioned n the user manual delimiter section. Are there any other undocumented special codes?

Really? “Notice how UDL’s special operator (( )) transforms string EOL into an end of line character”. Yes, that’s in Ivan Radić’s site, not the npp-user-manual.org site, but that’s because Don agreed that Ivan’s documentation was so good that when we created npp-user-manual.org, we weren’t going to try to re-create it – it has been officially linked through the Define Your Language dialog since before npp-user-manual.org existed, and the npp-user-manual.org has throughout its existence always linked to Ivan’s site.

Also the npp+ keywords screen has the prefix checkbox for each one, it would be nice to have a postfix/suffix checkbox whereby one can check either one or both. I think this would eliminate the escape characters stated above but in the end both the escape characters and the check boxes would be great to have…

This forum cannot help you with feature requests (I am sure you can find the forum’s FAQ on that topic). But, so you know, Notepad++ developers have shown no interest in adding new features to UDL in years, and have only rarely fixed bugs in it, despite the number of open UDL issues.

If you want more power than UDL offers, but don’t want to go so far as to write your own lexer plugin (which is the intended next step beyond UDL), there is the EnhanceAnyLexer plugin. It could handle some of your space/tab desires (as long as you just want to change the foreground color; it doesn’t have background colors, and cannot affect folding).

swegmike

@PeterJones I pretty much have my first cut on the DXL language via UDL. I’ve also copied the “C” language parser since DXL is “C” like into a dxl.xml file and changed various entries from “C” to “DXL” etc. in the functionlist subdirectory for NPP. I’ve done the same using the “C” file in the autocomplete subdirectory but copied as DXL with the appropriate changes.

Here’s list of the files changed based on corresponding c.xml files or entries. I created the
language ID as 61 temporarily for the DXL language as also similarly in the roaming directory:

autoCompletion/dxl.xml
functionList/dxl.xml
Notepad++/
functionList.xml
langs.model.xml
langs.xml

So the goal here, is to get my DXL language, autocomplete list, and functionlist to show up in the NPP menubar->Language drop down list so that I can see it working for my DXL source code. However, DXL doesn’t showup in the Language list drop down. However, if I select the “C” language for use on my DXL source code, it will find the functions and display in the function list, and the autocomplete popup of keywords will show up. Along with applying the default “C” language colorization of the code.

Thus, what am I missing to get my “DXL” language to show up in the language drop down list vs. the UDL language entry based on dxl.xml colorization file?

Hopefully the NPP langs are hardcoded into the npp executable binary file.

swegmike

@swegmike When I try to edit the post above for content and submit it, the forum thinks its spam…

PeterJones

@swegmike ,

UDLs show up below the line, under the User Defined Language menu – like the Markdown UDL

You cannot get a UDL to show up above the line, in the alphabetized section.

Hopefully the NPP langs are hardcoded into the npp executable binary file.

They are.

I created the language ID as 61 temporarily for the DXL language

UDLs do not get their own language ID – they are all handled by the UDL lexer. Just arbitrarily inserting that language ID into the other config files will not change that fact.

To make sure your DXL auto-complete and functionlist are working, all you have to do is activate the UDL selection for DXL.

Lycan Thrope

@swegmike ,

I don’t see you listing the overrideMap.xml as a file that has a change, as is necessary for a UDL to associate the “proper” menu name entry with the UDL created xml file.

For instance, I’m able to associate the UDL that I created that connects all the files together with the overrideMap.xml file entry like so:

			<association id= "dbaseplus.xml"		userDefinedLangName="dBASEPlus"/>
        <association id= "dbaseplus.xml"		userDefinedLangName="Dark_dBASEPlus"/>

This is clearly layed out in the documentation. Notice I have capitalized names that show in the menu, but they are linked to the dbaseplus.xml UDL file. This is possibly what you’re missing. Do make sure that the same case is used that you use to identify the UDL in the dxl.xml UDL file. The numbers, you say you’re using, are for the compiled languages that are in the Notepad++ object files, of which a UDL can not be.

I hope this helps, if not, please disregard.

swegmike

@Lycan-Thrope I’ve looked at the overrideMap.xml and also noticed that the id name specifies the language with xml on the end. So in my case I would specify “dxl.xml” which is specified in the functionList and autoCompletion subdirectories.

However, the overrideMap.xml is not present when npp was installed so I’m not sure if I have to create one and where to put it. Most of the online docs specify testing things and shows a “powereditor” directory path which I don’t have nor this particular editor.

I did modify the functionList.xml file which is similar to the overrideMap.xml, but in the userdefined area it doesn’t show as examples that the id as “mylang.xml” but specifies “mylang_function” thus I had done this as “dxl_function” and inserted a copy of the “C” lang parser which specifies “c_function” as its id but I renamed it as “dxl_function” and the language as “DXL”. Moreover, this change was made to the dxl.xml files in the autoCompletion and functionList subdirectories.

So if I’m understanding you, the id must be “dxl.xml” and not “dxl_function” so it can find the files to use. But I’m not clear on whether I just edit the functionList.xml in the notepad++ directory or if I need to create an overrideMap.xml that has only my userdefined associations and parser in it and notepad will then use the functionList.xml for the other languages or use the overridMap.xml that only has my language when needed?

Moreover, the “C” language specifies c.xml in the autoComplete and functionList subdirs and yet the functList.xml for the id doesn’t specify “c.xml” but “c_function”

swegmike

@swegmike I’ve changed the id from “dxl_function” to “dxl.xml” everywhere, and restared npp, and the one function my dxl code has doesn’t showup in the functionlist with the udl “dxl” selected as the language.

If I switch to the builtin “C” language and select view->function list, it finds my one function and displays it correctly in the functionlist.

So I dont know where the mismatch is for my DXL UDL and the files that I have changed.

I even copied the functionList.xml that was modified for the udl associations as overrideMap.xml and still can’t display the one function in the functionlist when I select dxl as the udl language, but it will showup when the “C” language is selected.

PeterJones

@swegmike ,

All these things – for the Function List with its override map, and the autoCompletion files, and where they all go – are described in the usermanual,

Function List - https://npp-user-manual.org/docs/config-files/#function-list
Auto Completion - https://npp-user-manual.org/docs/auto-completion/

I did modify the functionList.xml file which is similar to the overrideMap.xml,

Nope. The old functionList.xml is no longer used, starting with v7.9.2. Since you say you are on 8.4.6, you will never need to edit that file; if you have that file, delete it, as it will only confuse you, and will not do anything with v8.4.6.

overrideMap.xml goes in %AppData%\Notepad++\functionList – it should exist with a fresh install, but if you updated from an ancient Notepad++ at some point, it might never have been created. In which case, I recommend copying the default one from a portable zipfile and putting it in the right location.

So if I’m understanding you, the id must be “dxl.xml” and not “dxl_function” so it can find the files to use.

The ID in the overrideMap.xml must match your autoCompletion filename. If you autocompletion filename is dxl.xml, then the id should be dxl.xml. If your UDL’s name is DXL, then the userDefinedlangName should be set to DXL.

<association id="DXL.xml" userDefinedLangName="DXL"/>

ID was a bad name for that attribute, IMO; it should have been “filename” or something.

dxl_function

That’s the standard nomenclature inside the Function List definition XML file (in your dxl.xml). It doesn’t need to match anything from overrideMap, but than it cannot match any of the internal IDs of any of the other languages’ functionList definitions.

Think of it this way: Notepad++ reads overrideMap to find the filename, to be able to map the UDL name to a filename for the function list file. Then once it has that, Notepad++ reads the appropriate file from the functionList\ folder to get the definition.

Moreover, the “C” language specifies c.xml in the autoComplete and functionList subdirs and yet the functList.xml for the id doesn’t specify “c.xml” but “c_function”

That’s because the functionList.xml file was the old way of doing things, which has been irrelevant since v7.9.2. And neither the old functionList.xml nor the modern overrideMap.xml have any effect on the autoCompletion; autoCompletion has its own set of rules.

Most of the online docs specify testing things and shows a “powereditor” directory path which I don’t have nor this particular editor.

There is no official Notepad++ documentation that tells you to do anything with a “powereditor” directory path.