@Eileen-Ling ,
With just the User Defined Language (UDL) syntax, what you want is probably not achievable.
However, if you install the EnhanceAnyLexer plugin (using Plugins > Admin tool), then you can make sure your file has the right UDL active, then use Plugins > EnhanceAnyLexer > Enhance Current Language to edit the plugin’s config file. You can then define foreground color changes using “0xBBGGRR = regex” pairs.
32d726da-3676-466c-b50b-0f889bb555b6-image.png
[udf]
; color each word, 0x66ad1 is the color used, see the description above for more information on the color coding.
0x0080FF = \([^)\\]]*?\)
0x0080FF = \\[[^)\\]]*?\\]
0x00FF80 = \([^)\\]]*?\\]
0x00FF80 = \\[[^)\\]]*?\)
; check in the respective styler xml if the following IDs are valid
excluded_styles = 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,20,21,22,23
In my example, I defined four regular expressions: two will match the balanced [...] and (...); the other two match the imbalanced [...) and (...]
If you don’t want it to match multiline, change each of those four regex to have \r\n right after the ^, so that it doesn’t look for parenthesized text that spans across newline characters.
note: mine used [udf] as the header, because I was in an unnamed UDL. If you follow my instructions above, yours will have [YourUdlNameHere] instead.