Continuation of the previous post !

You certainly noticed that, in the 4 last cases of the previous post, the numbers are always moved back, right after the ABC DEF string, because digits, as bi-directional weak char, are considered as belonging to the last embedded directional range. So, how to force the numbers to stay, after the GHI JKL MNO PQR STU string, already written in the Right to Left direction ?

For such a task, and to compensate for the lack of the Isolates formating characters, we need to use one of the Implicit Directional Marks, below, which do work, too, with Notepad++ ;-))

-  LEFT-TO-RIGHT MARK          LRM    200E
-  RIGHT-TO-LEFT MARK          RLM    200F
-  ARABIC LETTER MARK          ALM    061C

These characters are very light-weight formatting and act, simply, as a bidirectional strong char, with the L or R type

So, we get 4 more cases :

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

ABC DEF <RLO>GHI JKL MNO PQR STU<PDF> <LRM>123, 456 VWX YZA

ABC DEF ‮GHI JKL MNO PQR STU‬ ‎123, 456 VWX YZA  ( 47 chars )

─► The "ABC DEF " string is written with the DEFAULT direction of the CURRENT embedding level (level 0), so with the "Left to Right" direction

─► The "GHI JKL MNO PQR STU" string, from right AFTER the <RLO> formating char, till the <PDF> formating char (level 1), is written in the "Right to Left" direction

─► The SPACE char, as a NEUTRAL char, is written in the CURRENT bi-directional direction, of level 0, so "Left to Right"

─► Finally, the INVISIBLE <LRM> mark, acting as a STRONG "Left to Right" character forces ALL remaining text, so the string "123, 456 VWX YZA", to be rewritten in the "Left to Right" direction

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

ABC DEF <RLO>GHI JKL MNO PQR STU<PDF> <LRM><RLO>123<PDF>, <LRM><RLO>456<PDF> VWX YZA

ABC DEF ‮GHI JKL MNO PQR STU‬ ‎‮123‬, ‎‮456‬ VWX YZA  ( 52 chars )

─► The "ABC DEF " string is written with the DEFAULT direction of the CURRENT embedding level (level 0), so with the "Left to Right" direction

─► The "GHI JKL MNO PQR STU" string, from right AFTER the FIRST <RLO> formating char, till the FIRST <PDF> formating char (level 1), is written in the "Right to Left" direction

─► The SPACE char, as a NEUTRAL char, is written in the CURRENT bi-directional direction, of level 0, so "Left to Right"

─► The INVISIBLE <LRM> mark, acting as a STRONG "Left to Right" character forces the context string "123, ", with the WEAK bidirectional digits , to be rewritten in the "Left to Right" direction

    ─► But the "123" number, from right after the SECOND <RLO> formating char, till the SECOND <PDF> formating char (level 1) is written, in the "Right to Left" direction

─► Again, the INVISIBLE <LRM> mark, acting as a STRONG "Left to Right" character forces the string "456", with the WEAK bidirectional digits , to be rewritten in the "Left to Right" direction, so AFTER the "123, " string

    ─► But the "456" number, from right after the THIRD <RLO> formating char, till the THIRD <PDF> formating char (level 1) is written, in the "Right to Left" direction
	
─► Finally, the " VWX YZA" string, after the THIRD <PDF> formating char, is written with the DEFAULT direction (level 0), so in the "Left to Right" direction

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

ABC DEF <RLO>GHI JKL MNO PQR STU<PDF> <LRM><RLO>123, 456<PDF> VWX YZA

ABC DEF ‮GHI JKL MNO PQR STU‬ ‎‮123, 456‬ VWX YZA  ( 49 chars )

─► The "ABC DEF " string is written with the DEFAULT direction of the CURRENT embedding level (level 0), so with the "Left to Right" direction

─► The "GHI JKL MNO PQR STU" string, from right AFTER the FIRST <RLO> formating char, till the FIRST <PDF> formating char (level 1), is written, in the "Right to Left" direction

─► The SPACE char, as a NEUTRAL char, is written in the CURRENT bi-directional direction, of level 0, so "Left to Right"

─► The INVISIBLE <LRM> mark, acting as a STRONG "Left to Right" character forces EACH remaining block of text, to be rewritten in the "Left to Right" direction

─► However, the "123, 456" string, from right AFTER the SECOND <RLO> formating char, till the SECOND <PDF> formating char (level 1) must be written, in the "Right to Left" direction

─► Finally, the " VWX YZA" string, after the SECOND <PDF> formating char, is written with the DEFAULT direction (level 0), so in the "Left to Right" direction

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

ABC DEF <RLO>GHI JKL MNO PQR STU<PDF> <LRM><RLO><LRO>123<PDF>, <LRO>456<PDF><LRM><PDF> VWX YZA

ABC DEF ‮GHI JKL MNO PQR STU‬ ‎‮‭123‬, ‭456‬‎‬ VWX YZA  ( 54 chars )

─► The "ABC DEF " string is written with the DEFAULT direction of the CURRENT embedding level (level 0), so with the "Left to Right" direction

─► The "GHI JKL MNO PQR STU" string, from right AFTER the FIRST <RLO> formating char, till the FIRST <PDF> formating char (level 1), is written in the "Right to Left" direction

─► The SPACE char, as a NEUTRAL char, is written in the CURRENT bi-directional direction, of level 0, so "Left to Right"

─► The INVISIBLE <LRM> mark, acting as a STRONG "Left to Right" character, forces the context-numbers string to NOT depends on the FIRST <RLO>........<PDF> range

─► Then the "123, 456", right after the SECOND <RLO> formating char, till the FOURTH <PDF> formating char, is rewritten in the "Right to Left" direction (leval 1)

    ─► But the "123" number, from right after the FIRST <LRO> formating char, till the SECOND <PDF> formating char (level 1) is written, in the "Left to Right" direction (level 2)

    ─► The ", " string is written according to the CURRENT direction of the SECOND <RLO>........<PDF>range, starting at the SECOND <RLO> formating char, till the FOURTH (level 1), so in the "Right to Left" direction

    ─► The "456" number, from right after the SECOND <LRO> formating char, till the THIRD <PDF> formating char (level 2) is written, in the "Left to Right" direction (level 2)

─► The INVISIBLE <LRM> mark, acting as a STRONG "Left to Right" character, forces the " VWX YZA" string, to be written in the "Left to Right" direction (level 0) However, note that I did NOT understand WHY I had to locate the <LRM> mark BEFORE the LAST <PDF> formating char ( and NOT after ) to get the right behavior !

So, in short, starting with the 2 sample texts ABC DEF GHI JKL MNO PQR STU VWX YZA and ABC DEF GHI JKL MNO PQR STU 123, 456 VWX YZA, the syntaxes, below :

ABC DEF <RLO>GHI JKL MNO PQR STU VWX YZA
ABC DEF <RLO>GHI JKL MNO PQR STU<PDF> VWX YZA
ABC DEF <RLO>GHI JKL<PDF> MNO <LRO>PQR STU<PDF> VWX YZA
ABC DEF <RLO>GHI JKL <LRO>MNO PQR<PDF> STU<PDF> VWX YZA

ABC DEF <RLO>GHI JKL MNO PQR STU<PDF> 123, 456 VWX YZA
ABC DEF <RLO>GHI JKL MNO PQR STU<PDF> <RLO>123, 456<PDF> VWX YZA
ABC DEF <RLO>GHI JKL MNO PQR STU<PDF> <LRO>123, 456<PDF> VWX YZA
ABC DEF <LRO><RLO>123<PDF>, <RLO>456<PDF><PDF> <RLO>GHI JKL MNO PQR STU<PDF> VWX YZA

ABC DEF <RLO>GHI JKL MNO PQR STU<PDF> <LRM>123, 456 VWX YZA
ABC DEF <RLO>GHI JKL MNO PQR STU<PDF> <LRM><RLO>123<PDF>, <LRM><RLO>456<PDF> VWX YZA
ABC DEF <RLO>GHI JKL MNO PQR STU<PDF> <LRM><RLO>123, 456<PDF> VWX YZA
ABC DEF <RLO>GHI JKL MNO PQR STU<PDF> <LRM><RLO><LRO>123<PDF>, <LRO>456<PDF><LRM><PDF> VWX YZA

produce the corresponding results, below :

( 36 chars )  ABC DEF ‮GHI JKL MNO PQR STU VWX YZA
( 37 chars )  ABC DEF ‮GHI JKL MNO PQR STU‬ VWX YZA
( 39 chars )  ABC DEF ‮GHI JKL‬ MNO ‭PQR STU‬ VWX YZA
( 39 chars )  ABC DEF ‮GHI JKL ‭MNO PQR‬ STU‬ VWX YZA

( 46 chars )    ABC DEF ‮GHI JKL MNO PQR STU‬ 123, 456 VWX YZA
( 48 chars )    ABC DEF ‮GHI JKL MNO PQR STU‬ ‮123, 456‬ VWX YZA
( 48 chars )    ABC DEF ‮GHI JKL MNO PQR STU‬ ‭123, 456‬ VWX YZA
( 52 chars )    ABC DEF ‭‮123‬, ‮456‬‬ ‮GHI JKL MNO PQR STU‬ VWX YZA

( 47 chars )    ABC DEF ‮GHI JKL MNO PQR STU‬ ‎123, 456 VWX YZA
( 52 chars )    ABC DEF ‮GHI JKL MNO PQR STU‬ ‎‮123‬, ‎‮456‬ VWX YZA
( 49 chars )    ABC DEF ‮GHI JKL MNO PQR STU‬ ‎‮123, 456‬ VWX YZA
( 54 chars )    ABC DEF ‮GHI JKL MNO PQR STU‬ ‎‮‭123‬, ‭456‬‎‬ VWX YZA

As you may verify, by yourself, the 4 first and the 8 last results, of the last table, have, seemingly, the same width. This proves that, in all cases, the Unicode formating characters are zero-width chars ;-))

Best Regards,

guy038

P.S. :

Some of my explanations may be confusing, approximate or false. It is quite natural because this is a complex area !!

Just out of curiosity and with the precious help of this Unicode article, below :

http://www.unicode.org/reports/tr9/

I decide to explore, a bit, all these Unicode formating characters !

This Unicode technical report is very difficult to be fully understood So, of course, I’ve just got a general idea of the Unicode Bidirectional algorithm !

To begin with , I’ll use the simple text ABC DEF GHI JKL MNO PQR STU VWX YZA

In that text, I’ll insert some formating characters, using their Unicode abbreviation, surrounded with the < and > symbols, ( for instance <RLO> ). Refer to the link, below, to get all the Unicode format characters available :

http://www.unicode.org/charts/PDF/U2000.pdf

To get the formating chars themselves, in the sample text, I’ll always use the following regex S/R :

SEARCH (?-i)(<LRM>)|(<RLM>)|(<LRE>)|(<RLE>)|(<LRO>)|(<RLO>)|(<PDF>)|(<LRI>)|(<RLI>)|(<FSI>)|(<PDI>)

REPLACE (?1\x{200E})(?2\x{200F})(?3\x{202A})(?4\x{202B})(?5\x{202D})(?6\x{202E})(?7\x{202C})(?8\x{2066})(?9\x{2067})(?10\x{2068})(?11\x{2069})

First, the Explicit Directional Isolates formating characters, below, as well as the terminating associated character :

-  LEFT-TO-RIGHT ISOLATE       LRI    2066
-  RIGHT-TO-LEFT ISOLATE       RLI    2067
-  FIRST STRONG ISOLATE        FSI    2068

-  POP DIRECTIONAL ISOLATE     PDI    2069

do NOT work, in Notepad++ and are just inserted in text, as invisible characters

For instance :

ABC DEF <RLI>GHI JKL MNO PQR STU<PDI> VWX YZA

does not change the display !

ABC DEF ⁧GHI JKL MNO PQR STU⁩ VWX YZA  ( 37 chars )

Secondly, on the same way, the Explicit Directional Embeddings formating characters, below :

-  LEFT-TO-RIGHT EMBEDDING     LRE   202A
-  RIGHT-TO-LEFT EMBEDDING     RLE   202B

produce nothing and are just inserted, in text, as invisible characters

For instance :

ABC DEF <RLE>GHI JKL MNO PQR STU<PDF> VWX YZA

gives the unchanged text :

ABC DEF ‫GHI JKL MNO PQR STU‬ VWX YZA  ( 37 chars )

Luckiky, the Explicit Directional Overrides formating characters, below :

-  LEFT-TO-RIGHT OVERRIDE      LRO   202D
-  RIGHT-TO-LEFT OVERRIDE      RLO   202E
                                          
-  POP DIRECTIONAL FORMATTING  PDF   202C

Do act as expected ;-))

So, in code blocks, below, there are some examples, where :

• The first line is the initial text, before running the regex S/R

• The second line is the resulting display of the of the formating characters, after execution of the regex S/R

• All the subsequent lines explain the bi-directional process !

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

ABC DEF <RLO>GHI JKL MNO PQR STU VWX YZA  ( 36 chars )

ABC DEF ‮GHI JKL MNO PQR STU VWX YZA

─► The "ABC DEF " string is written with the DEFAULT direction of the CURRENT embedding level (level 0), so with the "Left to Right" direction

─► The "GHI JKL MNO PQR STU VWX YZA" string, from right AFTER the <RLO> formating char(level 1), is written in the "Right to Left" direction till the PARAGRAPH separator ( CRLF )

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

ABC DEF <RLO>GHI JKL MNO PQR STU<PDF> VWX YZA

ABC DEF ‮GHI JKL MNO PQR STU‬ VWX YZA  ( 37 chars )

─► The "ABC DEF " string is written with the DEFAULT direction of the CURRENT embedding level (level 0), so with the "Left to Right" direction

─► The "GHI JKL MNO PQR STU" string, from right AFTER the <RLO> formating char, till the <PDF> formating char (level 1), is written in the "Right to Left" direction

─► The " VWX YZA" string is written, again, with the DEFAULT direction ( level 0 ), so in the "Left to Right" direction

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

ABC DEF <RLO>GHI JKL<PDF> MNO <LRO>PQR STU<PDF> VWX YZA

ABC DEF ‮GHI JKL‬ MNO ‭PQR STU‬ VWX YZA  ( 39 chars )

─► The "ABC DEF " string is written with the DEFAULT direction of the CURRENT embedding level (level 0), so with the "Left to Right" direction

─► The "GHI JKL" string, from right AFTER the <RLO> formating char, till the <PDF> formating char (level 1), is written in the "Right to Left" direction

─► The " MNO " string is written with the DEFAULT direction ( level 0 ), so with the "Left to Right" direction

─► The 'PQR STU', from right AFTER the <LRO> formating char, till the <PDF> formating char (level 1), is written in the "Left to Right" direction

─► The " VWX YZA" string is written with the DEFAULT direction ( level 0 ), so in the "Left to Right" direction

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

ABC DEF <RLO>GHI JKL <LRO>MNO PQR<PDF> STU<PDF> VWX YZA

ABC DEF ‮GHI JKL ‭MNO PQR‬ STU‬ VWX YZA  ( 39 chars )

─► The "ABC DEF " string is written with the DEFAULT direction of the CURRENT embedding level (level 0), so with the "Left to Right" direction

─► The "GHI JKL MNO PQR STU" string, from right AFTER the <RLO> formating char, till the LAST <PDF> formating char (level 1), SHOULD be written in the DEFAULT "Right to Left" direction

    ─► The "GHI JKL " string, BEFORE the <LRO> formating char (level 1) is written in the "Right to Left" direction

    ─► The "MNO PQR" string, from right AFTER the <LRO> formating char, till the NEAREST <PDF> formating char (level 2) is written in the "Left to Right" direction

    ─► The " STU" string, from right AFTER the FIRST <PDF> formating char, till the LAST <PDF> formating char (level 1) is written in the "Right to Left" direction

─► The " VWX YZA" string is written with the DEFAULT direction (level 0), so in the "Left to Right" direction

In the Unicode Annex #9 article, it is said :

However, because these right-to-left scripts use digits that are written from left to right, the text is actually bidirectional: a mixture of right-to-left and left-to-right text.

So, let’s consider the initial text, below, slightly modified, by inserting the string 123, 456, in order to test this kind of configuration :

ABC DEF GHI JKL MNO PQR STU 123, 456 VWX YZA

Remark : We should use, normally, Isolates formating characters, to force weak bi-directional numbers to be written in Right to Left direction. As this solution is, unfortunately, impossible in Notepad++, here are, below, some work-around to get the same results !

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

ABC DEF <RLO>GHI JKL MNO PQR STU<PDF> 123, 456 VWX YZA

ABC DEF ‮GHI JKL MNO PQR STU‬ 123, 456 VWX YZA  ( 46 chars )

─► The "ABC DEF " string is written with the DEFAULT direction of the CURRENT embedding level (level 0), so with the "Left to Right" direction

─► The "GHI JKL MNO PQR STU" string, from right AFTER the <RLO> formating char, till the <PDF> formating char (level 1) and the context- NUMBERS string " 123, 456" SHOULD be written in the "Right to Left" direction

    ─► The "GHI JKL MNO PQR STU" string is written in the "Right to Left" direction

    ─► The numbers, in the string " 123, 456", are written, in a "Right to Left" order, too BUT each digit is displayed in it's NATURAL "Left to Right" order

─► Then, the " VWX YZA" string is written, again, with the DEFAULT direction ( level 0 ), so in the "Left to Right" direction

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

ABC DEF <RLO>GHI JKL MNO PQR STU<PDF> <RLO>123, 456<PDF> VWX YZA

ABC DEF ‮GHI JKL MNO PQR STU‬ ‮123, 456‬ VWX YZA  ( 48 chars )

─► The "ABC DEF " string is written with the DEFAULT direction of the CURRENT embedding level (level 0), so with the "Left to Right" direction

─► The "GHI JKL MNO PQR STU" string, from right AFTER the <RLO> formating char, till the <PDF> formating char (level 1), is written in the "Right to Left" direction

─► Then, the SPACE char, as a NEUTRAL bi-directional character, is written in the  CURRENT "Right to Left" direction (Level 1)

 ─► The " 123, 456" string, from right AFTER the SECOND <RLO> formating char, till the SECOND <PDF> formating char (level 1) is written, also, in the "Right to Left" direction

─► Finally, the " VWX YZA" string, after the <PDF> formating char, is written with the DEFAULT direction (level 0), so in the "Left to Right" direction

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

ABC DEF <RLO>GHI JKL MNO PQR STU<PDF> <LRO>123, 456<PDF> VWX YZA

ABC DEF ‮GHI JKL MNO PQR STU‬ ‭123, 456‬ VWX YZA  ( 48 chars )

─► The "ABC DEF " string is written with the DEFAULT direction of the CURRENT embedding level (level 0), so with the "Left to Right" direction

─► The "GHI JKL MNO PQR STU" string, from right AFTER the <RLO> formating char, till the <PDF> formating char (level 1), is written in the "Right to Left" direction

─► Then, the SPACE char, as a NEUTRAL bi-directional character, is written in the  CURRENT "Right to Left" direction (level 1)

─► The context NUMBERS string "123, 456", from right after the <LRO> formating char, till the <PDF> formating char, is also written in the 'Right to Left" direction, so BEFORE the "right to left" string "UTS RQP ONM LKJ IHG" (level 1)

    ─► But the "123, 456" string,itself, from right after the <LRO> char, till the <PDF> formating char (level 2) is written in the "Left to Right" direction

─► The " VWX YZA" string, after the <PDF> formating char, is written with the DEFAULT direction (level 0), so in the "Left to Right" direction

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

ABC DEF <LRO><RLO>123<PDF>, <RLO>456<PDF><PDF> <RLO>GHI JKL MNO PQR STU<PDF> VWX YZA

ABC DEF ‭‮123‬, ‮456‬‬ ‮GHI JKL MNO PQR STU‬ VWX YZA  ( 52 chars )

─► The "ABC DEF " string is written with the DEFAULT direction of the CURRENT embedding level (level 0), so with the "Left to Right" direction

─► The context NUMBERS string "123, 456", from right after the FIRST <LRO> formating char (level 1), till the THIRD <PDF> formating char, SHOULD be written in the 'Left to Right" direction

    ─► Each digit of the "123" string, from right after a <RLO> formating char, till its corresponding <PDF> formating char (level 2), is written in the "Right to Left" direction

    ─► The sting ", " as NEUTRAL chars, are rewritten in the CURRENT direction of level 1, so in the "Left to Right" direction

    ─► Again, each digit of the "456" string, from right after a <RLO> formating char, till its corresponding <PDF> formating char (level 2), is written in the "Right to Left" direction

─► Then, the SPACE char, as NEUTRAL character, is written in the  CURRENT "Left to Right" direction (level 0)

─► The "GHI JKL MNO PQR STU" string, from right AFTER the <RLO> formating char, till the <PDF> formating char (level 1), is written in the "Right to Left" direction

─► Finally, the " VWX YZA" string, after the <PDF> formating char, is written with the DEFAULT direction (level 0), so in the "Left to Right" direction

More on the next post !

guy038

When I said :

Don’t ask me what these special characters mean !

Of course, Peter, I could have gone ahead for Google search or, even, on the site http://www.unicode.org and post some information about ! But, first, I was a bit lazy about doing so and, secondly, as you know, at the moment, in France, there is a period of heatwave ( 45° forecast, this afternoon, in Nîmes ). So, everyone is extremely tired with such temperatures :-((

But, luckily, you provided the right info, brilliantly and, certainly, in more fluent American-English that I could produce, anyway !

Now, regarding the heatwave period, in France, to get an idea of the situation, just have a look to these two pictures :

http://www.meteociel.fr/cartes_obs/archives/27-06-2019/temp-17.png

http://www.meteociel.fr/cartes_obs/archives/27-06-2019/temp_na-22.png

As you see, I’m afraid that real temperature, in some parts of France, seem equivalent to the present Death valley temperature !!

But, as Donald Trump says, there is no global warming underway ! Hummm ………

For present France temperatures ( 13h00 local ), see below :

http://www.meteociel.fr/observations-meteo/temperatures.php

Best Regards,

guy038

two invisible zero-length characters

You’re right. I didn’t go looking for others, but the second was there.

Don’t ask me what these special characters mean !

I had a guess, but did a quick search to confirm. See the W3C page about unicode bidirectional control

It looks like LRO forces left-to-right direction – so if you happened to be in a Right To Left language, the LRO would force these characters to be in left-to-right instead. And the PDF (warning: acronym collision!) “pops” back out of that mode, returning to whatever directional setting was previously active.

My guess is that some setting for the OP made the MS Calc want to ensure that the hex sequence was given in big-endian order for that particular copy/paste (or that the OP went through another app between MS Calc and Notepad++ which wanted to ensure that).

------

Off Topic Unicode Exploration

Now I want to explore those bidir characters slightly

Starting with

ABCD (<LRO>ABCD<PDF>) ABCD
ABCD (<RLO>ABCD<PDF>) ABCD

Then doing three search/replace:

• <PDF> => \x{202C}
• <LRO> => \x{202D}
• <RLO> => \x{202E}

It will become:

ABCD (‭ABCD‬) ABCD
ABCD (‮ABCD‬) ABCD

which, if your browser understands unicode properly, should appear to be

ABCD (ABCD) ABCD
ABCD (DCBA) ABCD

but will have the extra characters, and the bytes will be in ABCD order every time. And for me, Notepad++ honored the RLO properly, and rendered the text in that order. Nifty.

Nifty stuff.

Quite weird, indeed ! If we just take care to the part $1ff5, we have :

• The string $1ff5, of 5 characters long

• The string $‭1ff5‬, of 7 characters long

The former is just normal text and the later contains two invisible zero-length characters, which are :

• The Unicode LEFT-TO-RIGHT OVERRIDE character ( LRO ), of code-point \x{202D}, located after the $ symbol

• The Unicode POP DIRECTIONAL FORMATTING character ( PDF ), of code-point \x{202C}, located between digit 5 and the EOL chars

Refer, for that purpose, to :

http://www.unicode.org/charts/PDF/U2000.pdf

So, the sequence of 7 characters, followed with its line-break, below :

$‭1ff5‬
--

could be found with the regex \$\x{202D}1ff5\x{202C}\R

Don’t ask me what these special characters mean !

I agree, Peter, that your script is the right one to use and helps a lot to find out all these special chars ;-))

Best Regards,

guy038

However, there are Notepad++ features which may help you figure out what’s going wrong.

• Firstly, good choice in showing whitespace and EOL – that’s a good use of Notepad++ to help. Did you notice the extra whitespace on the second line before the CRLF? I don’t remember enough assembly from 25 years ago to know if that would affect things, but it jumps out as me as a difference.
• Secondly, the syntax highlighting is different on the two lines. I think something is going on there. You might want to try changing fonts for the assembly style configurator… my first guess would have been you were using a font that didn’t differntiate 1 and lower-case L

Ooh, I just copied your text over, and saw the difference in highlighting myself… and noticed something funky with the second $ being slightly darker. My font distinguishes 1 and L, so it was properlly a 1. But I backspaced the 1 and accidentally did a second backspace, and a zero-width character was deleted: there’s something between the $ and the 1 on the second line! I pulled up my PythonScript that’s in this recent post to look for the unicode characters. It tells me that, at least when I copy from your post and paste into a UTF-8 Notepad++ file, I see the U+202D, which is apparently the left to right override character. I think that character is your problem.