How do you revert command symbols (e.g. NUL) to actual characters?

[DeeFeeCee]

I’ve been meaning to take apart some code from a file (.ldb) & need to read the hex code to do so. Hex editor has been a pain for some reason, so I resolved to create my own font (I managed to make characters like null to appear as “0 0”).

The problem is that when I apply the font, instead of seeing “0 0” for null, I see the hex for the letters “NUL”. This is silly. How can I resolve this?

[andrecool-68]

The hex editor will write the value “NUL” as “0” … that’s right!

[andrecool-68]

There are just a few font types that work well in any Hex editor.

[PeterJones]

@DeeFeeCee said in How do you revert command symbols (e.g. NUL) to actual characters?:

The problem is that when I apply the font, instead of seeing “0 0” for null, I see the hex for the letters “NUL”. This is silly. How can I resolve this?

Not really sure. You haven’t told us how you font-plus-whatever works, so we cannot know what you may or may not be doing right and wrong.

The HexEditor plugin available for the 32-bit Notepad++ via the Plugins Admin properly handles NUL characters in the “text” (in quotes, because if it’s got a NUL, is it really text?). See the screengrab below for seeing the HexEditor in action:

Just so you know, the Scintilla editor component (the library that Notepad++ uses for implementing the editor portion of the application) defaults to rendering control characters (like NUL, SOH, EOT, …) with their control-code abbreviation in the little black box. I don’t know that if you have a font with the NUL character, whether Scintilla will just use that, instead of using it’s representation. Using one of the scripting plugins, like PythonScript, you can change Scintilla’s representation of any character. For example, in PythonScript, editor.setRepresentation('\x00', '000'):

But honestly, in all but the simplest use cases, I’ve found that there are better tools out there than Notepad++-with-HexEditorPlugin or Notepad++-with-custom-rolled-hex-editing-plugin-or-script. (HxD is where I turn, on the rare occasion when I need a true hex editor, rather than just a brief hex representation of the file)

[DeeFeeCee]

@andrecool-68 My font literally renders null as 00. If I type [null] [space] !, I’ll see 00 20 21. My hex editor has been having all sorts of bugs which render it unusable. I’d rather be able to switch to a font that shows the hex for each character rather than using a limited editor plugin.

[PeterJones]

@DeeFeeCee ,

My font literally renders null as 00. If I type [null] [space] !, I’ll see 00 20 21.

I think I know what’s going on. By default, as I said, Scintilla (the editor component) renders control characters as their 2-to-3-letter abbreviation in a black box, so the NUL (0x00) character renders as NUL. But if you’ve chosen a font that has a glyph for the character at codepoint 0, Sctintilla won’t know to use it, because ASCII/Unicode define those characters as control characters, not characters to display. Since internally, Scintilla uses the three letters N, U, and L in the current font (though obviously styled differently, to give the black box – see my animation above) for representing the NULL character, then your font will render that as the hex codes for N, U, and L, because those are what are being displayed.

Using the PythonScript plugin, you might be able to force Scintilla to not use a special representation by doing editor.clearRepresentation('\x00'), which stops scintilla from having a special representation for the NULL character. As to whether it will “correctly” (in your mind) render the glyph you created in your font, or “correctly” (in the minds of the definers of ASCII and Unicode) not render control characters with a glyph is something I don’t know, and you’d have to experiment with that.

[DeeFeeCee]

@PeterJones Hey, thanks for responding! My font literally renders x00 as 0 over 0. As in the video, Notepad++ (64-bit) uses the letters N U L to represent the null, which is the issue. It also seems like the characters from x80–x9F are also rendered as their “replacement” characters; Ÿ is the hex representation for 9F, but the character Ÿ itself is… looking it up U+0178. There are also characters lurking underneath those characters, like x1F & x20, which doesn’t make sense to me. Your code looks like it might work, I just don’t have time today to try it out. I really liked the animation, too!

As I said to @andrecool-68, the hex editor plugin is broken. If you want I can send you a video (preferably a screenshot instead of iPhone footage). Sufficeably, I can’t even see the Unicode numerals, making it useless. I’d rather edit hex in the vanilla Notepad++ interface anyway.

[PeterJones]

@DeeFeeCee said in How do you revert command symbols (e.g. NUL) to actual characters?:

If you want I can send you a video

Many users have problems with the HexEditor plugin (though some others use it successfully). I don’t know that we need a video to prove that.

It also seems like the characters from x80–x9F are also rendered as their “replacement” characters

x00-x1F and x7F-x9F are defined in the Unicode standard as control characters, which is why they are rendered as their “replacement” characters.

Ÿ is the hex representation for 9F, but the character Ÿ itself is… looking it up U+0178

In some of the ANSI-style 8-bit encodings, Ÿ was at codepoint 159 (x9F). In Unicode, Ÿ is U+0178.

Your code looks like it might work, I just don’t have time today to try it out.

Hope it works for you. If it does, and you are able to do animated screenshot, that would be great to share

I really liked the animation, too!

I use screentogif.exe, which is free and really easy to use.