Wow, I can’t believe it’s been 2 years since I ran into this issue…anyway I finally figured out a working fix.

To fix the signed integer anomaly, we apply a bitwise AND mask (& 0xFF). This converts Scintilla’s negative integers back into true unsigned byte values (0 to 255) before string formatting. The entire conversion happens entirely in memory and replaces the target range in a single, atomic undo action.

How to Use:
Open a binary file, highlight the bytes you want to convert, then run the script. All ÿ and NUL bytes will successfully be converted.

Cheers :)

# -*- coding: utf-8 -*-
# Notepad++ PythonScript: Convert selected bytes to hex, skip 0xFF (ÿ) and 0x00 (NUL). Substite with FF and 00 on second pass.

from Npp import *

def main():
    editor.beginUndoAction()
    try:
        start = editor.getSelectionStart()
        end = editor.getSelectionEnd()
        if start == end:
            return

        hex_parts = []
        
        for pos in range(start, end):
            raw_b = editor.getCharAt(pos)
            b = raw_b & 0xFF  # Keep the unsigned 0-255 byte conversion
                
            if b == 0xFF:
                # Direct conversion: No more placeholder strings needed!
                hex_parts.append("FF")  
            else:
                # Flawless 2-digit Hex formatting for NUL (00) and other bytes
                hex_parts.append("%02X" % b)  
                
        # Write the finalized, pure hex block directly to the document
        editor.setTargetStart(start)
        editor.setTargetEnd(end)
        editor.replaceTarget(''.join(hex_parts))
        
    finally:
        editor.endUndoAction()

if __name__ == '__main__':
    main()