As @decoderman said, you cannot edit it directly. However, you can
run regedit (Registry Editor)
Select the appropriate registry entry, or hierarchy level
File | Export > File name = blah.reg
Open blah.reg in Notepad++
Edit the text
save
exit
Either
In file explorer, double-click on blah.reg to auto-import it
or, in RegEdit, File | Import > File name = blah.reg
However, some registry edits require further action (like a reboot) to take effect.
If you already have the .reg file, you can just edit it directly in Notepad++, and then follow the Either-instructions to bring it into the active registry.
WARNING: If this wasn’t intuitively obvious to you to begin with – if anything I said above came as new information (e.g., if you didn’t already know how to export/import .reg files or if you didn’t already know that .reg files were text files so could be edited directly with a text editor like Notepad++) – I would then strongly caution you against manual registry edits, because editing your registry is something to be done only if you know what you’re doing, and can cause your system to stop working. Use these instructions at your own risk. I make no warranty, express or implied.
