Control symbols now inserted on macro

[PeterJones]

@guy038 ,

I tried (with fresh v7.8.1-64 zip, windows 10) to copy/paste your macro into shortcuts.xml (and reload – not shown), and tried running that macro; still shows the bug. Then I tried to record the macro using the ALT+0133, then playing back that macro: it does the same thing. As I experimented with earlier, the macro works just fine in v7.7.1.

So, if you’re good at the ALT+#### keysequences, and can remember it, and have a keypad, that’a a valid alternative for a single macro that only inserts the ellipsis. But if any of those conditions aren’t met (including if the ellipsis entry is just part of a larger macro), then the bug in v7.8.1 can insert extra characters (whatever happens to be after that ellipsis in random memory, it appears, because the extra chars change with each program load)

[guy038]

Hi, @oasisindesert, @peterjones and @alan-kilborn,

I did some other tests, including a short text before and after the ellipsis character and… everything still went right !?

So, Peter, if you don’t mind, I’m going to e-mail you a zip file contening the macro in the shortcuts.xml file and all the minimum amount of files, needed for the N++ 7.8.1 version 32 bits :-))

This zip file, named N++_781.zip ( 1,85 Mb ), contains the following elements ( minimum required ) :

    File Name               Path                    Size       Date Modified
  ----------------------------------------------------------------------------
    change.log              N++_781                 2,786       12/11/2019
    config.xml              N++_781                 5,218       28/11/2019
    contextMenu.xml         N++_781                 3,512       25/05/2018
    doLocalConf.xml         N++_781                     0       16/08/2017
    langs.model.xml         N++_781               336,651       27/09/2019
    notepad++.exe           N++_781             2,860,176       27/10/2019
    readme.txt              N++_781                 1,450       26/12/2016
    SciLexer.dll            N++_781             1,264,784       27/10/2019
    session.xml             N++_781                   656       28/11/2019
    shortcuts.xml           N++_781                 1,909       28/11/2019
    stylers.model.xml       N++_781               168,547       27/09/2019
    
    autoCompletion          N++_781                     0       28/11/2019
    
    localization            N++_781                     0       28/11/2019
    english.xml             N++_781\localization   76,503       23/10/2019
    
    plugins                 N++_781                     0       28/11/2019
    Config                  N++_781\plugins             0       28/11/2019
    
    themes                  N++_781                     0       28/11/2019
    updater                 N++_781                     0       28/11/2019
    userDefineLangs         N++_781                     0       28/11/2019
  ----------------------------------------------------------------------------
    Total                                       4,722,192 Bytes
  ----------------------------------------------------------------------------

So, Peter :

• Extract all contents of the N++_781.zip archive, let’s say,; on your destop

• Double click on the extracted N++_781 folder

• Rename the Notepad++.guy file as Notepad++.exe     ( Step added the next day : see below for explanations ! )

• Double click on Notepad++.exe ( 32 bits version )

• In any already opened file or a new file, just hit the Ctrl + Alt + . shortcut to activate the Ellipsis macro

=> Any shortcut hit writes the string ATGC…ATGC, preceded and followed with a space char, without any problem !

---

Now, Alan and @oasisindesert, if you would test my archive file, just e-mail me a short message to :

Best regards

guy038

[guy038]

Hi, @oasisindesert, @peterjones and @alan-kilborn,

I should have tried to e-mail my zip file first, because while sending to Peter, I got the message :

The message was blocked because its content presents a potential 5.7.0 security issue ?!

I even tried to rename the N++_781.zip as N++_781.guy, but no chance, either :-(

---

Now, may be this behavior only happens with a x64 version ? I’m really not convinced !

BR

guy038

[guy038]

Hi, @oasisindesert, @peterjones and @alan-kilborn,

This morning, an idea popped up in my brain ! Why don’t I try to rename notepad++.exe as notepad++.guy, in the N++_781.zip archive ?

Bingo ! This time, this attached archive was accepted ;-)) So, Peter, just an additional step to do :

• Once you have opened the N++_781 folder, rename the notepad++.guy file as notepad++.exe

You should be able to test my version !

Cheers,

guy038

[PeterJones]

@guy038 said in Control symbols now inserted on macro:

Now, may be this behavior only happens with a x64 version ? I’m really not convinced !

I am convinced.

With either the npp 7.8.1 32-bit portable that Guy sent me, or a fresh zip copy from the download page , the 32-bit works with either the pre-packaged shortcuts.xml that Guy sent, or doing a new macro recording of ALT+0133. If I take that exact same shortcuts.xml, and copy it into a fresh unzip of v7.8.1-64bit, the macro shows the bug we’ve been describing.

Looking at issue#7642 , it appears that @xylographe agrees that it happens on 64-bit and not 32-bit, but @rddim doesn’t see it on 64-bit on Win7. Mine is Win 10. So it might be a 64-bit Win10-only instance of the bug. Wow.

[Ekopalypse]

Might be an ANSI vs. unicode related issue as well?

[PeterJones]

@Ekopalypse said in Control symbols now inserted on macro:

Might be an ANSI vs. unicode related issue as well?

Brilliant.

I just took Guy’s edition, and ran his macro once in a new file with Encoding > ANSI set, and it worked as Guy described. Then I made a new file, set Encoding > UTF-8, and the bug showed itself – same load of the program, same shortcuts.xml.

[guy038]

Hi, @oasisindesert, @peterjones, @alan-kilborn, @ekopalypse and all

Peter, by rereading your last post, this morning, you said :

I just took Guy’s edition, and ran his macro once in a new file with Encoding > ANSI set, and it worked as Guy described. Then I made a new file, set Encoding > UTF-8, and the bug showed itself – same load of the program, same shortcuts.xml.

May be, I’m missing something obvious !

---

Opening N++ with the archive that I sent you, the default encoding of new 1 is UTF-8

But, after numerous tests, using, first, any option of the Encoding menu and, even, the options Encoding > Character Set > ... > from Windows-1250 to Windows-1258, which stand for the main ANSI encodings, then running the macro, I’ve never noticed any control character wrongly inserted !?

So, I suppose that the main point is to have an old 32-bit operating system and, therefore, to use the 32 bits version of N++, as well. In that case, the bug never seems to be happening !

Best Regards

guy038

[Ekopalypse]

So, to summarize, the current state is that

32bit version is not affected at all
64bit version works with document codec set to one of the 8bit encodings
and the troublesome is
64bit version with documents having unicode encoding.

Which leads to the assumption that macros working on a unicode text
with 64bit npp version has a bug.

[PeterJones]

@Ekopalypse said in Control symbols now inserted on macro:

32bit version is not affected at all

I disagree. As described above, by manually setting the Encoding > UTF-8 toggle, I replicated the bug in 7.8.1-32bit

[Ekopalypse]

Not sure if this is helpful in this case but using
Linux and Wine I don’t see the problem on a 64bit version at all.

Notepad++ v7.8.1 (64-bit)
Build time : Oct 27 2019 - 22:57:19
Path : Z:\home…\notepad-plus-plus\notepad-plus-plus.exe
Admin mode : ON
Local Conf mode : ON
OS Name : Microsoft Windows 7 (64-bit)
OS Build : 7601.0
WINE : 3.0.4
Plugins : mimeTools.dll NppConverter.dll NppExport.dll

[PeterJones]

@Ekopalypse : weird.

To re-confirm what I said earlier today, here’s a screenshot of the bug in action in NPP-32bit on Win10-64bit, showing encoding-dependency:

• @PeterJones: With both 7.8.1-64bit and 7.8.1-32bit on 64bit Win10 Home 1903 18362.476,
  • Shows with Encoding > UTF-8 but not with Encoding > ANSI
• @guy038: In WinXP 32bit (@guy038), 7.8.1-32bit, it appears to not show the bug
• @Ekopalypse: In Linux WINE 3.0.4 with Guest=Win7-64bit 7601.0, NPP 7.8.1-64bit, the bug does not show in either UTF-8 or ANSI

Strange bug.

[PeterJones]

updated https://github.com/notepad-plus-plus/notepad-plus-plus/issues/7642 with the new screenshot and the summary of my experiments, @guy038 results, and @Ekopalypse results (ie, basically, copied my last post to github issue)

[Ekopalypse]

@PeterJones

agreed - very strange. I’m wondering if XP and Wine share the same code
for the needed functionality is this case.
Would make some sense. Maybe some older unicode library or different than W7 and W10.
I thought about it for some time now, but I don’t see
how to start on this to track down the bug.

[andrecool-68]

@Ekopalypse There has been a problem with the definition of encodings for a very long time, sometimes it is solved but partially))
And then a new surprise appears with encodings))

[Ekopalypse]

@andrecool-68 said in Control symbols now inserted on macro:

There has been a problem with the definition of encodings for a very long time

:-) I agree but it has reached a new level I would say. :-(
I can live with limitation if I understand why their are,
but fishing in the dark for understanding drives me mad.

[andrecool-68]

@Ekopalypse This is no longer fishing … it’s sex with a concrete wall))

[Ekopalypse]

@andrecool-68
:-D ouch

[Alan Kilborn]

I finally was able to get this problem to replicate under debugger control (what I was missing before was that my file was coming up with ANSI encoding preselected – and I didn’t notice that; now when the debugger starts up I go in to the N++ Encoding menu and change it to UTF-8).

Here’s the problem:

void recordedMacroStep::PlayBack(Window* pNotepad, ScintillaEditView *pEditView)
{
	if (_macroType == mtMenuCommand)
	{
		::SendMessage(pNotepad->getHSelf(), WM_COMMAND, _wParameter, 0);
	}
	else
	{
		// Ensure it's macroable message before send it
		if (!isMacroable())
			return;

		if (_macroType == mtUseSParameter) 
		{
			char ansiBuffer[3];
			::WideCharToMultiByte(static_cast<UINT>(pEditView->execute(SCI_GETCODEPAGE)), 0, _sParameter.c_str(), -1, ansiBuffer, 3, NULL, NULL);

The ::WideCharToMultiByte call returns 0 – indicating failure – and then a call to GetLastError returns 122 which is ERROR_INSUFFICIENT_BUFFER.

Indeed, the size of ansiBuffer, i.e., 3, is one too small for this case.

Changing it to 4 cures the problem for the UTF-8 files:

char ansiBuffer[4];
::WideCharToMultiByte(static_cast<UINT>(pEditView->execute(SCI_GETCODEPAGE)), 0, _sParameter.c_str(), -1, ansiBuffer, 4, NULL, NULL);

[PeterJones]

@Alan-Kilborn said in Control symbols now inserted on macro:

recordedMacroStep::PlayBack
… ansiBuffer[4];

That makes some sense. At first I was wondering what change prompted that to start failing, and the GitHub blame shows those two lines haven’t been changed in 3-4 years. But really, if Scintilla changed the way it’s doing things, maybe the _sParameter.c_str() now has a 4byte char rather than a 3byte char, or something.

Alternately, MS Docs: WideCharToMultiByte shows that sending cbMultiByte (the 3 or 4 in the last arg before the NULLs) as a 0 will have it return the width needed, so wouldn’t it be more future proof to use:

int ansiBufferLength = ::WideCharToMultiByte(static_cast<UINT>(pEditView->execute(SCI_GETCODEPAGE)), 0, _sParameter.c_str(), -1, ansiBuffer, 0, NULL, NULL);  // grab needed ansiBuffer length
::WideCharToMultiByte(static_cast<UINT>(pEditView->execute(SCI_GETCODEPAGE)), 0, _sParameter.c_str(), -1, ansiBuffer, ansiBufferLength, NULL, NULL);

(though you’d need to either re-dim ansiBuffer, or make sure it’s always got enough room)