Performance hit due to "Allow undo on reload"

SinghRajenM

This new implementation is causing huge performance hit. Analysis is below (using 32 bit) -
Opening a plain approx 30MB file consumes around 90MB RAM. Now

list item Reloading this file 15 times, brings RAM utilize to 1.2 GB. And reloading around 23~25 times causes “File is too large to open” error.
list itemWhen Auto detect is enabled along with “Update silently”, then same file (30MB) after 23~25 will complain about “File is too large to open” error.
Memory consumption increases exponentially on each reload.

This is really a performance hit which needs attention and MUST be fixed.

dinkumoil

Seems like this new feature has to be reverted or buffering of the old file content has to be moved over to the hard disk. But that would mean it would nearly double the automatic backup feature.

SinghRajenM

buffering of the old file content has to be moved over to the hard disk

@dinkumoil when we have verbose backup available, this feature was not required at all. If someone is really concern about the old file content after the reload, then go to backup and pick whatever s/he is interested in.

Meta Chuh

@SinghRajenM

i can understand your concerns.

let’s focus on this calmly with the information we have:

reloading an unchanged file does not raise memory consumption on 7.6.6.
monitoring a file should not populate the undo buffer in 7.6.6 and should not produce an elevated memory consumption compared to older versions.
the file given at the issue that caused this reaction is a kind of log file, which should be monitored instead of being edited.
being able to undo on an accidentally reloaded file is a feature that many people wanted and wished they had in the past.

any further testing and brainstorming is appreciated.

many thanks and best regards.

SinghRajenM

@Meta-Chuh

reloading an unchanged file does not raise memory consumption on 7.6.6.

No, this is not true… I just created a 5MB (approx) file using command prompt fsutil file createnew largefile.txt 5000000 and trying to reload it. It consume huge memory.

Action size

Open npp (empty) 20MB
Open one 5MB file 60MB
1st crtl+R 70MB
2nd crtl+R 80MB
3rd crtl+R 90MB
10th crtl+R 160MB

Above result are based on x86 debug version with latest code. Result may differ on relase version and x64 version.

dinkumoil

@Meta-Chuh said:

being able to undo on an accidentally reloaded file is a feature that many people wanted and wished they had in the past

People often want things and don’t think about the consequences. Programs running stable is more important than having fancy features which counteract that. The current situation can cause data loss even after working some hours with a mid size file, the worst case for a text editor.

Meta Chuh

@SinghRajenM

you are correct.

opening the file for the first time seems to have the same memory consumption on 7.6.4, but each reloading the unchanged file consumes more memory on 7.6.6.

this is indeed a big and intriguing problem.
i have now tried it with your larger file fsutil nul file example and a 75mb test file.

stock npp.7.6.4.bin on w7 x64:
notepad++, change.log open: 6.5mb
load a 75mb test file: 165mb
reload file: still appx the same 165mb

Notepad++ v7.6.4   (32-bit)
Build time : Mar  6 2019 - 02:52:22
Path : C:\Users\User\Desktop\npp.7.6.4.bin\notepad++.exe
Admin mode : OFF
Local Conf mode : ON
OS : Windows 7 (64-bit)
Plugins : mimeTools.dll nppConverter.dll NppExport.dll

stock npp.7.6.6.bin on w7 x64:
notepad++, change.log open: 6.6mb
load a 75mb test file: 159mb
1 reload file: appx 310mb
2nd reload file: appx 462mb
3rd reload file: appx 614mb

Notepad++ v7.6.6   (32-bit)
Build time : Apr  3 2019 - 23:49:50
Path : C:\Users\User\Desktop\npp.7.6.6.bin\notepad++.exe
Admin mode : OFF
Local Conf mode : ON
OS : Windows 7 (64-bit)
Plugins : ComparePlugin.dll mimeTools.dll

many thanks and best regards.

Alan Kilborn

This seems like a significant problem. Is it getting any real attention? I see it at https://github.com/notepad-plus-plus/notepad-plus-plus/issues/5595 but not much activity?

Meta Chuh

@Alan-Kilborn

yes, it has real attention.

true, the significance has diminished a bit, as the worst thing that can happen to this feature of keeping the buffer after a reload, is to be reverted as suggested, if no-one can figure out and code a better performing and working alternative, than to outsource the undo buffer to disk, or clear the undo buffers manually, or clear them after a dialog triggered by a specific memory threshold.

any ideas and/or commits are welcome, as we are in a stage of one group saying it’s a cool feature not to remove, and the other group is running into memory issues because of it.