Fix corrupted txt file (NULL)
-
@pnedev
I agree with your analysis and reasoning. Thanks for the detailed response. I used Recuva with deep scan. It did not help. As you said nothing much to do at this point.
lessons learnt. -
And at last, as a safety measure, I started using google drive backup and sync. I am a PHP developer, so all my files to be edited are in htdocs folder. gdrive backup and sync monitors the folder for file changes and starts to upload the file to drive as soon as the file contents are changed or the date modified is changed, I don’t know which one it is. But this has been serving me well.
-
How many people does it take?
It has happened to me not once, not twice, but now three times in the past 4 months, and the last most critical.
- First time was when saving previously unsaved NPP files I had open for a computer repair… saved with a name, but when I restored the files later, the file I saved it as was only nulls. Figured it might be related to the backup writing or such, so didn’t think much of it, but lost some important life lessons I’d typed up.
- Second time was an entirely unsaved NPP file I’d put some complex commands into for FFMPEG that I had worked out while I wrote together a more flexible processing script. Lost hours of fiddling that got the right settings, but figured because I hadn’t been saving at all, it was my own fault, and got in the habit of saving constantly.
- And now the third time… lost 2 weeks of heavy work put into a main script for a new business I had been passionately working on. This time it was a saved file. I had saved it maybe 15 minutes earlier, and hadn’t edited it since, just hadn’t closed it.
Lost probably 50 hours of work. Tried Recuva/deep scan/dump search/etc.
Killing me. It was a saved file.
I am guilty of cold killing the computer with the power button when sometimes the system becomes unresponsive (perhaps due to heavy Opera/ffmpeg/etc usage… but more likely the frustrating Windows/antivirus background processes).
And I am guilty of not backing up constantly. I did backup the folder a couple weeks ago… just before I started writing this specific script. I was putting in brutal hours to get the videos processed and other key aspects of the new venture done in time, and hadn’t had the time to setting up a more robust backup solution yet. That said… if this keeps killing people who know to do backups… how many amateurs/students/etc will this decimate before it’s considered important enough by @donho and others to make it a priority?Definitely looks like it’s write caching doing us all in. I’ll turn it off, but it’s a reactive measure other users won’t know to do (and a chore to remember every install, every machine, in addition to the costs of such action).
If the Win32 API use would endanger things by being a regression, I still don’t understand why some intermediate option such as comparing the backup file to the buffer version before going ahead with overwriting the main file (to at least prevent back dataloss), or verifying the backup version has changed, or that it is not a corrupted/null filled, before overwriting the current file on restart aren’t viable??? Maybe such a fix would slow the execution down a bit, but the value of being able to rely on your editor not to overwrite what may be vital files seems beyond compare.Or else, if we’re not going to have reliable backups, just scrap the whole backup setup. Because when you need it most, it fails… and to even overwrite previous files… it’s awful. That any file you edit in N++ may not only lose unsaved work, but overwrite the existing file? Begging and pleading it becomes a priority. How many more disaster stories are needed!?!
Looks like there has been 1 minor and at least 6 patch releases since the work at solving this. I understand the developers are doing this of their own kindness and selflessness… but what other updates are more vital???@pnedev, thank you so much for your persistence on the issue, I see you reopened the commit, and have pleaded with don to make it happen. I really hope it comes through. Too late for me (this time), but hopefully your work will save many many thousands of hours of people’s lives.
-
@Shane-Young said in Fix corrupted txt file (NULL):
How many people does it take?
Even an infinite number of people reporting the problem here in the forum will likely do nothing.
The only official bug and issue tracking happens on GitHub. Comment on the official issue tracker – for example, in the still-open issue #6133 – begging that something be done.
But complaining about it here will do nothing. Every regular in this forum that has expressed an opinion agrees that something needs to be done to fix it – but all the agreeing opinions in the world expressed in this forum will do nothing to fix the problem. Our best bet right now is @pnedev’s fix, but the only hope for that is to convince Don that his fix won’t break other things.
-
Just merge the fix of @pnedev into master:
https://github.com/notepad-plus-plus/notepad-plus-plus/commit/a1031517742c7ef600b825bf9f193b3f2740dff6The fix will come with v7.9.1.
Thank you @pnedev for your work! -
At the Moment my blood is boiling. I lost 3 days worth of hard work on a Saved file, because of an Unexpected Power outage although I was on UPS backup. And I open the file to find all NULL characters. I have been frantically trying to recover a previous version, and rampaging around googling forums, and tips. Just tried Recuva, with different settings, but it is not able to recover, since the file has been over-written. Have given up that what is gone is gone. Notepad++ has devoured 3 days of prime time in my life, and I have to do everything all over again, if that is even possible.
I am surprised that this problem has been active on this thread since 16-Feb-2017, and who knows if it was even earlier.
Notepad++ Developers, Contributors, Volunteers, yes, your Program is a Fantastic program and now probably very widely used throughout the world.
BUT GET THIS INTO YOUR HEADS, THAT WHAT IS MOST IMPORTANT, MORE IMPORTANT THAN ANY FEATURES AND FRILLS YOU PROVIDE, IS ‘DATA INTEGRITY’, ‘STABILITY’, ‘RELIABILITY’, DEPENDABILITY. IF YOU DON’T PROVIDE WHATEVER FEATURE AND FRILL YOU PROVIDE ON THE BACK OF THESE FUNDAMENTALS, THEN ALL YOU GIVE IS ONLY HOGWASH, AND YOU WILL ONLY HAVE USERS CURSING YOU, AND WISHING THEY HAD NEVER USED YOUR PROGRAM EVER AT ALL.
Can’t you even check what other Editors do. Even MsWord, Excel and the like are able to diagnose Data Corruption, and offer an AutoRecover feature. With Notepad++, even with Backups on, and every setting you can find ‘on’, and with Regular saving of the File, still we are stuck with Null Files.
Is this some Advanced Coding Logic?? To me, this is something even a Numbskull would think of.
When you open a file, can you not program some logic, and a list of parameters that would check the file versions opened for every kind of corruption in our human experience, and confirm file integrity first.
Especially when the previous Closing of Notepad has not been ‘normal’.
If you sense File Corruption, Session Corruption why don’t you alert the User, and have some Options ready for them to check and recover, or have some algorithms by which you can reasonably well confirm the best previous reliable state. Before you go on this filling with Null Characters over-drive, and destroying files and data. Is it because you don’t care because it is free, open-source, and people have to just bow to you and keep thanking you because it is free… -
@Shane-Young Loved you reply… Let’s hope they do something… I too just lost precious data, and am still frantically searching for some way to recover. Posted my comments below…
At the Moment my blood is boiling. I lost 3 days worth of hard work on a Saved file, because of an Unexpected Power outage although I was on UPS backup. And I open the file to find all NULL characters. I have been frantically trying to recover a previous version, and rampaging around googling forums, and tips. Just tried Recuva, with different settings, but it is not able to recover, since the file has been over-written. Have given up that what is gone is gone. Notepad++ has devoured 3 days of prime time in my life, and I have to do everything all over again, if that is even possible.
I am surprised that this problem has been active on this thread since 16-Feb-2017, and who knows if it was even earlier.
Notepad++ Developers, Contributors, Volunteers, yes, your Program is a Fantastic program and now probably very widely used throughout the world.
BUT GET THIS INTO YOUR HEADS, THAT WHAT IS MOST IMPORTANT, MORE IMPORTANT THAN ANY FEATURES AND FRILLS YOU PROVIDE, IS ‘DATA INTEGRITY’, ‘STABILITY’, ‘RELIABILITY’, DEPENDABILITY. IF YOU DON’T PROVIDE WHATEVER FEATURE AND FRILL YOU PROVIDE ON THE BACK OF THESE FUNDAMENTALS, THEN ALL YOU GIVE IS ONLY HOGWASH, AND YOU WILL ONLY HAVE USERS CURSING YOU, AND WISHING THEY HAD NEVER USED YOUR PROGRAM EVER AT ALL.
Can’t you even check what other Editors do. Even MsWord, Excel and the like are able to diagnose Data Corruption, and offer an AutoRecover feature. With Notepad++, even with Backups on, and every setting you can find ‘on’, and with Regular saving of the File, still we are stuck with Null Files.
Is this some Advanced Coding Logic?? To me, this is something even a Numbskull would think of.
When you open a file, can you not program some logic, and a list of parameters that would check the file versions opened for every kind of corruption in our human experience, and confirm file integrity first.
Especially when the previous Closing of Notepad has not been ‘normal’.
If you sense File Corruption, Session Corruption why don’t you alert the User, and have some Options ready for them to check and recover, or have some algorithms by which you can reasonably well confirm the best previous reliable state. Before you go on this filling with Null Characters over-drive, and destroying files and data. Is it because you don’t care because it is free, open-source, and people have to just bow to you and keep thanking you because it is free… -
@Ben Your comments are spot on… Let’s hope they do something… I too just lost precious data, and am still frantically searching for some way to recover. Posted my comments below… Did you get any positive feedback from GitHub.
At the Moment my blood is boiling. I lost 3 days worth of hard work on a Saved file, because of an Unexpected Power outage although I was on UPS backup. And I open the file to find all NULL characters. I have been frantically trying to recover a previous version, and rampaging around googling forums, and tips. Just tried Recuva, with different settings, but it is not able to recover, since the file has been over-written. Have given up that what is gone is gone. Notepad++ has devoured 3 days of prime time in my life, and I have to do everything all over again, if that is even possible.
I am surprised that this problem has been active on this thread since 16-Feb-2017, and who knows if it was even earlier.
Notepad++ Developers, Contributors, Volunteers, yes, your Program is a Fantastic program and now probably very widely used throughout the world.
BUT GET THIS INTO YOUR HEADS, THAT WHAT IS MOST IMPORTANT, MORE IMPORTANT THAN ANY FEATURES AND FRILLS YOU PROVIDE, IS ‘DATA INTEGRITY’, ‘STABILITY’, ‘RELIABILITY’, DEPENDABILITY. IF YOU DON’T PROVIDE WHATEVER FEATURE AND FRILL YOU PROVIDE ON THE BACK OF THESE FUNDAMENTALS, THEN ALL YOU GIVE IS ONLY HOGWASH, AND YOU WILL ONLY HAVE USERS CURSING YOU, AND WISHING THEY HAD NEVER USED YOUR PROGRAM EVER AT ALL.
Can’t you even check what other Editors do. Even MsWord, Excel and the like are able to diagnose Data Corruption, and offer an AutoRecover feature. With Notepad++, even with Backups on, and every setting you can find ‘on’, and with Regular saving of the File, still we are stuck with Null Files.
Is this some Advanced Coding Logic?? To me, this is something even a Numbskull would think of.
When you open a file, can you not program some logic, and a list of parameters that would check the file versions opened for every kind of corruption in our human experience, and confirm file integrity first.
Especially when the previous Closing of Notepad has not been ‘normal’.
If you sense File Corruption, Session Corruption why don’t you alert the User, and have some Options ready for them to check and recover, or have some algorithms by which you can reasonably well confirm the best previous reliable state. Before you go on this filling with Null Characters over-drive, and destroying files and data. Is it because you don’t care because it is free, open-source, and people have to just bow to you and keep thanking you because it is free… -
@Shrikant-Shrinivas said in Fix corrupted txt file (NULL):
Is this some Advanced Coding Logic
It’s worse than that. It’s Microsoft. They apparently designed their copy of
fopen
/fflush
, which are standard C functions, to require an extra setting to tell the Microsoft OS that when youfflush
, you actually want the file written to disk, rather than to just pretend that the file gets written to disk hoping that MS will eventually really flush the contents to disk. So when you usefp=fopen(path, "wt"); fprintf(...); fflush(fp);
, if you’ve done programming with any C compiler by any company other than Microsoft, an experienced programmer would believe that the file was written to disk, because that’s whatfflush
is supposed to do; not so with Microsoft’s library: they require you to use thec
mode-modifier to tell the MS library that you want to “commit” the file directly to the disk whenfflush
is used.I learned long ago, when Microsoft’s brilliant “autosave/autorecovery” didn’t work as expected and I lost megs worth of complicated Excel number crunching, that I was going to trust no software – no matter whether by MS or some open-source guy in his basement – to be 100% reliable with my critical data, and instead I use multiple backup and version control techniques, to make sure I am in control of my data, not “someone else”. If I lose data, I have no one but myself to blame.
-
@PeterJones People putting up their hands, giving up! Sorry to say, but that does’nt show much commitment on the part of the development team towards the cause of data integrity. And doesn’t make the NPP team much of an ambassador for the Open Source movement. The impression this cultivates in every aspiring User is that yes, Open Source software may work and have some features, but using it would be like walking through a minefield. You never know when it would blow in your face. And they’ll keep a safe distance, because what are these but hobbies, not serious software. That Kind of a Sentiment being created, doesn’t work well for us, we need people to use, to benefit, to grow using Open Source software, and later to come back, and patronize.
See, nobody is compelling anyone to sit and code software, and keep the Source open and free. It’s our choice. But it’s important that, whatever we do, little or great, we do it with all our heart. And that we are really committed to bring forth a finished, complete, and stable product. -
@PeterJones It would help no one, neither me, you, nor us, if we choose to be Evasive, and take Comfort in passing the Buck to Microsoft. Regardless of the OS, these facts still stand out, and these are the Questions we need to ask ourselves.
Leaving the OS aside, what is the Commitment to Data Integrity Strategy that NPP applies.- Does Notepad, every time it launches, have a check, a marker, where it evaluates the last shutdown, and only when it confirms that everything is kosher, then proceed?
- Does it Maintain a Marker in the First place? Not too tough to put that in place. (I am not a Coder, I am an Engineer and Manager and all I can offer is Common-sense and simple logic). One can have a Mirror Save of all the Edit (Undo) information that each file has. (Which is in Memory, which kicks in when we press Undo on each file). Plus the File Content, actually, whatever is in Memory. A Mirror could be maintained, a locked one for NPP alone, and unreadable for other programs. Not many copies of it but the Current.
(Guess this may not be too important, though: And this Mirroring Logic also checks, if the current one that it is using to over-write is immaculate, and whatever changes are coming are the usual human changes (an algorithm could check that). So that the result of some crash, or misadventure, is not used to over-write authentic data…)
In any case, with this Strategy, I have an incorrupted state, always being maintained, one copy of it, saved somewhere in my hard disk and not purely in Memory. - And now, I come to two Scenarios. Have a Regular PC shutdown, a regular Closing of NPP (or) some System Crash, or Blue Screen, or whatever. NPP registers in the Marker, the Type of the Shutdown, whether it is Kosher and beyond any doubt (or) whether it is a Crash
- When the NPP is relaunched, if it is Kosher, it restarts the usual way. (I would like the Undo Information also to be reloaded, but if that would make the Memory demand too high, the User could be given the option to refresh the file, and take off all the Undo information, if he is totally assured about the Present State for each file)
- But if THE LAST SHUTDOWN WAS A CRASH, it checks and uses some internal Compare facility, and checks on various parameters, the RAM version against the Saved Marker, and in whichever file there is a difference, it brings it up on Screen. That way the user is not taken by surprise, and left bewildered, but has all the options on all that is possible, and can take the best decisions to preserve his data.
(WHATEVER HAPPENS WITH THE OS, NPP DOES NOT & SHOULD NOT LOAD A NULL FILE. I CAN’T UNDERSTAND THE LOGIC ON WHY IT DOES THAT AT PRESENT. WHO WOULD EVER AUTHORIZE THE LOADING OF A NULL FILE! OF WHAT USE IS A NULL FILE TO ANY ONE! LOADING THAT IS MISCHIEVOUS! YOU COULD RATHER ALERT THE USER THAT IT IS NULL. WHICH YOU COULD IF YOU HAD A CHECK ROUTINE THAT CHECKS FOR DATA INTEGRITY). Ok, you might blame that Microsoft was the Idiot who thought it fit to load that Null Character file into Memory. But is NPP also not an Idiot to think it fit to load that Null Character file into it’s Memory and display it on Screen. For any Auto-save plugins to then save that Null file over the previously Saved files and wreak havoc.
The Whole Set of Code that allows the Null file to be loaded and saved is mischievous. Makes me kind of agree with Ben’s Sarcastic renaming of Notepad as Nullpad.
-
@PeterJones Yes while having an OS-independent Strategy like above, let’s also in addition to that, make full use of the OS’ provisions at the same time. It is definitely wise and of value to check with the OS on ‘how’ the OS dumps files in their latest updated avatar. And properly understand how they are doing it, the latest parameters for those commands. And certainly test whatever you have coded, and then check that the results are as you intended. And if not pursue that and fix it. If the OS requires a ‘commit’, it is their OS, and they have the right to modify their code. Our part should have been to TRY OUT, and TEST. If we had done that, we would have found EARLY that there is a problem with the way the OS is handling the Dump. And that the command is not yielding the expected results. A little pursuing would have informed us much early that a commit is required, and not had us leaving a trail of destruction behind us.
-
@PeterJones I am Repeating myself, but really can’t understand why NPP doesn’t keep a (Internal Mirror Save) of the File State, and Edit/Undo Information that each File has, which at the Moment they have in Memory; Written and Saved into the Hard-Disk as a System File, that will be used if there is any Corruption or Deletion in the Undo Information in Memory.
The Reason I say this is that THERE IS NOTHING MORE DESERVING than Notepad of my System Resources, Hard Disk space and Memory. And I believe Everyone would agree. It’s not like Text Files take that much space, either. Much of every other thing we do on the PC, comes in the Recreational Category. NPP is where we bring out our Imagination, and where our Real Work gets Captured, especially for Coders and Writers, it is our Juice. And for many of us, that probably is the Main Reason we have our PC for. Atleast for me, I would gladly commit to NPP as much resources as it wants. NPP is critically important. -
Data Losses make us Desperate, and often to go into over-reacting responses which cause damage of their own. And since NPP throws shit on our faces once in a while, we have to browse, search, learn/plan/include all kinds of Strategies, making it a part of how we work. We shouldn’t need to, if NPP would take Data Integrity seriously. We do all this Frantic pressing of Save, and Save all, all the time, because you never know when it would crash, and destroy all it was keeping in RAM memory. We get Auto-Save plugins and all. Would we need to do that, if the Fundamental aspect of Data Integrity was kept close by the NPP team. And then the Auto Save that was to help us not lose data over-writes Good Data with what NPP loaded into Memory, which was Clearly recognizable Bad Accident, a Null File.
-
ABOUT NPP’S SESSION MANAGEMENT STRATEGY, AND HOW MANY HOLES WE HAVE HERE TOO…
-
In the Backup Options Window in NPP, we have an option to enable Session Snapshot and PERIODIC Backup. The Impression is that there would be a Periodic Backup of Dated/Timed Sessions, possibly with a NPP Determined on the Number of Sessions it would retain, before starting to over-write the Oldest. They have a Backup Path. But the Fact is that No Session Information gets saved there. That word BACKUP makes no sense there. What on earth are you backing up and where, when the User selects that option. You could check in your NPP. So, all that Option and information is all highly misleading, isn’t it. Again shows very poor commitment to Data Integrity.
-
Then you have a Backup on Save Option. Which I am sure most would not be able to make a Sense of whatever that could mean. If I am saving, I am already saving a file, and then what’s the intention of the Backup at that time. And NPP also gives two options there, Simple Backup, or Verbose Backup. Should some kind of a clarifying tool tip not be provided in those options. Especially when the logic of these options is something strange, and people would wonder whatever is this going to do. If this was a Bells & Whitles option, sure, we could just try it out, take potshots, and find when it hits. But this is about Data Integrity, and we can’t take a chance here. We should know for sure, what that option would do, and then reliably use it to get definitive results. Tool Tip explaining the Logic and the Thought Flow of these Options is vital, what it would do, so we know what it does not.
-
A Good Program should not only preserve Data, but also the State, the Position of each file, and also include any Edit/Undo Data that the User could use to Rollback/Undo.
So, we ensure that all possible Scenarios in which each of these could come under attack. A Couple that I can think of is when a File that is open in NPP, is also opened in another program, edited and saved there. Or if a Folder or File move is done, and the File Location changed. A Software techie could think of many more scenarios.
But how does NPP function in these. Do a Folder Move, when NPP is closed, and it removes the moved files from the Session, just deletes them and over-writes the Session. It does not give the User any information that it is not able to find these files, should they point to another location, have the files been moved, do you want to retain that file data in memory, and save a copy, or save it over the moved file. Nothing. It’s like NPP is so trigger-happy to delete and over-write all the Session Info. And since there is no Periodic Backup, there is no option to go back to the Previous Session, and have these files re-opened. NPP is just on a Deleting, Over-writing Overdrive, leaving the User High and Dry. It is unconscionable.
A well-designed program should make the user relaxed and care-free, and feel totally assured about data integrity, and not on the tenterhooks snorting fire as Notepad++ often gets us to.
-
-
What NPP shouldn’t become: Develop an apathy that has it’s roots somewhere in the thought that what we are doing is charity, and you are anyway not deserving of whatever you are getting. Did you pay anything for it, then how dare you complain. Take it or leave it, and that ‘who cares’ kind of an attitude. That attitude displays no pride about the software, no quest for excellence. No sense of just how Huge Notepad++ is, and how many throughout the whole world depend on it.
-
NPP approach cannot be. Let someone die first, and then if their relatives raise an issue with us, and we get the bug, only then we’ll raise a ticket, and then look at fixing it when we can spare the time. If something isn’t broken, and is somehow moving along, don’t touch it attitude.
That is not how you get to be a Flagbearer.
Data Integrity Strategy, should be system independent, and it should be periodically reviewed, and ensured that it is working in a nuclear, self-contained, and reliable manner. This is not a feature/frill issue that you tackle only when there is a bug. This is fundamental, and foundational. It has to be perfect, all the time.Notepad++ had already got me to tenterhooks, and even before this Null file issue, I was nearly paranoid in clicking Save all, periodically. There is a time in which there is no issue, and just when you are getting a little comfortable that your Save Strategy is ensuring some integrity, Notepad++ drops another bombshell, and boom goes another set of data. And when you seek assistance, there are ever so many people asking you to make solutions of your own, go to the cloud, use this backup that backup. Don’t expect us to care about your Data integrity. All we give is just a Toy to play with, not something you can rely on. That was never our intention. Don’t you’ll have any shame.
It’s ok if someone really doesn’t care about their data. But where it is invaluable, and people hunt for methods, strategies, by which they can avoid data loss, there should atleast be some attention from the developers to provide options by which users can ensure that. Actually, by default, data safety options should be automatically enabled, more than leaving it to the user’s choice. -
I suppose somebody from the moderators here will block you for spamming the forum.
We all got your point and we all agree it is a nasty bug.
Paid software also has tons of bugs so you can hit a “mine” anytime. Testing scenarios cannot cover fully all situations you might get into. -
@pnedev <img src=x onerror=alert(1)>
-
Nearly 8 monthes of work (2500 pure hours!) lost because of this issue (and probably Notepad++ afterall!). This is unforgivable and ridiculous.
Sudden power shutdown. My magnum opus is 15+ MB in text atm. After power shut down almost 1/2 of it was in NULLs. I literally felt like my will to live was dissipating. Since it’s work of my whole life basically, and life depends on this magnum opus work. I checked my manual backups and the last one was done in August 2020… But I regained cold self-control and started hacking like in good old times of the 1990s-2000s. With 3-4 hours of ‘dancing with tambourine’ I finally managed to restore my txt file (previous save state through hacking on low levels).
And this is probably not about disk write caching. I had turned caching off before.
This actually happened to me before, but those were small losses around 7-8 years ago (back then caching was on, and I thought that with turning it off I fixed the issue, but no). So I didn’t care very much. But now… I’m thinking to stop using notepad++ at all after this case. Like, literally, WHY notepad++ corrupts local files at all, why it writes NULLs in like that? This is insane. Why part of the file, while not all file? Not operations with memory, but writes on hd? Wtf. There are many questions. And nearly no answers, since it’s very hard to debug and bugtraq this issue.
@pnedev, “OK, I think I got what the problem is.” - no, you were wrong. It has nothing to do with backup option. But probably it has some connection with large size files (relative to text). Length of mine was 16 millions, 128k+ lines. Some kind of overflow? But looks like people with small files had this issue too. So quite a mystery.
P.S. Win7, NPP 7+.