File Too Big Inconsistency

[Paul Whaley]

File too big message (602,794 KB) so based on FAQ I restarted and it loaded the file. I have about 20 of them to process, tried to load the rest of the batch and got ‘File too big’ messages. I closed all (including the one that initially loaded) and tried to reopen the one that initially loaded, got the ‘file too big’ message.

Any ideas?
thanks
pmw

[PeterJones]

Memory usage varies – both in Notepad++ and in the other applications and background tasks running on your machine. The closing of Notepad++ and reloading will free all the memory except that used by the active file, which gives you the biggest chance. For files in the hundreds of megabytes, it’s probably better to use the sequence of exiting all NPP instances, then looping on “start new NPP instance with no files open; open huge file; process; close file; exit NPP”

You don’t say which version of Notepad++ you use, or whether you’re 32-bit or 64-bit (? > Debug Info would be able to easily tell us), but 64-bit should be able to handle bigger files than 32-bit (unfortunately, because of the way the Scintilla editor component is written, not files as big as 64-bit addresses would imply are possible).

Other than that, you might try closing as many other applications as you can, and doing anything you can to limit memory usage. You might try running without plugins ("c:\program files\Notepad++\notepad++.exe" -noPlugin "superhugefile.txt") to limit Notepad++'s memory usage.

If all your processing involves is running one or more regex inside the file, you might want to try getting a windows copy of sed or windows copy of gawk, or use a full programming language like Perl or Python – all of which may (or may not) be able to make the transformations in your text that you need without requiring the entire file to be held in memory – it depends on the processing you need to do. (Using the PythonScript plugin as a Python interpreter, you could write your script and run it inside Notepad++, even if you aren’t using the PythonScript-specific access to the currently-open-files: you’d basically be using the PythonScript.dll instance of Python instead of installing your own python.exe. Note: it wouldn’t be enough to use PythonScript to load the huge file into Notepad++, because that would still have the full memory requirement, which you are running up against; you’d have to just use standard Python text-processing, trying to do it line-at-a-time or chunk-at-a-time, rather than whole-file-in-memory.)

(edit: in case I wasn’t clear, the sed/gawk/perl/python solution would be outside of Notepad++; you could use NPP to write your script for one of those tools, and even use its run menu and similar functionality to launch the process… but it wouldn’t be doing the processing to the open files inside Notepad++, and wouldn’t be a solution that we’re really equipped to help you with, since this isn’t a general-programming forum.)