Big files cannot opened in v8.6.4 (x64)

Klaus Lehmann

notepad++-scintilla-code_1.jpg

Hi. with this version 8.6.4 I tried to open a 3GB file and than to edit. its mainly with ascii content.
I want to open it, please see error in jpg.
Next small windows comes with "cannot open file “f:\tmp\zdb.mrk”.

I’m not using every day to open bigger (than 2 GB) files, but sometimes 1-3 files per month.
What is happened? Is it an issue in V8.6.4?

Who knows more?
Thanks a lot.
Yours Klaus

PeterJones

@Klaus-Lehmann said in Big files cannot opened in v8.6.4 (x84):

with this version 8.6.4 I tried to open a 3GB

If “(x84)” was a typo for “(x86)”, which is the 32-bit version of Notepad++, then your problem is that a 32-bit character counter can only go to 2GB.

If “(x84)” was a typo for “(x64)”, which is how 64-bit is indicated, then I don’t know what your issue is.

Given that your typo includes pieces of both nomenclatures, it’s really just a contextual guess as to which you actually meant. Sharing the ?-menu’s Debug Info would remove the possibility of typos.

To open a file bigger than 2GB, you must have the 64-bit Notepad++ and enough free/available memory for Notepad++ to use.

Klaus Lehmann

@PeterJones
thanks a lot. ofcourse this was a (stupid) typo from me.
I do only use: notepad++ x64.

here’s some technical info:
Notepad++ v8.6.4 (64-bit)
Build time : Feb 20 2024 - 00:12:59
Path : c:\Program Files\Notepad++ allegro\notepad++.exe
Command Line : zdb.mrk
Admin mode : OFF
Local Conf mode : OFF
Cloud Config : OFF
OS Name : Windows 11 Pro for Workstations (64-bit)
OS Version : 23H2
OS Build : 22631.3296
Current ANSI codepage : 1252
Plugins :
mimeTools (3.1)
NppConverter (4.6)
NppExport (0.4)

my ram is only 64GB ;)
just now its used by 10GB, free ram is 53GB

thanks alot. Yours Klaus

Mark Olson

@Klaus-Lehmann
This doesn’t fix the underlying problem with Notepad++, but you might consider using my HugeFiles plugin. It reads files one chunk at a time (with customizable chunk size and other settings) so you don’t need to risk running out of memory or staring at the screen forever when you read enormous files like that one.

xomx

@Klaus-Lehmann

Translated errmsg:

Exception code: 1 (Scintilla)

An error occurred while loading the file!

This is not a problem with Notepad++, but with the underlying Scintilla library, where something bad happened during the file loading stage (file opening was ok). That code 1 means SC_STATUS_FAILURE, that exception was catched & reported here.

This is also not a not-enough-memory problem, in such case the code would be 2 - SC_STATUS_BADALLOC.

Does your 3GB file contain any EOL (CR/LF)?
What is the “.mrk” file for?

PeterJones

@xomx said in Big files cannot opened in v8.6.4 (x84):

What is the “.mrk” file for?

… and even more, @Klaus-Lehmann , do you have a built-in syntax highlighter or UDL syntax highlighter active for .mrk files? if so, you might want to use Settings > Preferences > Performance to disable syntax highlighting on large files, because >2GB may be too big for scintilla/lexilla to highlight.

mkupper

@Klaus-Lehmann I tested and verified that npp v8.6.4 (x64) works with 3gb files (3,221,225,472 byte) and 16gb (17,179,869,184 byte) files.

My test was using files that contained lines with 128 characters including the CRLF per line:

123456789.123456789.123456789.123456789.123456789.123456789.123456789.123456789.123456789.123456789.123456789.123456789.123456
123456789.123456789.123456789.123456789.123456789.123456789.123456789.123456789.123456789.123456789.123456789.123456789.123456
123456789.123456789.123456789.123456789.123456789.123456789.123456789.123456789.123456789.123456789.123456789.123456789.123456
123456789.123456789.123456789.123456789.123456789.123456789.123456789.123456789.123456789.123456789.123456789.123456789.123456
...

I used the .txt file extension.

Things to try:

Rename your zdb.mrk to zdb.txt and see if that opens.
Is your f:\tmp on a file or NAS server? That may have issues with large files. Try copying zdb.mrk to drive C: or other local disk drive that has space.
A bit harder to track down is if there is something about the .mrk file content that bothers Notepad++ or Scintilla.

Coises

As a test, would you download Single file 64-bit executable called Sc1 from SciTE and try to open your file?

SciTE is a demonstration application for the Scintilla edit control. Notepad++ depends on Scintilla, but adds a lot of features around it. If SciTE fails on your file in the same way, we can conclude that either Scintilla or the file itself is the problem. If SciTE can load it, then the problem must have something to do with Notepad++.

Klaus Lehmann

@PeterJones
thanks for the good question.
of course I don’t use “a built-in syntax highlighter or UDL syntax highlighter” active.
one of the worst was the xml-syntax-highliter, if I would use them in BIG xml-files ;-)
Yours klaus

Klaus Lehmann

@Coises
interesting. I have had not any knowledge about this tool “Sc1”
please look at the picture:

is this true, that there exists a 2GB border?
it means really no file has to be bigger than 2 GB?

bad moment: I can’t remember, use bigger than 2GB-text-files.
I f this true, than I know only one texteditor, CRISP. Her has no problem, to open bigger than 2 GB.
Before 5-10 years, I was a mainyl daily user with Crisp. but before 5-10 years I changed completely onto Notepad++ (and some years later onto 64bit-version).

Thanks, klaus

Klaus Lehmann

thanks You very nuch for Your test. this means for me, the problem is on my side. not on side from notepad++
strange.

Your advises like:
copy *.mrk onto *,txt. result: same thing.
*.mrk is a normal ascii-based file. every 100 charcters or 1000 chars there is a full \x0d\x0a (CRLF).
normally I do my work on local drives. and just know also on a local disc (ssd).

thanks Klaus

Klaus Lehmann

@xomx
“What is the “.mrk” file for?”

I forgot to answer: *.mrk is a normal extension for us, who are librarians.

if there is someone interested to bekome mor infos, please look at:
[http://loc.gov](Link Adresse) -> better in german here: https://www.alma-dach.org/alma-marc/
and than in english: [https://www.loc.gov/marc/](Link Adresse) (but its strong stuff, not for everyone ;-)

mark exist in 3 forms:
mrk, ist more human readable, mainly ascii.
mrc, is absolute binary. it has a veyr logic structure (but I have spend some years to understand it) ;-)
and than my “loveliest”: marc-xml (I hate xml!). 80-90% useless information. the rest is usefull (but, attention: its my personal opinion. bloody programmers loves xml ;-)

and If “we” use them, there can exist a mrk-file as a big data file:
my mrk ist 3 GB. as a mrc it is a little bit more: 3,3GB
as an xml? it is 11GB

and what is the “zdb”? ;-)
“Zeitschriften Datenbank”. worlds biggest catalogue with existing and old periodicals in libraries.
content: from 2024-03-15 12:25 655M 2.113.828 titels (ZDB-Titel Gesamtbestand, Format MARC 21 (Stand: 12.03.2024 01:30)
but this is stuff for librarians ;-)

Yours Klaus

Coises

@Klaus-Lehmann said in Big files cannot opened in v8.6.4 (x84):

is this true, that there exists a 2GB border?
it means really no file has to be bigger than 2 GB?

The message is not worded very well. It only means that the program is configured to warn you when you try to open a file larger than 2 GB. If you answer “Ja” it will proceed to open the file.

I checked, before posting that suggestion, that it can in fact open a very large file (mine was over 4 GB).

Coises

@Klaus-Lehmann said in Big files cannot opened in v8.6.4 (x84):

thanks You very nuch for Your test. this means for me, the problem is on my side. not on side from notepad++

I’m not sure to whom you are responding here, so my apologies if this doesn’t make any sense.

Have you checked your system’s error logs to see if there is a hardware error reading the file? I’m not sure if this might have changed in Windows 11, but in Windows 10 you right-click the Windows Start icon, then select Event Viewer, then expand Windows Logs and look at the Application and (most importantly) System logs. Once you select one of those, you can use Filter current log… in the right-hand pane to choose to see only Critical and Error messages; but most useful would be to recreate the failure at a known time first, so you will know at which time in the logs to look.

xomx

https://librarycarpentry.org/lc-marcedit/instructor/03-working-with-MARC-files.html

A sample data for testing:
https://github.com/LibraryCarpentry/lc-marcedit/blob/main/episodes/data/marc_sample_data.zip

I made a 3.5GB mrk-file from the included marc_sample_data.mrk (just by multiplying its content) - N++ loaded that ok.

I also tried to open that small included mrc-file there and it does not contain any EOL in it. So if someone tries to open such a mrc-file larger than 2GB, the exact exception as reported will occur, as the Scintilla currently cannot handle lines longer than the INT_MAX chars.

@Klaus-Lehmann
Could you share your problematic mrk-file for testing?

Coises

@xomx said in Big files cannot opened in v8.6.4 (x84):

Scintilla currently cannot handle lines longer than the INT_MAX chars.

I can’t find documentation for this, but it appears to be correct. I constructed some large files made of nothing but the character “a” repeated enough times to make 1GiB and 2GiB files, then tried to open them with SciTE and with Notepad++. The 1GiB file was very slow to open, and unusable even for viewing, in both programs; the 2GiB file just crashed with no message at all.

It appears that no Scintilla-based editor is a good choice for large files that are not line-oriented, and hence may have very long stretches with no x0A or x0C characters in them. I don’t know what would be a good editor or viewer for such files.

Klaus Lehmann

@xomx
sorry for small delay
here’s Your link: [http://allegronet.de/zdb.zip](Link Adresse) .
527MB as zip. content: a pure mrk-file. unzip it. content than: 3 GB.
content is public data. allegronet is my server. please don’t go first onto http://allegronet.de. it will not work. try please only with the exact link: [http://allegronet.de/zdb.zip](Link Adresse) . thanks.

xomx

@Klaus-Lehmann
I can open this file ok, used the same version as yours for the test:

Notepad++ v8.6.4   (64-bit)
Build time : Feb 20 2024 - 00:12:59
Path : C:\Program Files\Notepad++\notepad++.exe
Command Line : C:\TEMP\NPP\zdb.mrk
Admin mode : OFF
Local Conf mode : OFF
Cloud Config : OFF
OS Name : Windows 10 Enterprise LTSC 2021 (64-bit)
OS Version : 21H2
OS Build : 19044.4170
Current ANSI codepage : 1250
Plugins : 
    mimeTools (3.1)
    NppConverter (4.6)
    NppExport (0.4)

Will try your build of Win11 later.

xomx

@Klaus-Lehmann
I successfully opened it also on the Win11 22H2 (I do not have your 23H2 available).

So it is something specific on your PC. We can eliminate the possible problem with a N++ setting by downloading the portable version equivalent, which does not need to be installed, but just unzipped into a folder and then run the notepad++.exe from there. Then, if you are able to open that zdb.mrk in your portable N++, the problem lies with your N++ installation. Otherwise the problem is probably outside the N++ control (a Windows setting, installed app or shell extension, AV, storage place problem, etc…).

Klaus Lehmann

@xomx
sorry for me, that my answer arrived You very late. but there was too much jobs with family ;-)
ok. I think, I will see my problem and can explain it.
1st: thanks to @Coises , who gave me hint with Sc1.
Yes! with Sc1 I can open without problems m ymrk-file with 3GB content.
This talks to me, the propblem is based on my machine. but where?
some days have gone
deInstall and install N++ at new. and very certain: on another machine (why? it depends from appdata\roaming\notepadd++ . there resides config.xml

I’m shure, here was my problem:

old (not working since 2 weeks WITH 3GB file) config.xml:
<GUIConfig name="largeFileRestriction" fileSizeMB="4096" isEnabled="no" allowAutoCompletion="no" allowBraceMatch="no" allowSmartHilite="no" allowClickableLink="no" deactivateWordWrap="yes" suppress2GBWarning="yes" />

new (working) config.xml
<GUIConfig name=“largeFileRestriction” fileSizeMB=“200” isEnabled=“yes” allowAutoCompletion=“no” allowBraceMatch=“no” allowSmartHilite=“no” allowClickableLink=“no” deactivateWordWrap=“yes” suppress2GBWarning=“yes” />

I think last line brings me good results. all works.
again: last line was my solution. Yes I know, there are some "misunderstoodings":
isEnabled="yes"   and  fileSizeMB="200" . very unlogical. but it works!

Thanks for all, who has helped me.
Yours Klaus