Problem writing to a FatFs USB filesystem

[dhalbert]

Does NP++ do anything tricky when simply writing out a file?

I’m using NP++ to write a file to a simple USB Mass Storage FAT filesystem. The filesystem is implemented on a microcontroller by a variant of MicroPython (Adafruit CircuitPython).

If I write the file with Microsoft’s NOTEPAD.EXE, it works fine. If I write the identical file with NP++, CircuitPython can’t read it properly, and reports an error right around the 512-byte boundary in the file (which is about 556 bytes long). If I dump the contents of either file from a CMD.EXE command line, they are identical, and have identical length. The properties are also identical when examined from Windows Explorer. But NP++ appears to be doing something different when writing out the second 512-byte block of the file.

So is there something unusual NP++ is doing when doing file writes? Perhaps it’s using some Windows API calls that FatFs is not supporting properly.

Thanks for any suggestions or guesses.

[Claudia Frank]

@dhalbert
actually I don’t think so because from npp’s point of view it doesn’t care if fat or ntfs
filesystem is used. And I guess notepad uses the same api as notepad++ does.
Try to compare both files with an hex editor like hxd (my favorite hex editor).
If the files are the same they should behave the same.

Cheers
Claudia

[dhalbert]

Thanks. The files appear completely the same: I looked at them with GnuWin32 od. However, I’m looking at them in Windows, which has its own idea of the file contents (since it uses its local caches) and hides what is actually on the device.

Since I first posted this query, I’ve looked at the code in NP++ that writes files. It’s straightforward: just ::fwrite() followed by ::fclose().

The devie appears as a USB drive to Windows, and the write-back policy is set to write all changes immediately, and not to cache them.

However, what I do see is that NP++ (or the APIs it calls) doesn’t flush and write back the file completely as I’d expect. It writes the first 512-byte block of the file, and then 60-90 seconds later, it writes the remaining block. This is odd and I’ll need to research further why this might be happening in Windows.

(I disabled the change detection in NP++ just in case that was affecting the write-back, but it made no difference.)

[Claudia Frank]

@dhalbert

60-90 seconds is really strange.
But it reminds me about an issue I had with an usb stick but this would mean notepad.exe should also have this issue.
Not sure if it is worth trying but maybe install wireshark and the usb dissector addon to monitor usb traffic.

Cheers
Claudia

[dhalbert]

Some closure on this: a number of other editors show the same delayed-write behavior: Mu, Atom, Wordpad. The differences appear to be due to the args to CreateFile() to create a file handle, which is invoked directly by NOTEPAD but via fopen() by many other editors. If the file is not assumed to exist in advance, then there will be write-delay issues.

This is not an NP++ problem: it appears to be a Windows problem about delayed writes even when the USB drive is set for “Quick Removal”, which supposedly turns off write caching.

[dhalbert]

Final info:

Per Claudia’s suggestion, I used Wireshark and USBPcap to trace the USB traffic. It showed that the files on the USB drive were being written completely but that updates to the directory entry describing the file were delayed 30-90 seconds. If I “Eject” the drive, the writes are forced immediately (and the drive is not actually ejected). I tested this both with the emulated USB drive and a regular USB stick formatted as FAT. So as I said above, this is a Windows problem about delayed writes. It doesn’t really have to do with NP++.

[Claudia Frank]

@dhalbert

Thank you for letting us know.
But I’m still a little bit confused about notepad behaving differently to npp.
After you mentioned the differences about createfile and fopen I tried to understand
what’s going on and it started being more confusing as fopen seems to do createfile
under the hood. Still don’t understand it completely.
If I find something valuable I will post it in this thread.

Cheers
Claudia

[Bdm Ieq]

This post is deleted!