Problem with Encoding

[Shantanu Vachhani]

Hi all,

I have a data file but I have encountered a encoding number. Instead of Numbers , the data is coming in a weird format. How do I fix this?
I have tried changing the encoding but I am not getting the numbers
5175
(
(9CŸb? î>íy€?9å_“sÑ>aí )
(Hcù7DSa?ÕVŒá?Kñô×{6?7¡ )
(© þ
T[c?ŸHæ¸ @€?k¸i
ÿÅ>[
)
(ö
ˆùBb?ÞÂg‹µ€?ù~<<}M?Lª )
(é¾3£ÿ+a?“WBñÀ/?5G³Ý´?ƒ )
(
íÖ’òã`?—KÜ .?ÂÕã­?ÿ>û )
(ßô”€La?ô•KÊJ?ÞO˜Š? )
(É!ó“ta?”Š*hí€?@i&¶Þ?8¯ )
(8ðsÿ’Öb?§¤kf|€?̶²IBeþ>‰ )
(|[/!Jc?«Ž$û:€?¢ÆkZ‚É>øð )
(
‹ÝCwÃb?©Í3l]€?w–_jÒ»?÷ì )
(Ù&iÊ`?u[Ü;S9?‡Æuqàjü>¡ )
(lô$¸€a?·»´ú€?;y»~é²?¦ )
(¤W7ò`?ì.A$:?áfI¡7†à>\ )
(ýKd†
îa?8÷ná€?¸Â«¾?öú>Ò )
(Þs¹âݹb?ÈxÈ¿Ût€?˜T
é‘?Ó© )

[chcg]

N++ has a automatic encoding detection (Settings->Preferences->Misc->Autodect charecter encoding). Maybe this one is breaking your file. What encoding is detected and what should it be? Do you know that?

[Shantanu Vachhani]

Thanks for your reply,

However I don’t know the type of encoding it should be. I just know that the file is a set of x,y and z coordinates .So it should be all numbers.

[PeterJones]

There are lots of ways of storing “x,y,z coordinates … it should be all numbers” in a file. Not all of them involve actual text.

Many programs use a binary format for storing the data, because you can get an integer from 0 to 4 Billion (or +/-2 Billion) in four bytes instead of the up-to-11 characters required (-2147483648) to store it as text. And floating point numbers (numbers with a fraction) can get an even wider range of values in 4 bytes.

And most programs that store data in a binary format will also include other information in the file. Often, the files are proprietary, but sometimes they publish the exact format of their file.

If it’s a binary file, Notepad++ (a text editor) isn’t going to natively understand the format. But there may be a save-as option from the generator, which would allow you to read it in Notepad++; or there may be an already-existing converter that converts from file-type XXXX into a plain-text format. The questions below may help us help you find such an alternative.

• Do you know what generated this data file? (Are you allowed to tell? If it’s a custom application, you may not be allowed to tell us; if so, you might ask your support group for that application whether there’s a way to save to a text format)
• Does it have format choices? (If so, you might see if there’s a .txt or .csv option, rather than its native binary format.)
• What extension does it save it as?

The groups of parentheses seem to always have 27-30 characters. My guess is that in the original file, there were always 30, but some were NUL characters that then didn’t copy right from Notepad++ into the forum (often copy/paste does not maintain NUL characters). Given x,y,z, I am guessing that it’s 3 8-byte sequences of double-precision floating-point numbers, plus 4-6 other bytes which carry some other information I cannot guess. (Or 8 more bytes which include another floating-point number, and all your data has at least 2 NULs)

[Shantanu Vachhani]

@PeterJones said:

ta file? (Are you allowed to tell? If it’s a custom application, you may not be allowed to tell us; if

Sir thanks for your reply,

Basically the content of the file is that each line of the data file is the cartesian coordinates for a particle travelling in the fluid domain.

1. The file is generated by a open source software known as OpenFOAM which is used for Computational Fluid Dynamics Simulations
2. Mostly the simulations writes into a data file in either ASCII or binary format.

Sir the odd part is that the file when inputted into a post processing software it gives proper results. So the software is reading the data file correctly,so I think it is a representation problem.
However i need those coordinates for certain post-processing so I need those numbers.
Is encoding the problem?Is there someway I could rectify this problem?

Thanks and Regards,
Shantanu Vachhani

[PeterJones]

Based on the data you posted, I’d say your file was the binary format rather than the ASCII format – you might want to see if you can configure it to output ASCII instead. If it’s a binary file, then likely no general-purpose text editor (be it Notepad++, MSWin notepad, vi, emacs, …) will be able to natively recognize it. It’s not an “encoding” issue, in the same way that UTF-8 vs ISO 8859 vs Win-1252 vs …

I think this question has nothing to do with Notepad++ (in that I think any other text editor – be it windows notepad.exe, vi, emacs, … – would give similar results when trying to open a binary file).

I just spent 5 minutes via search engines and https://www.openfoam.com, and couldn’t quickly find a description of the output format(s) available. Sorry, that’s all the time I have to devote to it.

My searches also revealed a forum at https://www.cfd-online.com/Forums/ . You may be able to post your question there: they might be able to help you either natively output to an ASCII text file, or to point you to a converter or post-processing manipulator to give you access to the coordinates.