Problem with Encoding
-
Hi all,
I have a data file but I have encountered a encoding number. Instead of Numbers , the data is coming in a weird format. How do I fix this?
I have tried changing the encoding but I am not getting the numbers
5175
(
(9CŸb? î>íy€?9å_"sÑ>aí )
(Hcù7DSa?ÕVŒá ?Kñô×{6?7¡ )
(© þ
T[c?ŸHæ¸ @€?k¸i
ÿÅ>[
)
(ö
ˆùBb?ÞÂg‹µ€?ù~<<}M?Lª )
(é¾3£ÿ+a?“WBñÀ/?5G³Ý´?ƒ )
(
íÖ’òã`?—KÜ .?ÂÕã?ÿ>û )
(ßô"€La?ô•KÊJ?ÞO˜Š? )
(É!ó“ta?”Š*hí€?@i&¶Þ?8¯ )
(8ðsÿ’Öb?§¤kf|€?̶²IBeþ>‰ )
(|[/!Jc?«Ž$û:€?¢ ÆkZ‚É>øð )
(
‹ÝCwÃb?©Í3l]€?w–_jÒ»?÷ì )
( Ù&iÊ`?u[Ü;S9?‡Æuqàjü>¡ )
(lô$¸€a?·»´ú€?;y»~é²?¦ )
( ¤W7ò`?ì.A$:?áfI¡7†à>\ )
(ýKd†îa?8÷ná€?¸Â«¾?öú>Ò )
(Þs¹âݹb?ÈxÈ¿Ût€?˜T
é‘?Ó© ) -
N++ has a automatic encoding detection (Settings->Preferences->Misc->Autodect charecter encoding). Maybe this one is breaking your file. What encoding is detected and what should it be? Do you know that?
-
Thanks for your reply,
However I don’t know the type of encoding it should be. I just know that the file is a set of x,y and z coordinates .So it should be all numbers.
-
There are lots of ways of storing “x,y,z coordinates … it should be all numbers” in a file. Not all of them involve actual text.
Many programs use a binary format for storing the data, because you can get an integer from 0 to 4 Billion (or +/-2 Billion) in four bytes instead of the up-to-11 characters required (-2147483648) to store it as text. And floating point numbers (numbers with a fraction) can get an even wider range of values in 4 bytes.
And most programs that store data in a binary format will also include other information in the file. Often, the files are proprietary, but sometimes they publish the exact format of their file.
If it’s a binary file, Notepad++ (a text editor) isn’t going to natively understand the format. But there may be a save-as option from the generator, which would allow you to read it in Notepad++; or there may be an already-existing converter that converts from file-type XXXX into a plain-text format. The questions below may help us help you find such an alternative.
- Do you know what generated this data file? (Are you allowed to tell? If it’s a custom application, you may not be allowed to tell us; if so, you might ask your support group for that application whether there’s a way to save to a text format)
- Does it have format choices? (If so, you might see if there’s a .txt or .csv option, rather than its native binary format.)
- What extension does it save it as?
The groups of parentheses seem to always have 27-30 characters. My guess is that in the original file, there were always 30, but some were NUL characters that then didn’t copy right from Notepad++ into the forum (often copy/paste does not maintain NUL characters). Given x,y,z, I am guessing that it’s 3 8-byte sequences of double-precision floating-point numbers, plus 4-6 other bytes which carry some other information I cannot guess. (Or 8 more bytes which include another floating-point number, and all your data has at least 2 NULs)
-
@PeterJones said:
ta file? (Are you allowed to tell? If it’s a custom application, you may not be allowed to tell us; if
Sir thanks for your reply,
Basically the content of the file is that each line of the data file is the cartesian coordinates for a particle travelling in the fluid domain.
- The file is generated by a open source software known as OpenFOAM which is used for Computational Fluid Dynamics Simulations
- Mostly the simulations writes into a data file in either ASCII or binary format.
Sir the odd part is that the file when inputted into a post processing software it gives proper results. So the software is reading the data file correctly,so I think it is a representation problem.
However i need those coordinates for certain post-processing so I need those numbers.
Is encoding the problem?Is there someway I could rectify this problem?Thanks and Regards,
Shantanu Vachhani -
Based on the data you posted, I’d say your file was the binary format rather than the ASCII format – you might want to see if you can configure it to output ASCII instead. If it’s a binary file, then likely no general-purpose text editor (be it Notepad++, MSWin notepad, vi, emacs, …) will be able to natively recognize it. It’s not an “encoding” issue, in the same way that UTF-8 vs ISO 8859 vs Win-1252 vs …
I think this question has nothing to do with Notepad++ (in that I think any other text editor – be it windows notepad.exe, vi, emacs, … – would give similar results when trying to open a binary file).
I just spent 5 minutes via search engines and https://www.openfoam.com , and couldn’t quickly find a description of the output format(s) available. Sorry, that’s all the time I have to devote to it.
My searches also revealed a forum at https://www.cfd-online.com/Forums/ . You may be able to post your question there: they might be able to help you either natively output to an ASCII text file, or to point you to a converter or post-processing manipulator to give you access to the coordinates.