Search code examples
cbinarycygwinsolarissunos

Could there be any difference of binary unformatted data on different platforms?


I am doing a scientific project with two super old programs code A and code B. The interaction between the two codes is through binary unformatted data. The data are arranged as that each four bytes combined to form a float number. Both the two programs are written many years ago and the developers are now in different institutions and are difficult to contact. Since I need to see the result urgently for a defense and my programming is not that good, I use the same platforms as the developer or modder to compile and run the programs in order to avoid debugging troubles.

Code A is mainly written in C and the rest part of it is written in bash and FORTRAN. It is unclear on which platform the developer wrote this program but I am using a modified version passed from a student graduated last year. The student modified the program on a 64-bit Cygwin in Windows and I found that this is the only platform I could compile and run it without errors. For example, if I run it on OSX, Ubuntu or 32-bit Cygwin, I will run into errors which are out of the range I can handle due to my limited time and skill in programming.

Code B is written in C on SunOS and relies on the Sun Performance Library. Similarly, I can only successfully compile and run it on an antique Solaris machine in our computer lab. On other platforms I have tons of missing headers and it is very troublesome to collect all of them. According to the documentation, code B uses the output binary data of code A and the format is arranged in exactly the same way. If I copy the output data from code A to code B and run it, code B is complaining the validity of data /* screen output: negative values detected! */. Nevertheless, code B is running with no problem if I use the sample data provided in its directory - the data which the developer used to write and test his program.

Firstly, if I try to take only the data reading function of code B, create a simple C project and call it in my main() function, I found that in my IDE I cannot reproduce the BUG. The algorithm of data reading in the original code B looks like:

int i, nxyz, fildes;
char  msg[1024];
nxyz = nx * ny * nz;
if ((fildes=open(vmfile, O_RDONLY, 0664)) <= 1)
{
    perror(vmfile);
    return 1;
}
if (read(fildes, vmodel, nxyz*sizeof(float)) < 0)
{
    sprintf(msg, "Reading %s\0", vmfile);
    perror(msg);
    return 2;
}
/* checking validity codes go here */
close(fildes);

The function is reading my binary unformatted data correctly but not reading the developer's data correctly. This is contrary to what happened when trying to compile the project on Solaris.

Secondly, I try to bypass the data reading function of code B, write a new function to read in the data through an ASCII file and make sure the array storing the data is returned to the main() function properly. After making this modification, the data are loaded correctly but unexpectedly, right after loading the data, the program stopped and raised a Segmentation Fault which is never seen when running with the developer's sample data. I am not sure if I should continue working on this new error or give up the idea.

So, could there be any differences between binary unformatted data on 32-bit and 64-bit platforms, Linux and Windows, Solaris and other Linux machines? If there are, is there any way to convert the data without modifying the source codes in this urgent case /* since it must be a much faster way than doing any debugging */?


Solution

  • If the Solaris machine is a SPARC, then unless the programs were careful to format the data in a specified order, numbers larger than a byte will be stored in a different order than Intel CPU's use as you'd find on your Windows machine. This is called endianness.

    Floating point numbers could also be written in a variety of formats, but the IEEE floating point standard is probably what's been used here. Of course there's a lot more to know about how floating point numbers are handled if you want to understand the limitations and possible errors that may be seen with such data.