Search code examples
powershellcharacter-encoding

GnuPlot on Windows "Bad data on line 1" (edit: PowerShell encoding)


Context:

Data produced by complied C code can't be read by GnuPlot (on a Windows 10 pc).

EDIT: The actual problem was character encoding enforced by the environment of the Windows PowerShell.

Details:

Here's my (vastly simplified for testing) data file "data1"

0.0000e+00 0.0000e+00
5.0000e-01 2.5000e-01

This is produced by

#include <stdio.h>
void main()
{
    double x=0.0 , y=0.0;
    printf("%.4e %.4e\n",x,y);
    x=0.5; y=x*x;
    printf("%.4e %.4e\n",x,y);
}

compiled on Visual Studio Code 1.84.2 (using the MSYS2 version 12.2.0 of gcc) into an executable I call "SPAM". In the Windows power shell, in that directory, I call .\SPAM > data1.

EDIT: The culprit is the Windows PowerShell which, through the redirect character >, encodes the output as UTF-16 LE BOM.

Using GnuPlot 5.4 (patchlevel 8) gets me this trouble:

gnuplot> plot 'data1'
              ^
         Bad data on line 1 of file data1

Attempts:

Typing those data in directly using plot '-' works fine. Viewing the data file with Notepad++ shows no hidden or special characters. Copy-pasting the contents of the data file into a new file solves the problem! But I can not be in the business of duplicating all my (actually huge) datafiles.

Question:

  1. What is hidden in the datafile that GnuPlot does not like?
  2. Can I solve this through changing settings/flags in GnuPlot?
  3. Is there something wrong with my installation / build of VSC & gcc that's producing "weird" files?

EDIT: The actual problem was none of those things -- it was the encoding enforced by the Windows PowerShell.


Solution

  • On my Windows 10, Powershell produces an UTF-16 encoded data1 file with a size of 94 bytes.

    The "normal" command shell cmd.exe produces ASCII text with a size of 46 bytes.

    Therefore I would suggest to use cmd.exe instead of Powershell to run the command .\SPAM > data1. Or to write a batch scripts which runs this command, the batch script can be run from Powershell.

    Update:

    According to this superuser answer you can also run this command from Powershell:

    .\SPAM | Out-File -Encoding ascii data1

    If you want a deeper understanding, you might want to search for something like "printf powershell produces UTF-16".