Search code examples
linuxbashawkterminalcut

Unwanted zeros in AWK output

Problem explanation

Using bash on Ubuntu, I read some data packets continuously over the serial (it's connected to a wireless base-station) using a java prog and pipe it through cut and awk to get a result. I use unbuffer to see the output in real-time (otherwise the output waits till the buffer is full).

Syntax of the command that I issue on the terminal: 

java prog | unbuffer -p process1 | unbuffer -p process2

Actual command I issue: 

java net.tinyos.tools.Listen -comm serial@/dev/ttyUSB0:telosb |
unbuffer -p cut -c49-50,52-53,55-56,58-59,60,61-62,64-65,67-68,70-71,72,73-74,76-77,79-80,82-83 |
unbuffer -p awk -F " " '{printf("%d %5.4f %5.4f\n",("0x"$1)*1,("0x"$2)*2*0.0005714,("0x"$3)/612500)}'

My output: 

43301 1.9370 5.0124
0 0.0000 0.0000
43303 1.9279 5.0071
0 0.0000 0.0000
43305 1.9428 5.0125
0 0.0000 0.0000
43306 1.9428 5.0072
0 0.0000 0.0000
. 
.
.

Line of zeros aren't real but spurious. If I execute only the java prog (i.e., before cut) or java prog + process1 (i.e., before AWK), they don't appear. They appear in the output only after process2, after using awk.

What I want

I wanna find out why the zeros are there in my output and how.

Appreciate your help. Thanks.

Supplementary data

One of the received data packets looks like this (output after java prog). All are hex (ungrouped at this stage): 

 00 FF FF 00 01 1C 22 95 00 00 00 00 00 00 00 00 00 00 25 83 00 00 06 95 00 2E BF 60 00 2E C0 BB 00 00 00 00

Above, it looks like all numbers are in groups of two. But there are number clusters within -- hence why I cut them in such a way that I regroup them accordingly. Output after cut is as follows:

Output after java prog + process1: 

0000A995 00000697 002EBE00
0000A998 0000069F 002EE4FB
0000A99C 000006A4 002EBF9F
0000B421 000006A4 002ECC6B
 .
 .
 .

NOTE: The two outputs are not their own conversions. They were taken at different instances, just to provide an example only.

Solution

  • So, I answer my question inspired by the answer I got + the great comments and help tidbits.

    Problem

    Using awk. No gawk installed.

    Command: 

     java net.tinyos.tools.Listen -comm serial@/dev/ttyUSB0:telosb |
 unbuffer -p cut -c49-50,52-53,55-56,58-59,60,61-62,64-65,67-68,70-71,72,73-74,76-77,79-80,82-83|
 unbuffer -p awk -F " " '{printf("%d\t%5.4f\t%5.4f\t\n","0x"$1,("0x"$2)*2*0.0005714,("0x"$3)/612500)}'

    Output: 

     2723   1.9233  5.0164  
 0  0.0000  0.0000  
 2726   1.9268  5.0164  
 0  0.0000  0.0000  
 2730   1.9428  5.0073  
 0  0.0000  0.0000  
 2732   1.9348  4.9866  
 0  0.0000  0.0000
 .
 .
 .

    Zeros were undesirable.

    Solution

    Installed gawk. Using it, I changed the command as in the following:

    Command: 

     java net.tinyos.tools.Listen -comm serial@/dev/ttyUSB0:telosb |
 unbuffer -p cut -c49-50,52-53,55-56,58-59,60,61-62,64-65,67-68,70-71,72,73-74,76-77,79-80,82-83|
 gawk -F " " --non-decimal-data '{printf("%d\t%5.4f\t%5.4f\t\n","0x"$1,("0x"$2)*2*0.0005714,("0x"$3)/612500)}'

    Output: 

     2723   1.9233  5.0164  
 2726   1.9268  5.0164  
 2730   1.9428  5.0073  
 2732   1.9348  4.9866  
 .
 .
 .

    Zeros gone. Only changes were gawk instead of awk, addition of --non-decimal-data option, and removal of unbuffer -p.

    Note: When I leave unbuffer -p on, the zeros reappear. Food for thought. Perhaps, someone can attend to that later.