Search code examples
gobinaryfiles

How to read packed binary data in Go?


I'm trying to figure out the best way to read a packed binary file in Go that was produced by Python like the following:

import struct
f = open('tst.bin', 'wb')
fmt = 'iih' #please note this is packed binary: 4byte int, 4byte int, 2byte int
f.write(struct.pack(fmt,4, 185765, 1020))
f.write(struct.pack(fmt,4, 185765, 1022))
f.close()

I have been tinkering with some of the examples I've seen on Github.com and a few other sources but I can't seem to get anything working correctly (update shows working method). What is the idiomatic way to do this sort of thing in Go? This is one of several attempts

UPDATE and WORKING

package main

    import (
            "fmt"
            "os"
            "encoding/binary"
            "io"
            )

    func main() {
            fp, err := os.Open("tst.bin")

            if err != nil {
                    panic(err)
            }

            defer fp.Close()

            lineBuf := make([]byte, 10) //4 byte int, 4 byte int, 2 byte int per line

            for true {
                _, err := fp.Read(lineBuf)

                if err == io.EOF{
                    break
                }

                aVal := int32(binary.LittleEndian.Uint32(lineBuf[0:4])) // same as: int32(uint32(b[0]) | uint32(b[1])<<8 | uint32(b[2])<<16 | uint32(b[3])<<24)
                bVal := int32(binary.LittleEndian.Uint32(lineBuf[4:8]))
                cVal := int16(binary.LittleEndian.Uint16(lineBuf[8:10])) //same as: int16(uint32(b[0]) | uint32(b[1])<<8)
                fmt.Println(aVal, bVal, cVal)
            }
    }

Solution

  • The Python format string is iih, meaning two 32-bit signed integers and one 16-bit signed integer (see the docs). You can simply use your first example but change the struct to:

    type binData struct {
        A int32
        B int32
        C int16
    }
    
    func main() {
            fp, err := os.Open("tst.bin")
    
            if err != nil {
                    panic(err)
            }
    
            defer fp.Close()
    
            for {
                thing := binData{}
                err := binary.Read(fp, binary.LittleEndian, &thing)
    
                if err == io.EOF{
                    break
                }
    
                fmt.Println(thing.A, thing.B, thing.C)
            }
    }
    

    Note that the Python packing didn't specify the endianness explicitly, but if you're sure the system that ran it generated little-endian binary, this should work.

    Edit: Added main() function to explain what I mean.

    Edit 2: Capitalized struct fields so binary.Read could write into them.