How to handle char * from packed struct in cgo?

Since Go doesn’t support packed struct I found this great article explains everything with examples how to work with packed struct in go. https://medium.com/@liamkelly17/working-with-packed-c-structs-in-cgo-224a0a3b708b

The problem is when I try char * in place of [10]char it's not working. I'm not sure how this conversion works with [10]char and not with char * . Here is example code taken from above article and modified with char * .

package main

/*
#include "stdio.h"
#pragma pack(1)
typedef struct{
    unsigned char a;
    char b;
    int c;
    unsigned int d;
    char *e; // changed from char[10] to char *
}packed;

void PrintPacked(packed p){
    printf("\nFrom C\na:%d\nb:%d\nc:%d\nd:%d\ne:%s\n", p.a, p.b, p.c, p.d, p.e);
}

*/
import "C"
import (
    "bytes"
    "encoding/binary"
)

//GoPack is the go version of the c packed structure
type GoPack struct {
    a uint8
    b int8
    c int32
    d uint32
    e [10]uint8
}

//Pack Produces a packed version of the go struct
func (g *GoPack) Pack(out *C.packed) {
    buf := &bytes.Buffer{}
    binary.Write(buf, binary.LittleEndian, g)
    *out = *(*C.packed)(C.CBytes(buf.Bytes()))
}

func main() {
    pack := &GoPack{1, 2, 3, 4, [10]byte{}}
    copy(pack.e[:], "TEST123")
    cpack := C.packed{} //just to allocate the memory, still under GC control
    pack.Pack(&cpack)
    C.PrintPacked(cpack)
}

I'm working with cgo first time so correct me if i am wrong at any point.

Solution

You are writing ten (zero) bytes of GoPack.e into the packed.e which is of type char *. This won't work, because pointers will be 4 or 8 bytes depending on your system, so even if the bytes represented a valid pointer, you are overflowing the amount of memory allocated.

If you want to create a valid structure with a valid packed.e field, you need to allocate 10 bytes of memory in the C heap, copy the bytes into that, and then point packed.e to this allocated memory. (You will also need to free this memory when you free the corresponding packed structure). You can't do this directly with binary.Write.

You can take this as a starting point:

buf := &bytes.Buffer{}
binary.Write(buf, binary.LittleEndian, g.a)
binary.Write(buf, binary.LittleEndian, g.b)
binary.Write(buf, binary.LittleEndian, g.c)
binary.Write(buf, binary.LittleEndian, g.d)
binary.Write(buf, binary.LittleEndian, uintptr(C.CBytes(g.e))
*out = *(*C.packed)(C.CBytes(buf.Bytes()))

The function C.CBytes(b) allocates len(b) bytes in the C heap, and copies the bytes from b into it, returning an unsafe.Pointer.

Note that I've copied your *out = *(*C.packed)... line from your code. This actually causes a memory leak and an unnecessary copy. Probably it would be better to use a writer that writes bytes directly to the memory pointed to by out.

Perhaps this?

const N = 10000 // should be sizeof(*out) or larger
buf := bytes.NewBuffer((*[N]byte)(unsafe.Pointer(out))[:])

This makes a bytes.Buffer that directly writes to the out struct without going through any intermediate memory. Note that because of unsafe shenanigans, this is vulnerable to a buffer overflow if you write more bytes of data than is pointed to by out.

Words of warning: this is all pretty nasty, and prone to the same sorts of problems you'd find in C, and you'd need to check the cgo pointer rules to make sure that you're not vulnerable to garbage collection interactions. A point of advice: given that you say you "don't have much experience with pointers and memory allocation", you probably should avoid writing or including code like this because the problems it can introduce are nefarious and may not be immediately obvious.