cgo calling share library: cannot find lib or function?

I'm using the sample code of chapter 13 of The Go Programming Language as below:

$ cat bzip2.c
#include <bzlib.h>

int bz2compress(bz_stream *s, int action,
                char *in, unsigned *inlen, char *out, unsigned *outlen) {
    s->next_in = in;
    s->avail_in = *inlen;
    s->next_out = out;
    s->avail_out = *outlen;
    int r = BZ2_bzCompress(s, action);
    *inlen -= s->avail_in;
    *outlen -= s->avail_out;
    s->next_in = s->next_out = NULL;
    return r;
}

$ cat usebzip2.go
// Package bzip provides a writer that uses bzip2 compression (bzip.org).
package main
import "C"

import (
    "io"
    "log"
    "os"
    "testing"
    "unsafe"
)

type writer struct {
    w      io.Writer // underlying output stream
    stream *C.bz_stream
    outbuf [64 * 1024]byte
}

// Close flushes the compressed data and closes the stream.
// It does not close the underlying io.Writer.
func (w *writer) Close() error {
    if w.stream == nil {
        panic("closed")
    }
    defer func() {
        C.BZ2_bzCompressEnd(w.stream)
        C.bz2free(w.stream)
        w.stream = nil
    }()
    for {
        inlen, outlen := C.uint(0), C.uint(cap(w.outbuf))
        r := C.bz2compress(w.stream, C.BZ_FINISH, nil, &inlen,
            (*C.char)(unsafe.Pointer(&w.outbuf)), &outlen)
        if _, err := w.w.Write(w.outbuf[:outlen]); err != nil {
            return err
        }
        if r == C.BZ_STREAM_END {
            return nil
        }
    }
}

// NewWriter returns a writer for bzip2-compressed streams.
func NewWriter(out io.Writer) io.WriteCloser {
    const blockSize = 9
    const verbosity = 0
    const workFactor = 30
    w := &writer{w: out, stream: C.bz2alloc()}
    C.BZ2_bzCompressInit(w.stream, blockSize, verbosity, workFactor)
    return w
}

func main() {
    w := NewWriter(os.Stdout)
    if _, err := io.Copy(w, os.Stdin); err != nil {
        log.Fatalf("bzipper: %v\n", err)
    }
    if err := w.Close(); err != nil {
        log.Fatalf("bzipper: close: %v\n", err)
    }
}

First I compile the .c file:

gcc -I/usr/include -L/usr/lib -lbz2 --shared bzip2.c -fPIC -o libbzip2.so

The linux environment LD_LIBRARY_PATH contains ".", and then go build fails:

go build usebzip2.go
# command-line-arguments
/tmp/go-build677611698/b001/_x002.o: In function `_cgo_22d5d7fabfe4_Cfunc_bz2compress':
/tmp/go-build/cgo-gcc-prolog:118: undefined reference to `bz2compress'
collect2: error: ld returned 1 exit status

So how to fix it? I'm using ubuntu 18.04 LTS. Thanks a lot.

Solution

Don't run:

go build usebzip2.go

but rather:

go build

(and you don't need to invoke gcc directly on bzip2.c). When you use this process, you'll get many more (but different) errors because you have not put in the right directives before the:

import "C"

line. You need a comment (or series of comments) telling cgo about the functions you intend to provide, or providing those functions inline, and to direct the link phase to use -lbz2. In particular, you will need to:

#include <bzlib.h>
provide a bz2alloc function
provide a bz2free function
provide a declaration for your bz2compress function
set the LDFLAGS to include -lbz2

The actual bz2alloc and bz2free are short and simple and therefore can be included directly in this header block:

package main

/*
#cgo LDFLAGS: -lbz2     
#include <bzlib.h>
#include <stdlib.h>
bz_stream *bz2alloc() { return calloc(1, sizeof(bz_stream)); }
int bz2compress(bz_stream *s, int action,
    char *in, unsigned *intlen, char *out, unsigned *outlen);
void bz2free(bz_stream* s) { free(s); }
*/      
import "C"

If you insert this and run go build you will now see a different and more useful error:

./usebzip2.go:60:2: cannot use w (type *writer) as type io.WriteCloser in return argument:
        *writer does not implement io.WriteCloser (missing Write method)

which is of course because type writer does not implement Write.

(There's a completed version of exercise 13.3—not mine—at https://github.com/torbiak/gopl/tree/master/ex13.3. Note that they have augmented theirs to use locking as well, making it safe to call the write function from multiple goroutines simultaneously.)