Search code examples
gocgogem5

Modifying go to use Gem5 m5ops: go tool dist shows import message then quit

I am pretty new to golang.

I'm trying to insert some C function calls (specifically m5ops) to golang's runtime (specifically src/proc.go).

I was trying to use cgo, which results in adding this import to the beginning of proc.go:

// #cgo CFLAGS: -I$(GEM5_PATH)/include -static
// #cgo LDFLAGS: -L$(GEM5_PATH)/util/m5/build/x86/out -lm5
// #include <gem5/m5ops.h>
import "C"

but when I tryna rebuild, using ./all.bash, the script output this then exited:

./all.bash
Building Go cmd/dist using /usr/local/go. (go1.20.5 linux/amd64)
Building Go toolchain1 using /usr/local/go.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
go tool dist: /home/landerx/goroot/src/runtime/proc.go imports C

A more verbose version of above output:

./all.bash -v
Building Go cmd/dist using /usr/local/go. (go1.20.5 linux/amd64)
cmd/dist

Building Go toolchain1 using /usr/local/go.
internal/goarch
internal/coverage/rtcov
internal/goos
internal/race
internal/unsafeheader
internal/goexperiment
unicode
runtime/internal/syscall
sync/atomic
runtime/internal/atomic
internal/cpu
unicode/utf8
internal/itoa
math/bits
internal/abi
runtime/internal/math
runtime/internal/sys
bootstrap/internal/goarch
bootstrap/internal/goexperiment
bootstrap/math/bits
encoding
bootstrap/internal/unsafeheader
bootstrap/internal/platform
bootstrap/internal/coverage
internal/bytealg
bootstrap/internal/abi
math
unicode/utf16
bootstrap/internal/goversion
internal/goversion
bootstrap/internal/race
bootstrap/cmd/compile/internal/test
runtime
internal/reflectlite
runtime/metrics
sync
internal/testlog
internal/godebug
bootstrap/internal/bisect
math/rand
errors
sort
bootstrap/sort
io
internal/oserror
internal/safefilepath
path
strconv
bootstrap/strconv
syscall
bootstrap/container/heap
hash
text/tabwriter
bytes
strings
hash/adler32
hash/fnv
hash/crc32
crypto
reflect
bootstrap/internal/types/errors
go/build/constraint
bootstrap/go/build/constraint
bufio
regexp/syntax
html
internal/syscall/execenv
internal/syscall/unix
time
regexp
context
io/fs
internal/poll
internal/fmtsort
bootstrap/internal/saferio
encoding/binary
os
bootstrap/cmd/internal/notsha256
encoding/base64
bootstrap/cmd/internal/sys
crypto/md5
bootstrap/internal/lazyregexp
internal/lazyregexp
path/filepath
fmt
os/exec
log
bootstrap/internal/buildcfg
flag
bootstrap/cmd/internal/src
text/scanner
bootstrap/cmd/internal/edit
bootstrap/cmd/internal/pkgpath
go/token
bootstrap/compress/flate
bootstrap/internal/zstd
go/doc/comment
bootstrap/debug/dwarf
encoding/json
runtime/debug
bootstrap/math/big
bootstrap/cmd/internal/bio
net/url
go/scanner
bootstrap/cmd/internal/objabi
bootstrap/cmd/compile/internal/abt
bootstrap/cmd/internal/goobj
bootstrap/cmd/internal/gcprog
bootstrap/cmd/internal/quoted
bootstrap/compress/zlib
encoding/hex
compress/flate
go/ast
internal/buildcfg
internal/goroot
bootstrap/cmd/internal/dwarf
runtime/trace
bootstrap/cmd/internal/archive
bootstrap/cmd/internal/obj
bootstrap/debug/macho
bootstrap/debug/elf
bootstrap/debug/pe
bootstrap/internal/xcoff
bootstrap/cmd/link/internal/dwtest
compress/gzip
bootstrap/internal/profile
runtime/pprof
go/doc
go/printer
go/internal/typeparams
go/parser
bootstrap/cmd/internal/codesign
bootstrap/go/constant
bootstrap/internal/pkgbits
bootstrap/cmd/compile/internal/syntax
bootstrap/cmd/link/internal/benchmark
bootstrap/cmd/compile/internal/pgo/internal/graph
go/build
go/format
bootstrap/cmd/cgo
bootstrap/cmd/internal/obj/loong64
bootstrap/cmd/internal/obj/mips
bootstrap/cmd/internal/obj/wasm
bootstrap/cmd/internal/obj/arm
bootstrap/cmd/internal/obj/arm64
bootstrap/cmd/internal/obj/s390x
bootstrap/cmd/internal/obj/riscv
bootstrap/cmd/internal/obj/ppc64
bootstrap/cmd/internal/obj/x86
bootstrap/cmd/asm/internal/flags
bootstrap/cmd/compile/internal/base
bootstrap/cmd/compile/internal/logopt
bootstrap/cmd/asm/internal/lex
bootstrap/cmd/link/internal/sym
bootstrap/cmd/link/internal/loader
bootstrap/cmd/compile/internal/bitvec
bootstrap/cmd/compile/internal/types
bootstrap/cmd/compile/internal/types2
bootstrap/cmd/link/internal/loadelf
bootstrap/cmd/link/internal/loadmacho
bootstrap/cmd/link/internal/loadpe
bootstrap/cmd/link/internal/loadxcoff
bootstrap/cmd/compile/internal/typebits
bootstrap/cmd/compile/internal/ir
bootstrap/cmd/asm/internal/arch
bootstrap/cmd/link/internal/ld
bootstrap/cmd/asm/internal/asm
bootstrap/cmd/asm
bootstrap/cmd/compile/internal/abi
bootstrap/cmd/compile/internal/deadcode
bootstrap/cmd/compile/internal/typecheck
bootstrap/cmd/compile/internal/objw
bootstrap/cmd/compile/internal/staticdata
bootstrap/cmd/compile/internal/compare
bootstrap/cmd/compile/internal/pgo
bootstrap/cmd/compile/internal/coverage
bootstrap/cmd/compile/internal/loopvar
bootstrap/cmd/compile/internal/importer
bootstrap/cmd/compile/internal/escape
bootstrap/cmd/compile/internal/inline
bootstrap/cmd/compile/internal/reflectdata
bootstrap/cmd/compile/internal/devirtualize
bootstrap/cmd/compile/internal/staticinit
bootstrap/cmd/compile/internal/ssa
bootstrap/cmd/link/internal/amd64
bootstrap/cmd/link/internal/loong64
bootstrap/cmd/link/internal/arm
bootstrap/cmd/link/internal/mips
bootstrap/cmd/link/internal/mips64
bootstrap/cmd/link/internal/arm64
bootstrap/cmd/link/internal/ppc64
bootstrap/cmd/link/internal/riscv64
bootstrap/cmd/link/internal/s390x
bootstrap/cmd/link/internal/wasm
bootstrap/cmd/link/internal/x86
bootstrap/cmd/link
bootstrap/cmd/compile/internal/liveness
bootstrap/cmd/compile/internal/ssagen
bootstrap/cmd/compile/internal/amd64
bootstrap/cmd/compile/internal/arm
bootstrap/cmd/compile/internal/arm64
bootstrap/cmd/compile/internal/dwarfgen
bootstrap/cmd/compile/internal/loong64
bootstrap/cmd/compile/internal/mips
bootstrap/cmd/compile/internal/mips64
bootstrap/cmd/compile/internal/wasm
bootstrap/cmd/compile/internal/s390x
bootstrap/cmd/compile/internal/riscv64
bootstrap/cmd/compile/internal/ppc64
bootstrap/cmd/compile/internal/walk
bootstrap/cmd/compile/internal/x86
bootstrap/cmd/compile/internal/noder
bootstrap/cmd/compile/internal/pkginit
bootstrap/cmd/compile/internal/gc
bootstrap/cmd/compile

Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
runtime
go tool dist: /home/landerx/goroot/src/runtime/proc.go imports C

Tried the CGO_ENABLED=1 flag, seemingly didn't make any difference.

Tried running only make.bash, seemingly didn't make any difference.

I wonder if it is a dependency problem, but didn't see any related message to the pseudo-package cgo, didn't see any error from my vscode either.

Solution

  • Ok... Finally, let me myself answer this.

    Why it doesn't work

    After some investigation, I found that seemingly Go (tool dist) doesn't allow importing cgo (import "C") in most of the go inner source files (link)

    Workarounds

    Still there are some workarounds to achieve the effect, which is to use m5ops in go.

    1. Pass function pointers through global vars

    U can define global function pointers in proc.go, and in ur go application, import cgo, and assign those global function pointers with ur functions that do m5ops. Calling C function pointers is not supported, so u need to wrap it with a golang function.

    2. Go Asm

    Go uses a lot Go Asm in its inner source files, to handle stack switch, etc.

    On the other hand m5ops will at the end be inlined as a single pseudo-instruction.

    So following Go Asm's syntax and function binding, which happens in asm_amd64.s and stubs.go, it is feasible to write an asm function that includes the m5ops instruction, then call it wherever in ur runtime.go should be good.

    Other info

    • It seems that in runtime.go, when it switch to system stack, it can no longer execute function outside of the file(or the package?), so method 1 won't work if u try to insert those function calls into code which will run on system stack. So I finally discovered the second method.