Search code examples
stringpointersgounsafe

Convert byte to string using reflect.StringHeader still allocates new memory?


I've got this small code snippet to test 2 ways of converting byte slice to string object, one function to allocate a new string object, another uses unsafe pointer arithmetic to construct string*, which doesn't allocate new memory:

package main

import (
    "fmt"
    "reflect"
    "unsafe"
)

func byteToString(b []byte) string {
    return string(b)
}

func byteToStringNoAlloc(b []byte) string {
    if len(b) == 0 {
        return ""
    }
    sh := reflect.StringHeader{uintptr(unsafe.Pointer(&b[0])), len(b)}
    return *(*string)(unsafe.Pointer(&sh))
}

func main() {
    b := []byte("hello")
    fmt.Printf("1st element of slice: %v\n", &b[0])

    str := byteToString(b)
    sh := (*reflect.StringHeader)(unsafe.Pointer(&str))
    fmt.Printf("New alloc: %v\n", sh)

    toStr := byteToStringNoAlloc(b)
    shNoAlloc := (*reflect.StringHeader)(unsafe.Pointer(&toStr))
    fmt.Printf("No alloc: %v\n", shNoAlloc) // why different from &b[0]
}

I run this program under go 1.13:

1st element of slice: 0xc000076068
New alloc: &{824634204304 5}
No alloc: &{824634204264 5}

I exptect that the "1st element of slice" should print out the same address like "No alloc", but acturally they're very different. Where did I get wrong?


Solution

  • First of all, type conversions are calling a internal functions, for this case it's slicebytetostring. https://golang.org/src/runtime/string.go?h=slicebytetostring#L75

    It does copy of slice's content into new allocated memory.

    In the second case you're creating a new header of the slice and cast it into string header the new unofficial holder of slice's content. The problem of this is that garbage collector doesn't handle such kind of cases and resulting string header will be marked as a single structure which has no relations with the actual slice which holds the actual content, so, your resulting string would be valid only while the actual content holders are alive (don't count this string header itself). So once garbage collector sweep the actual content, your string will still point to the same address but already freed memory, and you'll get the panic error or undefined behavior if you touch it.

    By the way, there's no need to use reflect package and its headers because direct cast already creates new header as a result:

    *(*string)(unsafe.Pointer(&byte_slice))