Concurrency-safe templates in Go: How do I do it?

Question

I have the following call:

import (
  "text/template"
)

//...

template.New(filepath.Base(name)).Funcs(templateFunctions).Parse(string(asset))

which is called in several Go routines concurrently, which in turn causes the following panic:

fatal error: concurrent map iteration and map write

Here is the backtrace:

goroutine 140 [running]:
text/template.addValueFuncs(0xc00188e000?, 0xc00188e000?)
        [...]/go/src/text/template/funcs.go:88 +0x76
[...]/modules/template.loadEmbeddedTemplates({0x38ff6cb?, 0xc001cf8060?})
        [...]/src/modules/template/configBased.go:163 +0x749

The line on src/modules/template/configBased.go:163 quoted above. It is template.New(...).

The surrounding function is called from goroutines concurrently.

That is the code from go/src/text/template/funcs.go:88 is it helps:

// addValueFuncs adds to values the functions in funcs, converting them to reflect.Values.
func addValueFuncs(out map[string]reflect.Value, in FuncMap) {
    for name, fn := range in {
        if !goodName(name) {
            panic(fmt.Errorf("function name %q is not a valid identifier", name))
        }
        v := reflect.ValueOf(fn)
        if v.Kind() != reflect.Func {
            panic("value for " + name + " not a function")
        }
        if !goodFunc(v.Type()) {
            panic(fmt.Errorf("can't install method/function %q with %d results", name, v.Type().NumOut()))
        }
        out[name] = v
    }
}

If template.New is concurrent-safe, why this line produces this panic, and how am I supposed to handle it properly?

Update.

The code of the nasty function loadEmbeddedTemplates:

func loadEmbeddedTemplates(templateFile string) (*template.Template, error) {
    var t *template.Template

    templateFile = filepath.Join("share", "templates", filepath.Base(templateFile))
    dir := filepath.Dir(templateFile)
    names := assets.GetAssetNames()

    // All templates except + the last one at the end
    filteredNames := []string{}

    for _, name := range names {
        if !strings.HasPrefix(name, dir+"/") || !strings.HasSuffix(name, ".tmpl") {
            continue
        }

        if name != templateFile {
            filteredNames = append(filteredNames, name)
        }
    }

    filteredNames = append(filteredNames, templateFile)

    for _, name := range filteredNames {
        asset, err := assets.GetAsset(name)
        if err != nil {
            return nil, err
        }

        if t == nil {
            t, err = template.New(filepath.Base(name)).Funcs(templateFunctions).Parse(string(asset))
        } else {
            t, err = t.New(filepath.Base(name)).Parse(string(asset))
        }

        if err != nil {
            return nil, err
        }
    }

    return t, nil
}

The function simply loads all templates from share/templates/ one after another

Answer 1

Your loadEmbeddedTemplates() function accesses the templateFunctions variable, passes it to Template.Funcs() which will obviously read it (will iterate over it).

And you are likely populating it concurrently, in another goroutine. Hence the concurrent map write error. Access to it must be synchronized.

If possible, populate it first, and only after that start using it (passing it to Template.Funcs()). That way no additional synchronization or locking will be required (concurrent read only is always OK).