Changing the last character of a file

Question

I want to continuously write json objects to a file. To be able to read it, I need to wrap them into an array. I don't want to read the whole file, for simple appending. So what I' doing now:

comma := []byte(", ")
    file, err := os.OpenFile(erp.TransactionsPath, os.O_WRONLY|os.O_APPEND|os.O_CREATE, 0666)
    if err != nil {
        return err
    }
    transaction, err := json.Marshal(t)
    if err != nil {
        return err
    }
    transaction = append(transaction, comma...)
    file.Write(transaction)

But with this implementation I will need to add []scopes by hand(or via some script) before reading. How can I add an object before closing scope on each writing?

Answer 1

You don't need to wrap the JSON objects into an array, you can just write them as-is. You may use json.Encoder to write them to the file, and you may use json.Decoder to read them. Encoder.Encode() and Decoder.Decode() encode and decode individual JSON values from a stream.

To prove it works, see this simple example:

const src = `{"id":"1"}{"id":"2"}{"id":"3"}`
dec := json.NewDecoder(strings.NewReader(src))

for {
    var m map[string]interface{}
    if err := dec.Decode(&m); err != nil {
        if err == io.EOF {
            break
        }
        panic(err)
    }
    fmt.Println("Read:", m)
}

It outputs (try it on the Go Playground):

Read: map[id:1]
Read: map[id:2]
Read: map[id:3]

When writing to / reading from a file, pass the os.File to json.NewEncoder() and json.NewDecoder().

Here's a complete demo which creates a temporary file, uses json.Encoder to write JSON objects into it, then reads them back with json.Decoder:

objs := []map[string]interface{}{
    map[string]interface{}{"id": "1"},
    map[string]interface{}{"id": "2"},
    map[string]interface{}{"id": "3"},
}

file, err := ioutil.TempFile("", "test.json")
if err != nil {
    panic(err)
}

// Writing to file:
enc := json.NewEncoder(file)
for _, obj := range objs {
    if err := enc.Encode(obj); err != nil {
        panic(err)
    }
}

// Debug: print file's content
fmt.Println("File content:")
if data, err := ioutil.ReadFile(file.Name()); err != nil {
    panic(err)
} else {
    fmt.Println(string(data))
}

// Reading from file:
if _, err := file.Seek(0, io.SeekStart); err != nil {
    panic(err)
}
dec := json.NewDecoder(file)
for {
    var obj map[string]interface{}
    if err := dec.Decode(&obj); err != nil {
        if err == io.EOF {
            break
        }
        panic(err)
    }
    fmt.Println("Read:", obj)
}

It outputs (try it on the Go Playground):

File content:
{"id":"1"}
{"id":"2"}
{"id":"3"}

Read: map[id:1]
Read: map[id:2]
Read: map[id:3]