Chrome native messaging host in golang fails when JSON size is more than 65500 characters

Question

I am trying to write a native messaging host for chrome in golang. For this purpose, I tried using chrome-go as well as chrome-native-messaging packages. Both presented with the same problem as explained below.

Here is the code. I have added the relevant parts from the chrome-go package to the main file instead of importing it for easy understanding.

The following code actually works when I send a json message to it like {content:"Apple Mango"}. However, it stops working once the length of the json goes over approximately 65500 characters, give or take a 100 characters. There is no error output either.

package main

import (
  "encoding/binary"
  "encoding/json"
  "fmt"
  "io"
  "os"
)

var byteOrder binary.ByteOrder = binary.LittleEndian

func Receive(reader io.Reader) ([]byte, error) {
   // Read message length in native byte order
   var length uint32
   if err := binary.Read(reader, byteOrder, &length); err != nil {
       return nil, err
   }

// Return if no message
if length == 0 {
    return nil, nil
}

// Read message body
received := make([]byte, length)
if n, err := reader.Read(received); err != nil || n != len(received) {
    return nil, err
}
return received, nil
}

type response struct {
    Content string `json:"content"`
}

func main() {

  msg, err := Receive(os.Stdin)
  if err != nil {
    panic(err)
  }
  var res response
  err = json.Unmarshal([]byte(msg), &res)
  if err != nil {
     panic(err)
  }
  fmt.Println(res.Content)
 }

For those interested in testing, I have set up a repository with instructions. Run the following

  git clone --depth=1  https://[email protected]/tesseract-index/chrome-native-messaging-test-riz.git && cd chrome-native-messaging-test-riz
 ./json2msg.js < test-working.json | go run main.go
 ./json2msg.js < test-not-working.json | go run main.go

You will see that test-not-working.json gives no output, although its difference with test-working.json is a few hundred characters only.

What is the issue here?

Answer 1

There is a limitation of a pipe buffer which varies across systems. Mac OS X, for example, uses a capacity of 16384 bytes by default.

You can use this bash script to check your buffer capacity:

M=0; while printf A; do >&2 printf "\r$((++M)) B"; done | sleep 999

So it is not related to go, because I tried to change your code to read from file and Unmarshal and it worked:

func main() {
    reader, err := os.Open("test-not-working.json")
    if err != nil {
        panic(err)
    }

    var res response
    decoder := json.NewDecoder(reader)
    err = decoder.Decode(&res)
    if err != nil {
        panic(err)
    }

    fmt.Println(res.Content)
}