Search code examples
bashshellawksedtr

Shell script, escape newlines but emit others?


Given a filename, I want to write a shell-script which emits the following, and pipes it into a process:

Content-Length:<LEN><CR><LF>
<CR><LF>
{ "jsonrpc":"2.0", "params":{ "text":"<ESCAPED-TEXT>" } }

where <ESCAPED-TEXT> is the content of the file but its CRs, LFs and quotation marks have been escaped as \r and \n and \" (and I guess all other JSON escapes will eventually be needed as well), and where <LEN> is the length of final JSON line that includes the escaped text.

Here's my current bash-script solution. It works but is ugly as heck.

(
  TXT=`cat ~/a.py | sed -E -e :a -e '$!N; s/\n/\\\n/g; ta' | sed 's/"/\\\"/g'`
  CMD='{"jsonrpc":"2.0", "params":{ "text":{"'${TXT}'"}} }'
  printf "Content-Length: ${#CMD}\r\n\r\n"
  echo -n "${CMD}"
) | pyls

Can anyone suggest how to do this cleaner, please?

  • This sed script only replaces LFs, not CRs. It accumulates each line into the buffer and then does a s//g to replace all LFs in it. I couldn't figure out anything cleaner that still worked on both Linux and OSX/BSD.

  • I used both printf and echo. First printf because I do want to emit the CRLFCRLF after the Content-Length header, and you apparently need printf for that because the behavior of echo with escapes isn't uniform across platforms. Next echo because I don't want the \r and \n literals inside TXT to be unescaped, which printf would do.

Context: there's a standard called "Language Server Protocol". Basically you run something like the pyls I'm running here, and you pipe in JsonRPC to it over stdin, and it pipes back stuff. Different people have written language servers for Python (the pyls I'm using here), and C#, and C++, and Typescript, and PHP, and OCaml, and Go, and Java, and each person tends to write their language server in their own language.

I want to write a test-harness which can send some example JsonRPC packets into any such server.

I figured it'd be better to write my test-harness in just the common basic shell-scripting stuff that's available on all platforms out of the box. That way everyone can use my test-harness against their language server. (If I wrote it on Python instead, say, it'd be easier for me to write, but it would force the C# folks to learn+install python just to run it, and likewise the Typescript, PHP, OCaml, Go and other folks.)


Solution

  • a.py:

    print("alfa")
    print("bravo")
    

    Awk script:

    {
      gsub("\r", "\\r")
      gsub("\42", "\\\42")
      z = z $0 "\\n"
    }
    END {
      printf "Content-Length: %d\r\n", length(z) + 42
      printf "\r\n"
      printf "{\42jsonrpc\42: \0422.0\42, \42params\42: {\42text\42: \42%s\42}}", z
    }
    

    Result:

    Content-Length: 81
    
    {"jsonrpc": "2.0", "params": {"text": "print(\"alfa\")\r\nprint(\"bravo\")\r\n"}}