Sending data as binary via websockets encodes numbers as text

I'm working on a project where I'll be sending binary data across a websocket connection to an LED matrix. I want to send the binary data in the form of a byte array, e.g. drawing a diagonal line from right to left on the matrix would look like this:

The problem that I run into when I try to do this is that when I try to send this data, regardless of the websocket client tester I use, the numbers get encoded as strings.

That is, when I try to send the binary number 1, instead of sending a 0x01, the socket client sends a 49 in decimal/0x31 in hex which is the character code for the string '1' in ascii or unicode.

At first I thought this issue was on the arduino side in the code that's driving the matrix, but then I traced it back through the websocket server and all of the way to the testing clients. If I pop up wireshark and look at the data in flight from the client to the server, the encoding happens at the client, i.e. the first part of the websocket transmission.

I thought this may be me just using Firecamp wrong, but the same thing happens when using a different client, in this case websocat:

So, my question is: what is the proper way of sending binary numbers via websockets? Am I just misunderstanding how the binary feature is supposed to work, like is it expected that I send my numbers as strings and convert them back to numbers on the other side?

Solution

So after doing some reading and thinking I figured out my issue. @myst touched on it in the comment above as well.

In the example above where I'm typing the number I want into the websocat command's interactive prompt, I thought what I was typing in was being accepted as a number because the number was an integer and I didn't wrap it in quotes. Turns out that's still recognized as a string (makes sense in hind sight considering how cli tools accept arguments).

In the example above I also tried creating a test file and added the numbers in there, again thinking that they would be accepted as numbers, but they were read in as strings.

While I was troubleshooting I realized that if I was to send binary in another form, like say an image, the image file I was sending wouldn't be read in as text, it would be, you know, binary.

I.e. if you cat myimage.png you're not going to get a bunch of binary numbers, you're going to get blanks and garble.

It's not garble, it's just that you're spitting out a bunch of binary code and as the numbers come out some of them don't map to a character encoding (the blanks) and then some of them do match a character encoding but it's not in any meant-to-be-readable order so it looks like gibberish.

So, I looked up how to create a simple binary file by hand and found this super user post

You can use the printf command for it.

$ printf "\x08\x00\x00\x10" > file1
$ hexdump file1
00000000  08 00 00 10                                   |....|
00000004

So with that knowledge in hand I created a binary file, populating it with a simple hext count from 1 to 8 and then cat-ed that to the websocat tool:

 printf "\x01\x02\x03\x04\x05\x06\x07\x08" > binfile \
> cat binfile | websocat -b ws://localhost:3002

and THIS gives the expected output:

And actually after writing this post up I went back and tried it without the write -> cat of a file and it works fine that way too:

printf "\x01\x02\x03\x04\x05\x06\x07\x08" | websocat -b ws://localhost:3002

I'm not sure why this didn't work in Firecamp, but I'm already talking with their engineers and looking at a canary version for them so hopefully we'll be able to resolve it there too.

SO I'm good to go. Now I have to go back and rewrite some code :P

More info

I also spent some time working with the developers of Firecamp and it turns out their Array Buffer and Array Buffer View message types were indeed sending the ascii code for the values vs the actual numbers. They've since corrected the issue (I confirmed it in a canary release they passed me) and should be in v2.0.