Search code examples
arraysamazon-web-servicesflutterdartamazon-transcribe

Unable to decode network response from AWS


I'm trying to setup a websocket for AWS transcribe, but I've been stuck for over a day trying to decode the byte data / Uint8ArrayView that I receive as a response. Any help is very much appreciated, I've tried tons of decoding, ByteBuffers, etc, and can't figure it out.

Here is a little document on how the data should be coming in: https://docs.aws.amazon.com/transcribe/latest/dg/event-stream.html

Here is how somebody easily does this with the same response in Javascript: Using marshaller (Full code on GitHub)

let messageWrapper = eventStreamMarshaller.unmarshall(Buffer(message.data));
let messageBody = JSON.parse(String.fromCharCode.apply(String, messageWrapper.body));

And here is the data that I receive as a response:

[0, 0, 3, 212, 0, 0, 0, 103, 76, 21, 128, 165, 15, 58, 101, 120, 99, 101, 112, 116, 105, 111, 110, 45, 116, 121, 112, 101, 7, 0, 25, 73, 110, 118, 97, 108, 105, 100, 83, 105, 103, 110, 97, 116, 117, 114, 101, 69, 120, 99, 101, 112, 116, 105, 111, 110, 13, 58, 99, 111, 110, 116, 101, 110, 116, 45, 116, 121, 112, 101, 7, 0, 16, 97, 112, 112, 108, 105, 99, 97, 116, 105, 111, 110, 47, 106, 115, 111, 110, 13, 58, 109, 101, 115, 115, 97, 103, 101, 45, 116, 121, 112, 101, 7, 0, 9, 101, 120, 99, 101, 112, 116, 105, 111, 110, 123, 34, 77, 101, 115, 115, 97, 103, 101, 34, 58, 34, 84, 104, 101, 32, 114, 101, 113, 117, 101, 115, 116, 32, 115, 105, 103, 110, 97, 116, 117, 114, 101, 32, 119, 101, 32, 99, 97, 108, 99, 117, 108, 97, 116, 101, 100, 32, 100, 111, 101, 115, 32, 110, 111, 116, 32, 109, 97, 116, 99, 104, 32, 116, 104, 101, 32, 115, 105, 103, 110, 97, 116, 117, 114, 101, 32, 121, 111, 117, 32, 112, 114, 111, 118, 105, 100, 101, 100, 46, 32, 67, 104, 101, 99, 107, 32, 121, 111, 117, 114, 32, 65, 87, 83, 32, 83, 101, 99, 114, 101, 116, 32, 65, 99, 99, 101, 115, 115, 32, 75, 101, 121, 32, 97, 110, 100, 32, 115, 105, 103, 110, 105, 110, 103, 32, 109, 101, 116, 104, 111, 100, 46, 32, 67, 111, 110, 115, 117, 108, 116, 32, 116, 104, 101, 32, 115, 101, 114, 118, 105, 99, 101, 32, 100, 111, 99, 117, 109, 101, 110, 116, 97, 116, 105, 111, 110, 32, 102, 111, 114, 32, 100, 101, 116, 97, 105, 108, 115, 46, 92, 110, 92, 110, 84, 104, 101, 32, 67, 97, 110, 111, 110, 105, 99, 97, 108, 32, 83, 116, 114, 105, 110, 103, 32, 102, 111, 114, 32, 116, 104, 105, 115, 32, 114, 101, 113, 117, 101, 115, 116, 32, 115, 104, 111, 117, 108, 100, 32, 104, 97, 118, 101, 32, 98, 101, 101, 110, 92, 110, 39, 71, 69, 84, 92, 110, 47, 115, 116, 114, 101, 97, 109, 45, 116, 114, 97, 110, 115, 99, 114, 105, 112, 116, 105, 111, 110, 45, 119, 101, 98, 115, 111, 99, 107, 101, 116, 92, 110, 88, 45, 65, 109, 122, 45, 65, 108, 103, 111, 114, 105, 116, 104, 109, 61, 65, 87, 83, 52, 45, 72, 77, 65, 67, 45, 83, 72, 65, 50, 53, 54, 38, 88, 45, 65, 109, 122, 45, 67, 114, 101, 100, 101, 110, 116, 105, 97, 108, 61, 65, 75, 73, 65, 84, 52, 55, 65, 90, 72, 82, 81, 79, 82, 88, 72, 77, 85, 77, 79, 37, 50, 70, 50, 48, 50, 49, 48, 52, 49, 48, 37, 50, 70, 117, 115, 45, 101, 97, 115, 116, 45, 50, 37, 50, 70, 116, 114, 97, 110, 115, 99, 114, 105, 98, 101, 37, 50, 70, 97, 119, 115, 52, 95, 114, 101, 113, 117, 101, 115, 116, 38, 88, 45, 65, 109, 122, 45, 68, 97, 116, 101, 61, 50, 48, 50, 49, 48, 52, 49, 48, 84, 49, 56, 48, 50, 52, 57, 90, 38, 88, 45, 65, 109, 122, 45, 69, 120, 112, 105, 114, 101, 115, 61, 56, 54, 52, 48, 48, 38, 88, 45, 65, 109, 122, 45, 83, 105, 103, 110, 101, 100, 72, 101, 97, 100, 101, 114, 115, 61, 104, 111, 115, 116, 38, 108, 97, 110, 103, 117, 97, 103, 101, 45, 99, 111, 100, 101, 61, 101, 110, 45, 85, 83, 38, 109, 101, 100, 105, 97, 45, 101, 110, 99, 111, 100, 105, 110, 103, 61, 112, 99, 109, 38, 115, 97, 109, 112, 108, 101, 45, 114, 97, 116, 101, 61, 49, 54, 48, 48, 48, 92, 110, 104, 111, 115, 116, 58, 116, 114, 97, 110, 115, 99, 114, 105, 98, 101, 115, 116, 114, 101, 97, 109, 105, 110, 103, 46, 117, 115, 45, 101, 97, 115, 116, 45, 50, 46, 97, 109, 97, 122, 111, 110, 97, 119, 115, 46, 99, 111, 109, 58, 56, 52, 52, 51, 92, 110, 92, 110, 104, 111, 115, 116, 92, 110, 101, 51, 98, 48, 99, 52, 52, 50, 57, 56, 102, 99, 49, 99, 49, 52, 57, 97, 102, 98, 102, 52, 99, 56, 57, 57, 54, 102, 98, 57, 50, 52, 50, 55, 97, 101, 52, 49, 101, 52, 54, 52, 57, 98, 57, 51, 52, 99, 97, 52, 57, 53, 57, 57, 49, 98, 55, 56, 53, 50, 98, 56, 53, 53, 39, 92, 110, 92, 110, 84, 104, 101, 32, 83, 116, 114, 105, 110, 103, 45, 116, 111, 45, 83, 105, 103, 110, 32, 115, 104, 111, 117, 108, 100, 32, 104, 97, 118, 101, 32, 98, 101, 101, 110, 92, 110, 39, 65, 87, 83, 52, 45, 72, 77, 65, 67, 45, 83, 72, 65, 50, 53, 54, 92, 110, 50, 48, 50, 49, 48, 52, 49, 48, 84, 49, 56, 48, 50, 52, 57, 90, 92, 110, 50, 48, 50, 49, 48, 52, 49, 48, 47, 117, 115, 45, 101, 97, 115, 116, 45, 50, 47, 116, 114, 97, 110, 115, 99, 114, 105, 98, 101, 47, 97, 119, 115, 52, 95, 114, 101, 113, 117, 101, 115, 116, 92, 110, 52, 57, 54, 98, 50, 57, 49, 57, 54, 100, 100, 101, 98, 100, 57, 55, 101, 53, 100, 56, 54, 56, 54, 98, 54, 54, 100, 97, 101, 102, 48, 97, 100, 98, 102, 56, 53, 49, 48, 54, 53, 54, 99, 53, 52, 102, 100, 99, 98, 98, 101, 101, 100, 98, 49, 50, 54, 101, 55, 100, 54, 51, 57, 99, 39, 92, 110, 34, 125, 25, 123, 100, 219]

Solution

  • Huge thanks to @Anon-Coward I converted his python code into flutter code which decodes the message and makes the headers and body readable. It's a bit sloppy, but here it is: (testVal is the response from Amazon which is in my question).

                            Uint8List sample = Uint8List.fromList(testVal);
                            ByteData bytes = sample.buffer.asByteData();
    
                            var totalLength = bytes.lengthInBytes; //980
                            var headersLength = bytes.getInt8(7); //103
    
                            var data = sample.sublist(12);
                            
                            var left = headersLength;
    
                            while(left > 0){
                              //get the header name
                              var nameLength = data[0];
                              data = data.sublist(1);
                              var name = utf8.decode(data.sublist(0, nameLength));
                              data = data.sublist(nameLength);
                              //get the value type
                              var valueType = data[0];
                              data = data.sublist(1);
                              //get the value
                              var tempLenList = data.sublist(0, 2);
                              var tempLenBytes = tempLenList.buffer.asByteData();
                              var valueLength = tempLenBytes.getUint16(0, Endian.big);
                              data = data.sublist(2);
                              var value = data.sublist(0, valueLength);
                              data = data.sublist(valueLength);
                              var valueFinal;
                              if(valueType == 7){
                                valueFinal = utf8.decode(value);
                              }
    
                              left -= (4 + nameLength + valueLength);
                              print(name+ '   ' + valueFinal);
                            }
    
                            var remainingData = utf8.decode(data.sublist(0, (data.lengthInBytes - 4)));
                            print(remainingData);