How to get PubSub message data in an API?

Question

I want to get this kind of message json data from Pub/Sub

{
  "message": {
    "data": {
      "from": "no-reply@example.com",
      "to": "user@example.com",
      "subject": "test",
      "body": "test"
    }
  }
}

And parse its data to use for other service.

private parseMessage(message: Message) {
  try {
    const decoded = Buffer.from(message.data.toString(), 'base64').toString().trim();
    return JSON.parse(decoded);
  } catch (err) {
    throw new BadRequestException(
      'Parse Error: ' + message,
    );
  }
}

But when run the API got this error:

SyntaxError: Unexpected token � in JSON at position 0
  at JSON.parse (<anonymous>)
  at EventController.parseMessage (../myapp/src/api/posts/posts.controller.ts:44:18)
response: {
  statusCode: 400,
  message: 'Parse Error: [object Object]',
  error: 'Bad Request'
},
status: 400

It seems this post isn't right:

curl -X 'POST' \
  'http://localhost:3000/posts' \
  -H 'Content-Type: application/json' \
  -d '{
  "message": {
    "data": {
      "from": "no-reply@example.com",
      "to": "user@example.com",
      "subject": "test",
      "body": "test"
    }
  }
}'

Then how to make fake Pub/Sub message data?

Answer 1

I think you need to encode your data into Base64.

Base64 encoding schemes need to encode binary data that needs to be stored and transferred over media that are designed to deal with ASCII. This is to ensure that the data remain intact without modification during transport.

You can also refer to this GCP public documentation.

Eg. From the Doc:

  # 'world' base64-encoded is 'd29ybGQ='
curl localhost:8080 \
  -X POST \
  -H "Content-Type: application/json" \
  -d '{
        "context": {
          "eventId":"1144231683168617",
          "timestamp":"2020-05-06T07:33:34.556Z",
          "eventType":"google.pubsub.topic.publish",
          "resource":{
            "service":"pubsub.googleapis.com",
            "name":"projects/sample-project/topics/gcf-test",
            "type":"type.googleapis.com/google.pubsub.v1.PubsubMessage"
          }
        },
        "data": {
          "@type": "type.googleapis.com/google.pubsub.v1.PubsubMessage",
          "attributes": {
             "attr1":"attr1-value"
          },
          "data": "d29ybGQ="
        }
      }'