Search code examples
curlgoogle-cloud-pubsub

GCP PubSub: Publish message via CURL type of request

Does anyone have a working example of how to publish a message to a GCP PubSub topic via CURL type of commands, directly from shell?

I'm trying to not use the CLI and not use the client libraries, and I'm getting hung up on the OAUTH stuff.

I'd be great to have a bullet list of the things a bash script running on Linux would need to do, if anyone has such or can cobble one together I'd appreciate it very much.

Items I already have:

  • I have a Linux compute engine I'm working on
  • I've got working examples of publishing via CLI and Python
  • I've got a service account which is working nicely with pubsub (see previous bullet)

I fully recognize Google recommends using the CLI or API Client Libraries, but I need to be able to run this on a host with minimal installations (no CLI, no python libraries, etc.).

I think I need to do the following:

  1. base64 encode my data
  2. create a JSON Web Tokens (JWT)
  3. use the JWS to get a OAUTH token
  4. use token to call the API - e.g. POST https://pubsub.googleapis.com/v1/projects/myproject/topics/mytopic:publish

Ideas appreciated and thank you very much...Rich

Reference links:

https://cloud.google.com/pubsub/docs/publisher#pubsub-publish-message-protocol https://groups.google.com/forum/#!topic/cloud-pubsub-discuss/8fGaG5cWiTk https://groups.google.com/forum/?hl=sw#!topic/cloud-pubsub-discuss/8fGaG5cWiTk https://developers.google.com/identity/protocols/OAuth2WebServer https://developers.google.com/identity/protocols/OAuth2ServiceAccount

Solution

  • Yes, it is possible to publish with the curl command. You would need to pass in the access token into the request. To get it, log into the desired service account that you will use for publishing. If you haven't created one, you can do so and limit to only allow publishing:

    gcloud iam service-accounts create curl-publisher

gcloud projects add-iam-policy-binding <project with topic> --member=serviceAccount:curl-publisher@<project with created service account>.iam.gserviceaccount.com --role=roles/pubsub.publisher

    Now, you can log into the service account from the command line and get the access token:

    gcloud auth login curl-publisher@<project with created service account>.iam.gserviceaccount.com

gcloud auth application-default print-access-token

    You only need to run these above steps once in order to get the access token. Now, you can use that access token in a curl command:

    PROJECT=my-project
TOPIC=my-topic
ACCESS_TOKEN=<token printed out above>
curl -H 'content-type: application/json' -H "Authorization: Bearer $ACCESS_TOKEN" -X POST --data $'{  "messages": [{"data": "abcd"}]}'  https://pubsub.googleapis.com/v1/projects/$PROJECT/topics/$TOPIC:publish