How do I publish a message to an IoT thing topic on AWS using HTTPS and Postman?

I would like to publish a message to an IoT thing topic on AWS using HTTPS and Postman.

EDIT: I have edited majority of the post to include a step-by-step process of the thing i created

I created a thing called TestDevice by following these steps:

AWS IoT console > Manage (left dashboard panel) > All devices > Things > Create Things (on the ceneter page) > Create Single Thing > Named it TestDevice (No shadow and additional configurations) > Auto generate a new certificate > created a policy named TestDevice-policy (this will open a new browser tab where you will create the policy > attach it to the thing > Finish creating the thing and download all the certificates

The policy looks like this:

The certificates downloaded looks like this :

Now according to the AWS developer guide, my endpoint should be:

https://xxx-ats.iot.ap-northeast-1.amazonaws.com/topics/TestTopic?qos=1

I opened the AWS IoT MQTT test client in the console and subscribed to # which should allow me to see all incoming messages.

Setting up postman as @Ermiya Eskandary Instructed

I get a socket hang-up error on POSTMAN when hitting send:

So i can gurantee that it is not an AWS IoT setting problem because running the python examples in the AWS developer guide i get an "OK" response

So i must have done some postman settings wrong

Solution

Postman can definitely connect to the HTTPs endpoint and publish a message, though the AWS IoT docs admittedly aren't the best on explaining this.

I want to make sure I answer all of your questions so there's a lot of detail here, but we'll go through it bit by bit.

Now i opened the AWS Iot test console and subscribe to a #

Correct.

# and + are the MQTT wildcard characters. Subscribing to either of them will subscribe you to all topics.

It's easy to confuse them with the AWS IAM policy wildcard characters - * & ? - which you can use in IoT Core policies but not for subscriptions. Using them as topic names for subscriptions won't yield any messages.

Which certificates do I use?

Your downloaded connection kit for Windows (as the names will differ for Linux/Mac) will have at a minimum:

{thing-name}.cert.pem: device certificate
{thing-name}.private.key: public key file
{thing-name}.public.key: private key file

I'm using the eu-west-1 region which uses the above naming convention however, clearly it's not consistent across all regions.

Nonetheless, you can deduce it most of the time as the device certificate is the .pem file, the private key has private in the name & the public key has public in the name.

In your case, they are respectively:

certificate.cert.pem: device's certificate
public.pem.key: device's public key file
private.pem.key: device's private key file

You then may also have 1 or more CA (certificate authority) certificate files. Currently, per docs, AWS IoT Core server authentication certificates are signed by one of the following root CA certificates (but you might also see CA 2 and 4 :

RSA 2048 bit key: Amazon Root CA 1
ECC 256 bit key: Amazon Root CA 3

In your case, you have both Root CA 1 & Root CA 3 with names that conveniently point out the cipher suite:

RSA2048AmazonRootCA1.pem (can be downloaded from https://www.amazontrust.com/repository/AmazonRootCA1.pem)
ECC256AmazonRootCA3.pem (can also be downloaded from https://www.amazontrust.com/repository/AmazonRootCA3.pem)

Cipher suites are outside of the scope of this answer but we'll stick to RSA 2048.

So, what do we need for Postman?

We need:

1 certificate authority: RSA2048AmazonRootCA1.pem
device's certificate: certificate.cert.pem
device's private key: private.pem.key

This is how they map in Postman.

Settings > Certificates > CA Certificates

Toggle to ON
Select your root CA (any would work with Postman, but other tools may not always support ECC)

Add Certificate

We have a few values here:

Host:

https://, set by Postman
your AWS IoT Data-ATS endpoint (aws iot describe-endpoint --endpoint-type iot:Data-ATS), set by you e.g. xxx-ats.iot.eu-west-1.amazonaws.com
8443, set by you, as the port - do not use 443

CRT file: select your device's certificate.

Postman is using the .crt filename convention but that's all it is - a filename convention. The format of .crt files are either in DER or PEM. AWS has used .pem here to signal the PEM format i.e. Base64 encoding of the certificate.

Postman fully supports .pem but doesn't currently support DER, so we can just keep this .pem file as it is.

Side note: feel free to add .crt on to the end to have Windows recognise it as a certificate file & allow you to double click on it to check details.

KEY file: select your device's private key
PFX file: leave as-is
Passphrase: leave as-is

Click Add - this should be your final configuration

Postman is now correctly configured to send requests to the HTTPS endpoint.

The reason IoT throws Missing authentication when trying to publish messages via the HTTPS endpoint is that port 443 requires a custom ALPN protocol name of x-amzn-http-ca. You can only communicate over port 443 without a custom ALPN protocol name set, if you use AWS Sig V4 for authentication.

In this case, as we're using X.509 client certificate authentication, port 8443 allows us to not need to configure ALPN. ALPN is impossible to configure within Postman & difficult to configure for curl, so we should opt for either

Port 443, AWS Signature v4 authentication
Port 8443, X.509 certificate authentication

I've opted for certificate authentication.

This is highlighted in the AWS docs, with my emphasis on the bold items:

Protocol	Operations Supported	Authentication	Port	ALPN Protocol Name
MQTT over WebSocket	Publish, Subscribe	Signature Version 4	443	N/A
MQTT over WebSocket	Publish, Subscribe	Custom authentication	443	N/A
MQTT	Publish, Subscribe	X.509 client certificate	443	`x-amzn-mqtt-ca`
MQTT	Publish, Subscribe	X.509 client certificate	8883	N/A
MQTT	Publish, Subscribe	Custom authentication	443	mqtt
HTTPS	Publish only	Signature Version 4	443	N/A
HTTPS	Publish only	X.509 client certificate	443	`x-amzn-http-ca`
HTTPS	Publish only	X.509 client certificate	8443	N/A
HTTPS	Publish only	Custom authentication	443	N/A

Final thing to check is that the policies attached to your certificate allow you to publish to the topic you want to publish to, using that certificate.

At the very least, you need to have the below policy attached with the iot:Publish policy action:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "iot:Publish",
      "Resource": "arn:aws:iot:eu-west-1:xxx:topic/my-topic"
    }
  ]
}

Replace eu-west-1 with your region, xxx with your AWS account ID & my-topic with your topic name.

If you want to be able to publish to any topic, use the * wildcard in the ARN as detailed before (and not #!) e.g.

arn:aws:iot:eu-west-1:xxx:topic/*

Once you have the correct CA, the correct device certificate set up with the host set to the ATS signed data endpoint, the port set to 8443, the KEY file set to your private key and your certifcate has the right policy attached in the console, you're ready to send your request.

Method: POST
URL: https://xxx:8443/topics/yyy?qos=1, replacing xxx with your ATS endpoint and yyy with your topic name
Body: raw - Text/JSON as content type, doesn't really matter
Headers: very important to keep Content-Length otherwise you will get Message cannot be displayed in specified format. in the AWS MQTT test client

Hit send & a 200 OK response signals success:

{
    "message": "OK",
    "traceId": "xxx-xxx-xxx-xxx-xxx"
}

If you haven't set up your policies correctly to allow publishing to the topic, you'll instead get a 403 Forbidden response that signals failure.

Note message is set to null:

{
    "message": null,
    "traceId": "xxx-xxx-xxx-xxx-xxx"
}

If it's successful, check the console & your message will be there:

P.S. the developer guide does indeed have a typo with the rogue ". I've submited feedback to hopefully rectify this :)