Wildcard topic names and capturing message from all topics

Question

We are using AWS IoT.

We have predefined topics, (+/device/), where the devices publish the messages.

But there is a possibility that, the devices can publish the messages to any other topics.

I want to calculate the number of messages which are published to all the topics, by these individual devices and implement throttling.

I tried to create IoT rules using wildcard topic names like ( +/* or /), but none of these wildcard topics seem to work.

Is there any wildcard topic name, which I can use to capture the messages from all the topics?

Or is there any way to dump all the messages on all the topics somewhere in DynamoDB or S3 and calculate the number of messages from individual devices in a specific time period?

Answer 1

I tried to create IoT rules using wildcard topic names like ( +/* or /), but none of these wildcard topics seem to work.

Is there any wildcard topic name, which I can use to capture the messages from all the topics?

+ and # are the relevant wildcards for AWS IoT rules. See https://docs.aws.amazon.com/iot/latest/developerguide/iot-sql-from.html

You can configure a rule with the following statement to capture messages from all topics.

SELECT * FROM '#'

Or is there any way to dump all the messages on all the topics somewhere in DynamoDB or S3 and calculate the number of messages from individual devices in a specific time period?

One approach is to create a rule based on the one above and also pass the client ID on every message (using the clientid() function). The action for this rule could write the client Id to DynamoDB or S3. Then this information is available to do your calculation.

An alternate approach might be to write the messages and clientid to a Kinesis Data Stream and use Kinesis Data Analytics to detect the errant devices.