MQTT topic name cannot start with $-sign?

Products & Services 🇬🇧 Ruuvi Gateway Router
1

I have tried to configure Gateway to send MQTT messages into AWS cloud and I finally figured out working setup with IoT Core service but I would like to avoid big bills similar like in this thread. AWS pricing in preferred region is $1.20 / million messages / month and with 0.78 Hz sending rate it would be 2 021 760 messages and about $2.43 per month per sensor.

This is when I use MQTT broker and it works like a charm but if I only need one subscriber (which is now true), I can bypass MQTT broker and avoid messaging costs completely with “Basic Ingest”. That would be achieved by sending messages to topic with “$aws/rules” prefix, but when I add dollar sign into topic name, my Lambda authorizer function doesn’t run anymore, so I think that Gateway doesn’t event try to send messages.

When I try to remove dollar sign from topic name, authorizer function will run again and I will receive correct data through expensive MQTT broker. Is my hunch correct and is it possible to configure Gateway to use topic name which starts with dollar sign?

2

Thank you for the details.

@Alexey Does prefix with special characters work?

3

Hi @TommiFore,
Ruuvi Gateway sends data with MQTT topics started from ‘$’ correctly, but the problem might be on the client side. Since topics started from the ‘$’ interpreted as control messages, they can be filtered out by a server or clients and you need to specify the topic prefix to display it (in ‘mosquitto_sub’).
For example, I configured my gateway to use the custom prefix ‘$aws/rules/my_rule/C8:25:2D:8E:9C:2C’:

image
image991×555 41.8 KB

I checked the network traffic to make sure that the data sent from the gateway was correct:

image
image2280×888 441 KB

But you won’t get any data if you try to subscribe to all message with the following command:

mosquitto_sub -h 192.168.1.119 -p 1883 -t "#" -v

You need to specify the topic prefix:

mosquitto_sub -h 192.168.1.119 -p 1883 -t "\$aws/#" -v

or

mosquitto_sub -h 192.168.1.119 -p 1883 -t "\$aws/rules/my_rule/C8:25:2D:8E:9C:2C/#" -v

Also, please look at this issue MQTT drops messages when topic starts with '$' ¡ Issue #2884 ¡ node-red/node-red ¡ GitHub, maybe it is related to your problem.

1 Like
4

Great, thanks for your big effort @Alexey

I configured my Gateway exactly like you with no success. That is some special case in AWS, which triggers the IoT rule directly without need of explicit subscribing. IoT rule is like action after message has been received. It’s harder to debug because you can’t subscribe to those topics with MQTT test client in AWS console, but you can send test messages to those topics and it’s triggering my rule correctly.

Because the authorizer function doesn’t run at all, I would think that it has to be related to connectivity issues. Maybe I need to use some other endpoint URL with those $-prefixed topics. I will continue to investigate documentation and try to solve the issue, which has to be in AWS side. I didn’t find working example of this scenario anywhere, but I will get back to forum to tell if I can solve the problem. :slightly_smiling_face:

And I almost forgotten to thank you guys great products!

5

Now I can give some extra light to this issue. I tried all given URL possibilities with both encrypted connection methods in Gateway: MQTT over SSL and MQTT over Secure Websocket. There is nothing new to me, only combination that works is MQTT over Secure Websocket, my IoT endpoint’s hostname without any extra paths and port 443.

I will assume that you cannot use X.509 client certificate with Gateway while it’s preferred method in AWS side to detect which client is connecting. Without certificate you have to use custom authorizer (= Lambda function, which can restrict by source IP, client ID etc.) and port 443. I think that HTTPS port with MQTT control messages ($-prefixed) might be the issue…

There is a piece of the IoT debug log, where I first have same topic name without $-prefix and messages are published to my MQTT test client in AWS console, because I had subscribed for them. These messages didn’t trigger my IoT rule “Ruuvi”, because it’s configured only for topics “Gateway/#”. When I am reconfiguring my Gateway by adding $-prefix, it published offline and online states accordingly. I don’t know why $-prefixed message was marked with log level error and why it doesn’t trigger my rule.

But then for two messages, my rule matched and executed correctly. When I have Lambda function to print out event, there is proper payload from Gateway. But after those nothing happens, so I’m thinking that Gateway “accidentally” sent those first two messages into $-prefixed topic but it doesn’t like $-prefixed “control messages” targeted into port 443. Is it possible reason or what the heck could cause this issue? Do you have any thoughts @Alexey on this?

EDIT: I searched online and found software MQTTX to publish messages straight into AWS IoT Core. Now I can confirm that any message published into $aws/rules/Ruuvi/ prefixed topic, triggers my IoT rule no matter what. Full connection string can be wss://MY_IOT_ENDPOINT_HOSTNAME/(mqtt):443 so it works with both paths / and /mqtt but only with wss protocol and port 443. According to my investigations my opinion is that Gateway does not send those messages.