Ruuvi Gateway to Mosquitto MQTT problem

Iā€™m trying to connect a Ruuvi Gateway to a Mosquitto MQTT Broker (mosquitto 2.0.11 on Ubuntu 22.04 LTS) and not having a lot of success.

I have included the configuration settings I have used for both ends below. Please let me know if there are any obvious errors. I have limited familiarity with this area.

I have used ā€˜openssl s_client -verify_hostnameā€™ from another system which suggests that the server certificate is probably not the problem.

However, this line from the putty log (extract below) looks concerning:
MQTT_CLIENT: Certificate bundle feature is not available in IDF version v4.2.2

Thanks for any help you can offer

John

Ruuvi Gateway Configuration
Select language: English
Select connection type: Ethernet
IP address configuration: DHCP
Current firmware version: v1.11.2
Auto update: true
Password protected with the default password (default, safe)
Use a custom server and configure more settings: true
MQTT: true
MQTT transport type: MQTT over SSL
Server: <FQDN>
Port: 8883
User name: mosquitto
Password: <secret>
Client name: AA:BB:CC:DD:EE:FF
Topic prefix: ruuvi/AA:BB:CC:DD:EE:FF/SENSOR_MAC_ADDRESS>
Use ā€˜ruuviā€™ on the prefix: true
Use Ruuvi Gatewayā€™s MAC address on the topic prefix: true
Use a custom topic prefix: false
Allow sending statistics: false
Bluetooth Scanning: All (including third party beacons)
Use coded PHY (long range): true
Use 1 MBit/s PHY: true
Use extended payload: true
Use channel 37: true
Use channel 38: true
Use channel 39: true

/etc/mosquitto/mosquitto.conf
pid_file /run/mosquitto/mosquitto.pid
persistence true
persistence_location /var/lib/mosquitto/
per_listener_settings true
log_type all
log_dest file /var/log/mosquitto/mosquitto.log
include_dir /etc/mosquitto/conf.d

/etc/mosquitto/conf.d/localhost
listener 1883
protocol mqtt
bind_interface lo
allow_anonymous true

/etc/mosquitto/conf.d/ruuvigateway.conf
protocol mqtt
listener 8883
bind_interface eno1
require_certificate false
capath /etc/ssl/certs
certfile /etc/mosquitto/certs/fullchain.pem
keyfile /etc/mosquitto/certs/privkey.pem
password_file /etc/mosquitto/passwd

/etc/mosquitto/passwd
mosquitto:<secret>

$ openssl s_client -verify_hostname <FQDN> -connect <local IP>:8883
[ā€¦]
Verification: OK
Verified peername: <FQDN>
[ā€¦]
$ mosquitto_sub -h localhost -p 1883 -t ā€˜ruuvi/#ā€™
{ā€œstateā€: ā€œonlineā€}
{ā€œstateā€: ā€œonlineā€}

putty.log
[ā€¦]
(2301746) ADV_POST_TASK: [rx_parse_task] Canā€™t send, MQTT is not connected yet
(2301766) ADV_POST_TASK: [rx_parse_task] Canā€™t send, MQTT is not connected yet
(2301767) MQTT: [mqtt_task] MQTT_EVENT_BEFORE_CONNECT
(2301769) MQTT_CLIENT: Certificate bundle feature is not available in IDF version v4.2.2
(2301782) esp-tls-mbedtls: No server verification option set in esp_tls_cfg_t structure. Check esp_tls API reference
(2301789) esp-tls-mbedtls: Failed to set client configurations
(2301796) esp-tls: create_ssl_handle failed
(2301802) esp-tls: Failed to open new connection
(2301806) TRANS_SSL: Failed to open a new connection
(2301812) MQTT_CLIENT: Error transport connect
(2301817) MQTT: [mqtt_task] MQTT_EVENT_ERROR
(2301823) MQTT: [mqtt_task] MQTT_EVENT_DISCONNECTED
(2301828) LEDS: [mqtt_task] leds_indication_network_no_connection
(2301835) LEDS: [mqtt_task] LED: Start blinking, interval: 200 ms, duty cycle: 50%
(2301846) ADV_POST_TASK: [rx_parse_task] Adv report table full, adv dropped
(2301851) ADV_POST_TASK: [rx_parse_task] Canā€™t send, MQTT is not connected yet
[ā€¦]

Found one problem with my configuration! The password file entries needed to be hashed. Fixed that and can successfully run the following:
mosquitto_pub -h <FQDN> -p 8883 --capath /etc/ssl/certs -t "ruuvi/AA:BB:CC:DD:EE:FF/01:02:03:04:05:06" -m "{\"value1\":20,\"value2\":40}" -d -u mosquitto -P "<secret>"
Ruuvi Gateway still not connecting ā€¦

Thought that perhaps fullchain.pem might have been the source of the problem. Split it into cert.pem and chain.pem and edited the configuration file to:

/etc/mosquitto/conf.d/ruuvigateway.conf
protocol mqtt
listener 8883
bind_interface eno1
require_certificate false
cafile /etc/mosquitto/certs/chain.pem
certfile /etc/mosquitto/certs/cert.pem
keyfile /etc/mosquitto/certs/privkey.pem
password_file /etc/mosquitto/passwd

Still works with mosquitto_pub. Also works with ā€™ --capath /etc/ssl/certsā€™ option removed.

Ruuvi Gateway still not connecting ā€¦

I see a similar problem was reported as V1.10.0 Failed to set client configurations.

In that thread the question How have you created the SSL certificate for your MQTT server? was asked. In my case the certificate was created using Letā€™s Encrypt. This results in two files: fullchain.pem and privkey.pem. I have further simplified the mosquitto configuration file to:

/etc/mosquitto/conf.d/ruuvigateway.conf
protocol mqtt
listener 8883
bind_interface eno1
certfile /etc/mosquitto/certs/fullchain.pem
keyfile /etc/mosquitto/certs/privkey.pem
password_file /etc/mosquitto/passwd

Which works using mosquitto_pub with no --capath or --cafile options.

Hello,

We have had some issues with MQTT SSL certificates, If I recall correctly the issue was fixed in v1.11.1 with ESP-IDF SDK update but we had to revert the SDK update in v1.11.2 due to issue with radio amplifier control introduced in new SDK version.

This should be fixed for good in v1.12 versions, if you want you can try to manually update to v1.12.1 by selecting ā€œspecify URL manuallyā€ in update screen and putting ā€œv1.12.1ā€ in place of the ā€œv1.11.2ā€ shown as default.

We have some issues in v1.12.1 because of which we wonā€™t roll out it to everyone, automatic update for v1.12.2 will be available later

Many thanks. Managed to transfer data successfully using v1.12.1 as suggested. Much less chatter in putty attached to the USB, too.

Do you have any plans to offer client certificate support as an alternative to username/password with secure MQTT?

I observed that port 80 is open on the Ethernet. Is there anything available on it during normal operations?

Thanks again. this should be sufficient for testing until v1.12.2 becomes available.

Yes, you can track the development at Support user SSL certs Ā· Issue #476 Ā· ruuvi/ruuvi.gateway_esp.c Ā· GitHub. Please add a comment if you have some specific details youā€™d like to see in your mind.