Spikes in data collected with rpi3

[Andre_Norrback]

I have two different raspi3s in different locations, 100m appart in different buildings. Ruuvi Collector on both and they are collecting to my “own cloud” where I have InfluxDB and grafana.

To the problem, I have spikes in data from one of my two raspis. I have 5 tags in one location and one tag in the other location. I never have any type of spikes in data at my home location (5 tags) but on the other location (1 tag) I always have spikes in data. If I take one tag from my home location and put it in the other location both get spikes, so the tag itself is not the problem its the raspi and collector.

In this picture I changed the location on garaget tag to the other location at the black line and as you see, after an hour I have spikes in both.

There are one difference in the RPI setups, the one at home has the ruuvi image, the other pi (problem pi) is made according to this guide: https://github.com/bostrom/ruuvi-on-aws/ @bostrom

Anyone having similar problems? Any ideas how to fix this issue?

/Andre’

[ejk]

I have the same issue with a ruuviberry image updated to “Raspbian GNU/Linux 9.9 (stretch)”.
But the one at “Raspbian GNU/Linux 9.3 (stretch)” does not show those “spikes”.
For example battery voltage values can be as large as 64.438 volts, which is impossible considering that the nominal battery voltage of CR2477 is three (3) volts.
IMHO newer the Raspbian, more there are errors in the readings.

[Andre_Norrback]

From another post where similar problems where found.

Outo lukema oudolta RuuviTagilta

Mulla on joskus tullut virheellisiä lukemia Raspberry Pin kanssa kun käytän kahta bluetoothia käyttävää ohjelmaa yhtäaikaa. Bluetooth-paketeissa itsessään pitäisi olla jokin CRC-tarkistus, mutta ehkä virheellinen paketti silti pääsee jostain kohtaa ketjua läpi jossain tilanteessa

in english, otso believes that there could be several programs running that trying to use BT on the RPI and that could in some way create the bad data.

how can I be sure I don’t have several apps using BT, so I could eliminate that possibility for error? I’m no linux guru, in other words no ide how to check that from the console.

/Andre’

[dgerman]

Do you have the most recent version of ruuvi collector?

Scrin/RuuviCollector

v0.2.5

Scrin released this on Jun 26

• Improved error tolerance while parsing for raw data

Closed Issue#23 # Provide better logging on unexpected behavior

Issue: Provide better logging on unexpected behavior

++++++++
Although I frequently run multiple hcidump commands I have not seen this problem.

ps -el|grep hci # display any processes running Blue Tooth stuff (hopefully)

Can you find any errors in /var/log ?

grep ' RuuviCollector' /var/log/*log |tail -100

++++++
What is you skill level? linux, influx
++++++
Perhaps displaying the exact data stored in the influx database might provide some clues:
influx -precision rfc3339 -database ruuvi -execute "select mac,temperature from ruuvi_measurements where temperature < 20 group by mac limit 50"

[ejk]

I did some digging around and ended up here. I dropped the bluetooth uart speed to half, and it worked for me, e.g. no bluetooth errors in dmesg output in the last eight hours, and the spikes have gone from Grafana.

$ sudo -i
# cp -pi /usr/bin/btuart /usr/bin/btuart.orig
# sed -e 's/921600/460800/g' < /usr/bin/btuart.orig > /usr/bin/btuart
# diff -u /usr/bin/btuart.orig /usr/bin/btuart
--- /usr/bin/btuart.orig        2018-10-29 14:02:08.000000000 +0200
+++ /usr/bin/btuart     2019-08-09 22:12:36.026493166 +0300
@@ -15,7 +15,7 @@
        if [ "$uart0_pins" = "16" ] ; then
                $HCIATTACH /dev/serial1 bcm43xx 3000000 flow - $BDADDR
        else
-               $HCIATTACH /dev/serial1 bcm43xx 921600 noflow - $BDADDR
+               $HCIATTACH /dev/serial1 bcm43xx 460800 noflow - $BDADDR
        fi
 else
        $HCIATTACH /dev/serial1 bcm43xx 460800 noflow - $BDADDR
# shutdown -r now

[Andre_Norrback]

great, I made these changes this morning, let’s see how it works out. I’ll let you know.

[Andre_Norrback]

no spikes since the change. Thanks!

[Andre_Norrback]

Just to follow up, no spikes so far after the change.

[ejk]

That’s excellent news. FWIW I made this shell script to cleanup the spikes or invalid values, because I wanted to get rid of the spikes.

1. battery voltage more than 3,31 volts
2. battery voltage less than 1,1 volts
3. air pressure is lower than 90 kPa,. The lowest air pressure measured was 87,70747 kPa and the highest was 108,3983 kPa. Source, Encyclopaedia Britannica

Each measurement is copied to “broken_ruuvi_measurements” before it is removed from “ruuvi_measurements”.

#!/bin/sh
timestamps=$(mktemp /tmp/XXXXXXXXXXXXXXXXXXXX)

printf '\nToo high battery voltages\n'
influx -database 'ruuvi' -execute 'select time,batteryVoltage from ruuvi_measurements where batteryVoltage > 3.31' -format 'csv'|tee -a "${timestamps}"

printf '\nToo low battery voltages\n'
influx -database 'ruuvi' -execute 'select time,batteryVoltage from ruuvi_measurements where batteryVoltage < 1.1' -format 'csv'|tee -a "${timestamps}"

printf '\nPressure too low\n'
influx -database 'ruuvi' -execute 'select time,pressure from ruuvi_measurements where pressure < 90000' -format 'csv'|tee -a "${timestamps}"

for timestamp in $(awk -F',' '/^ruuvi_measurements,/ {print $2}' "${timestamps}"|sort|uniq)
do
        influx -database 'ruuvi' -execute "SELECT * INTO broken_ruuvi_measurements FROM ruuvi_measurements WHERE time = ${timestamp} GROUP BY *" 
        influx -database 'ruuvi' -execute "delete from ruuvi_measurements where time =  ${timestamp}" 
done

rm "${timestamps}"

There are now two measurements in “ruuvi” database:

$ influx
Connected to http://localhost:8086 version 1.7.7
InfluxDB shell version: 1.7.7
Enter an InfluxQL query
> use ruuvi
Using database ruuvi
> show measurements
name: measurements
name
----
broken_ruuvi_measurements
ruuvi_measurements
> quit
$