Monitoring on Pi Zero W with multiple RuuviTags

alexhk · March 22, 2019, 4:31pm

Hi guys,

I am new here, so apologize if I am a bit behind. I just purchased 6 RuuviTags and hopefully a few more will follow soon. I am not a developer, rather a simple user who wants to monitor temperature, humidity and possibly pressure on multiple locations (within BT range) with multiple RuuviTags.

I plan to order a Pi Zero W with power supply and then use it to manually monitor (ie. look at my screen when I want to know temp & humidity) my RuuviTags and to log the data if possible (ie. I can see what happened during the last few weeks). I will have it plugged in to power and connected to WLAN, so no issues with power etc.

I actually thought that the Ruuvi Station on iOS could do all that, but well it can’t and I am not complaining since I know how restrictive iOS can be…

But just to make sure I am buying the right things, I’d like to confirm if the following is all I need:

Pi Zero W, power supply.
According to the RuuviTag firmware page page, I can use the official Raspberry Pi Image (does that work with the Pi Zero W or do I need to get a Pi 3B+ or 3B for that image?).
I may need to configure RuuviCollector and Grafana to collect/display data for all my 6 RuuviTags (and a few more later), but InfluxDB should already be setup to work with RuuviCollector and Grafana so no need to do any configuratoin for InfluxDB.

Does that sound about right?

Thanks for any pointers.

PS: The Ruuvi Station app on iOS shows up as Ruuvi Sation (t missing in Station) on my phone.

otso · March 23, 2019, 1:10pm

I’m not 100% sure if the image will work out of the box with Zero W, but setting the whole distribution up is not too difficult. The image has been tested on Pi 3B (not 3B+).

icewood · March 23, 2019, 6:03pm

I don’t want to advertise but I created a service that solves exactly this problem. It’s easier to setup and we’ll take care of running and maintaining it.

So if you are looking for easy solution, check out Meazurem. The iOS client is under development and should be released in couple of months.

dgerman · March 24, 2019, 11:54am

I can make an image of the systemI have running available if you like.
It includes RuuviCollector InfluxDB and grafana.
Is your other system MS Windows, Mac OS or linux?
Do you have an ethernet adapter on your PiZeroW?
Can you use a 32GB sd card?

alexhk · March 24, 2019, 3:37pm

Thanks a lot for all your help!

I’d appreciate the image. Yes, I have a 32GB SD card and a USB/Ethernet adapter for my Pi Zero W (although untested so far). Both Linux or Windows is fine.

Thanks again! I will report back either way if it works with your image or if I have to install fresh.

dgerman · March 24, 2019, 4:11pm

standby…

vergil · March 25, 2019, 2:57pm

Hello,
I am in a similar situation like you @alexhk and would like to know, if there are any known issues if I want to connect more ruuvitags, lets say up to 50 pcs to one Pi Zero W?
Many thanks for your answers.

dgerman · March 26, 2019, 12:18am

I (and I imagine many others ) would be really interested in what your application is for using 50 Ruuvis.

dgerman · March 26, 2019, 12:56am

My Pi Zero W image is available at
MyBeacons/RaspRuuvixxxx.gz.

It is exactly 5,040,031,819 bytes. You should expect it to take in the neighborhood of 1 hour to download at 1 MegaBYTE/second .

Use https://www.balena.io/etcher/ to write it to a 32GB SD card (without uncompressing it first). Takes about 40 minutes to write and verify.

Connect via USB keyboard and mouse (alternately since pi Zero will not provide enough power for most USB hubs) and HDMI monitor
or
Use an USB ethernet and connect via ssh from another system.

As pi/raspberry.

Use

sudo raspi-config

to
Change user pi’s password &
Localisation to change timezone &
network options to change the wifi SSID and password AND hostname …
and shutdown/reboot.

It should start collectingRuuvi Tag data using the RuuviCollector written by Scrin and storing the data in an influx database after a few minute delay.

View the data with Grafana by pointing a web browser at hostname:8888.
Mouse over the graph icon on the left , select ruuvi.autogen, ruuvi_measurments and select a MAC. Note that selecting temperature will include the MEAN() and you may not want that.

You can also use influx from the command line using something like:

influx -precision rfc3339 -database ruuvi
select mac,ceil(temperature*9/5)+32 from ruuvi_measurements group by mac limit 5

That should cover it.
As you should expect: this is provided in the hopes it will be useful without any suggestions that it will actually work for you! Neither me nor anybody else can be responsible if there are any problems related to your use of the system.

That being said , please let me know you use it and have any problems or questions.

vergil · March 26, 2019, 1:37pm

@dgerman no secret here: we want to measure temperature around a bigger hall in different containers

vergil · March 27, 2019, 10:02am

Hello again, does anybody have any experience with connecting so many ruuvi tags to one BW gw? Maybe @otso could you please provide some info? Many thanks in advance.

otso · March 27, 2019, 12:24pm

Hello @vergil,

As the data is sent in broadcasts in random interval, there isn’t really a hard limit on how many beacons your receiver can hear. At some point the broadcasts will saturate the bandwidth, and you’ll get increasing packet loss.

I happen to be researching practical limits right now, if you’re interested in previewing the blog please feel free to look at https://blog.ruuvi.com/bluetooth-beacon-density-maximum-92bcb947ee99 and give any feedback on if the approach makes sense. Also please note that it’s a work in progress right now, i.e. not polished.

TLDR version: If you’re broadcasting once per second, you could expect highest throughput of data with 769 tags and have 2/3 packets lost. To test it out I programmed 16 RuuviTags to broadcast at 100 ms interval (equivalent to 160 tags, once second interval) and got 0.7 tx per second out of 10 tx per second to Mac regardless of how many tags were active.

My Raspberry Pi had a code which opened a new connection to backend for each sample and the connectivity could not keep up after a peak of 150 tx / second. I’l try later with Scrins RuuviCollector to see if it would hold up better under the load.

Your case of 50 RuuviTag should be fine by the Bluetooth, but Zero W probably has a shared antenna for Bluetooth and WiFi which might limit the rate of data you’ll get or even crash the collector program if it does not handle undelivered samples gracefully.

vergil · March 28, 2019, 9:52am

Hello @otso,
thank you very much for your insight and link to the blog post. In my case, I will not need the data with such granularity, i.e. sending the temperature once per minute up to once per 5 minutes should be just fine. Therefore also the chance to get to Zero W data rate’s limit should be minimized. And as you suggest in the blog, the best way to find out is to try it in real

alexhk · March 29, 2019, 6:34pm

@dgerman thanks a lot for your help and for the effort.

I tried your image, bit it failed with Etcher. The official OS images for the Raspberry Pi Zero W worked with Etcher, so I went ahead and installed from scratch. It’s all up and running now, logging my RuuviTag data nicely.

I am waiting for my second 32GB SD card to be delivered, so I will try your image again in a few days and report back (out of curiosity, if it was a one time problem or if the image is corrupt).

I’ll create a separate post to show what and how I installed it.

alexhk · March 29, 2019, 6:47pm

So for everyone who is interested, I setup the Raspberry Pi Zero W headless with Raspbian Stretch Lite, InfluxDB, Grafana, and RuuviCollector. This was done more or less according to the guide posted at https://blog.ruuvi.com/rpi-gateway-6e4a5b676510 which doesn’t work for the Raspberry Pi Zero W, so I had to make some changes.

Long story short, I followed the guide above but when it came to Grafana it didn’t work with the Rasbperry Pi Zero W, because the official binary is an armv7 binary while the Pi Zero W needs an armv6 binary.

Here is what I had to run/install after I connected to the Pi Zero W via SSH:

wget https://dl.influxdata.com/influxdb/releases/influxdb_1.7.5_armhf.deb
sudo dpkg -i influxdb_1.7.5_armhf.deb

sudo systemctl start influxd
systemctl status influxd

influx
CREATE DATABASE ruuvi;

At this point I found that the official Grafana armhf binary doesn’t work on the Zero W and I found a compatible armv6 binary in this thread on Github: https://github.com/grafana/grafana/issues/15695

wget https://raw.githubusercontent.com/trashware/grafana-rpi-zero/master/grafana_6.0.1_armhf.deb
sudo apt-get install adduser libfontconfig
sudo dpkg -i grafana_6.0.1_armhf.deb

sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable grafana-server
sudo /bin/systemctl start grafana-server
sudo /bin/systemctl status grafana-server

sudo apt-get install bluez-hcidump

sudo setcap 'cap_net_raw,cap_net_admin+eip' `which hcitool`
sudo setcap 'cap_net_raw,cap_net_admin+eip' `which hcidump`

sudo apt-get install screen

sudo apt-get install openjdk-8-jre-headless

screen
java -jar ruuvi-collector-0.2.jar

Now I logged in to Garafana’s webUI and setup the RuuviTags.

dgerman · March 29, 2019, 6:50pm

I downloaded mybeacons.info/RaspRuuvixxxx.gz and Etchered it before you did and it worked without problems. Did your download get a 5040031819 byte file?

Glad to hear you got it working the “hard” way.
What kind of containers are you monitoring?

PS you may be interested in my findings mostly related to starting/rebooting:

influxdb takes a while to get ready to accept transactions (especially after a powerfail)
I have found it helpful to add this to RuuviCollectorStart.sh just before java -jar target/ruuvi-collector-*.jar

for i in 240 220 200 180 160 140 120 100 80 60 40 20
do
logger " ++ RuuviCollector sleeping $i more seconds to give influx a start."
sleep 20
done

To slow down influxdb restarts after failures add this to service section of /etc/systemd/system/influxd.service

RestartSec=600

alexhk · March 29, 2019, 6:56pm

I have 5,040,031,819 bytes, so that shouldn’t be the problem. I am trying again soon with Etcher (just made a backup of my only 32GB SD card which I need to test anyway).

I am not monitoring containers, that was another user.

I am just dropping a few RuuviTags in different rooms and front and back yard for now. Going to add a few more now that I have a working system.

Btw, do you autostart your RuuviCollector?

I now used screen and start RuuviCollector manually. It’s been ages since I setup screen to start at boot.

dgerman · March 29, 2019, 7:15pm

I have

/etc/systemd/system > ls -l ruuvicollector.service.requires/

lrwxrwxrwx 1 root root 36 Dec 30 21:44 influxdb.service -> /lib/systemd/system/influxdb.service

and

/lib/systemd/system/ruuvicollector.service

[Unit]
Description=RuuviCollector
After=network.target
[Service]
ExecStart=/home/pi/RuuviCollector/RuuviCollector
WorkingDirectory=/home/pi/RuuviCollector
StandardOutput=inherit
StandardError=inherit
Restart=always
#DGG added RestartSec=
RestartSec=300
User=pi
[Install]
WantedBy=multi-user.target
# # depends on influxdb

I also am enamored with syslog so my (May 26’19 version) RuuviCollector script is

 #!/bin/bash
 #                           link RuuviCollector, so systemctl and syslog talk nice:   ln RuuviCollectorStart.sh RuuviCollector
 #  sudo systemctl start ruuvicollector
 cd "${0%/*}"; pwd
 logger -p user.notice " ++ /lib/systemd/system/ruuvicollector.service:ExecStart=/home/pi/RuuviCollector/start.sh pwd=$PWD"

 # To disable RuuviCollector:  edit this script:  change  echo and logger to say NOT start and uncomment out the exit 0
  #                             then  sudo systemctl stop ruuvicollector
  echo                    ++ Did     start /home/pi/RuuviCollector/start.sh
  logger -p user.notice ' ++ Did     start /home/pi/RuuviCollector/start.sh'
  #exit 0
 
 # This may take  27 seconds if influxdb is running #  1/14/19 took 11 minutes before influx was ready accept a connection
 # This takes 19 seconds to timeout if influxdb is not even running  $? = 1 ALL CPU BOUND!!
 # If influxdb is not ready influx outputs:
 #  Failed to connect to http://localhost:8086: Get http://localhost:8086/ping: dial tcp [::1]:8086: connect: connection refused
 #  Please check your connection settings and ensure 'influxd' is running.
 
 ############################### wait until influxdb is ready  
 for(( i=1; i<30; i++ )) # wait  19+5 * 30  = 12 minutes  before giving up
 do
 
 /home/pi/RuuviCollector/startupQuery.sh
 rc=$?
 
 if [ $rc -eq 0 ] ; then logger --priority user.info "++ ruuvicollector: influxdb now accepting transactions"; break; # loop and go
      else               logger --priority user.info "++ ruuvicollector waited for influxdb $i times, rc=$rc and still waiting"
                          sleep 5
 fi
 
 done
 if [ $rc -ne 0 ] ; then  logger --priority user.emerg "++ ruuvicollector gave up waiting for influxdb"; exit $rc; fi
 #############################################
  #  Since  sudo systemctl stop ruuvicollector issues  TERM ; trap that here to generate log entry
  trap 'logger --priority user.alert " ++ /home/pi/RuuviCollector got sig TERM" ;exit 15' TERM # will exit with x'80' + 15 i.e.143
  trap 'logger --priority user.emerg " ++ /home/pi/RuuviCollector got sig QUIT" ;exit  3' QUIT
 
  java -jar target/ruuvi-collector-*.jar
 
 logger -p user.notice " ++ /home/pi/RuuviCollector/start.sh exited with rc=$?"

and
startupQuery.sh contains
echo 'select last(temperature) from ruuvi_measurements' | influx -database ruuvi

my /etc/rsyslog.conf includes (among many other things)

 user.*          @dalogger.dyndns.org
 user.*           -/var/log/user.log

dalogger.dyndns.org is a raspberrypi I use to consolidate all my logs.

alexhk · March 29, 2019, 7:38pm

Thanks a lot, that will be very helpful!

Etcher definitely fails with the image you uploaded. Possibly the Etcher version?! In any case, I am going to restore my own image/backup and will have a look at your startup routine tomorrow morning (it’s getting late over here).

Strangely, Etcher first counts to 100% within the first 2 minutes of writing the image, then it switches to display the GB in the “progress bar” and at exactly 32GB it fails.

Still with us:

alexhk · March 29, 2019, 7:39pm

(new users can only put 1 image per post)

Now at 32GB it failed: