Proposed next high-precision Data Format

Thanks, Iā€™ll adjust the voltage and make packet counter uint32_t just in case.
Do you already have settings for lis2dh12 interrupts? I was thinking about using interrupt 2 with high-pass filtering, maybe 0.1 G, both up and down.

uint32_t should be sufficient, based on my quick calculation, youā€™d have to run with 60ms interval for 8 years without reboots to overflow the counter. 16bit would overflow in less than 2 hours at 100ms interval. :stuck_out_tongue: I havenā€™t gotten that far with my adventures to the world of LIS2DH12 interrupts yet, so I donā€™t have anything usable yet.

Hereā€™s a compiled, but not tested package:


Source:

Itā€™s based on merge-drivers pull request. Changes:

  • BME280.h: Added bme280_environmental_t struct which contains raw values
  • LIS2DH12 driver: Added support for 2nd interrupt pin
  • sensortag.c/h added support for new format
    *main.c Configure interrupt, call updated format.

Raw data as sent by tag:
ADVERTISEMENT
(flags) 0x02 01 06
(data) 0x1B FF 99 04 05 13 7E 45 0A C3 C3 00 84 00 70 04 14 9D F6 01 00 BC FB 01 7A 3A 64 E9
Translates to:
Length 27 (base 10)
Type FF manufacturer specific
Manufacturer ID 0499 (Ruuvi)
Format 05 (Raw v2)
Temperature 137E (4990, * 0.005 = 24,95) // Ok
Humidity 450A (17675, * 0.0025 = 44,1875) // Most of my tags show 30 - 40, but maybe room Iā€™m in is more humid?
Pressure C3C3 (50115, + 50000 = 100115) // Fits well the other tags
X-Y-Z acceleration 0084 0070 0414 // Bit over 1G, I have to check the scaling
Battery 9DF6 (04EF, 1263 + 1600 = 2863) // Ok
TX Power F6 (16, 22 *2 -40 = 4) // ok
Activity events 01 // Was hardcoded, fix in 2.1.1
Sequence number 00BC // Could be
ID FB 01 7A 3A 64 E9 // MAC is EA 17 E9 64 3A 7A, fix in 2.1.1

SCAN RESPONSE
(length) 03
data 0x19 00 00
type 0x19 (appearance)
value 00 00 // Generic

(length) 0x0A
(data) 0x09 52 75 75 76 69 29 35 38 63 37
type 09 //Full name
Value 52 75 75 76 69 29 35 38 63 37 // Ruuvi 58c7

Iā€™ll post again once fixes above are done

p.s.
Accelerometer config:

//Clear memory
lis2dh12_reset();

//Wait for reboot
nrf_delay_ms(10);

//Enable XYZ axes
lis2dh12_enable();

lis2dh12_set_scale(LIS2DH12_SCALE2G);

//Sample rate 10 for activity detection
lis2dh12_set_sample_rate(LIS2DH12_RATE_10);
lis2dh12_set_resolution(LIS2DH12_RES10BIT);

//XXX If you read this, I'm sorry about line below.
#include "lis2dh12_registers.h"

//Configure activity interrupt - TODO: Implement in driver, add tests.
uint8_t ctrl[1];

//Enable high-pass for Interrupt function 2
//CTRLREG2 = 0x02
ctrl[0] = LIS2DH12_HPIS2_MASK;
lis2dh12_write_register(LIS2DH12_CTRL_REG2, ctrl, 1);

//Enable interrupt 2 on X-Y-Z HI/LO
//INT2_CFG = 0x7F
ctrl[0] = LIS2DH12_HPIS2_MASK;
lis2dh12_write_register(LIS2DH12_INT2_CFG, ctrl, 1);    
//Interrupt on 64 mg+ (highpassed, +/-)
//INT2_THS= 0x04 // 4 LSB = 64 mg @2G scale
ctrl[0] = 0x04;
lis2dh12_write_register(LIS2DH12_INT2_THS, ctrl, 1);
    
//Enable LOTOHI interrupt on nRF52    
err_code |= pin_interrupt_enable(INT_ACC2_PIN, NRF_GPIOTE_POLARITY_LOTOHI, lis2dh12_int2_handler);

//Enable Interrupt function 2 on LIS interrupt pin 2 (stays high for 1/ODR)
lis2dh12_set_interrupts(LIS2DH12_I2C_INT2_MASK, 2);

Looks very promising. Iā€™ll test this as well when I get home

2.1.1 Has acceleration and MAC address fixed, sources & release are at Github
[EDIT] 2.1.2 has bugfix in accelerometer config

To sum up latest discussions etc from Slack, the latest proposal would be as follows:

offset = description
0      = Data format (8bit) whatever the next number will be (likely 5)
1-2    = Temperature (16bit signed) in 0.005 degrees (-163.84 to 163.83 range)
3-4    = Humidity (16bit unsigned) in 0.0025% (0-163.83% range, though realistically 0-100%)
5-6    = Pressure (16bit unsigned) as it is in format 3
7-8    = Acceleration-X (16bit signed) as it is in format 3
9-10   = Acceleration-Y (16bit signed) as it is in format 3
11-12  = Acceleration-Z (16bit signed) as it is in format 3
13-14  = Power info (11+5bit unsigned), first 11bits unsigned is the battery voltage above 1.6V, in millivolts (1.6V to 3.647V range). last 5 bits unsigned is the TX power above -40dBm, in 2dBm steps. (-40dBm to +20dBm range)
15     = Movement counter (8bit unsigned), incremented by motion detection interrupts from LIS2DH12
16-17  = Measurement sequence number (16bit unsigned), each time a measurement is taken, this is incremented by one, used for measurement de-duplication (depending on the transmit interval, multiple packets with the same measurements can be sent, and there may be measurements that never were sent)
18-23  = Tag ID, the 48bit MAC address (rather than the hw ID on the nRF chip to provide compatibility with current datasets that differentiate tags based on the MAC)

Most notable was replacing packet sequence number with measurement sequence number, as itā€™s more relevant for measurement de-duplication.

2 Likes

@io53 Do you have any feedback on the format?

FYI The following is a response I got back from bosh-sensortech.com in response to my question: from the data sheet AH is +/- 3% in the range of 20% - 80% . What is the accuracy above 80%?

From the factory:

According to our characterization, the absolute accuracy is also +/-3%RH from 80%RH ~ 90%RH range and within +/-6%RH from 90%RH ~ 100%RH.
The raw data from BME280 is 16-bit unsigned integer named as UH. After the formula based on factory trimmed values and final temperature (FT) reading, UH is converted to final humidity value FH. Every digit change in UH, will cause about 0.008%RH change in FH. This is where the 0.008%RH resolution parameter comes from.

reminder. shutdown occurs at 1.7V

So using a 16bit unsigned int makes sense in the Data format for humidity. Regarding the reported voltage, I think the measured shutdown voltage should be tested, as based on my prior measurements, the tag reports about 0.1V less voltage than the battery actually supplies, so 1.6V minimum in the format might actually be too high, if the tag can go down to 1.7V real. :thinking:

@otso do I remember correctly that you have an adjustable lab power supply?

When this version is official?

Weā€™ll continue to ship the tags with current format for the time being. Updated format will be offered as DFU package.

Should there be a field for firmware version? (not packet format) Hardware version?

You might consider using format type 4 (not to be confused with ā€˜#4ā€™ used by the URI which includes the single ID byte)

I think that kind of rarely/never changing info (firmware version and hardware version) would belong to the ā€œscan responseā€ along with a friendly name and other things (to be implemented in the future).

Also, the data format id does not directly differentiate between URL and RAW modes, you can very well send formats 2 and 4 as manufacturer-specific data, or format 3 in an Eddystone URL if you use a short enough domain. This allows the actual data to be sent in many different formats, in different ways (encoded in an Eddystone URL, sent as manufacturer-specific data in a undirected BLE advertisement, or sent as a scan response, or even sent along a ā€œtraditionalā€ directed communication)

1 Like

Format 3 uses 14 bytes which encodes to 19 characters which exceeds the ā€œencoded URLā€ length without the domain name.

Correct, for Base 64. Nobody says you have to use Base 64 :slight_smile: Base 92 can fit 13 bytes of data per 18 characters, and you could as well come up with your own encoding

Just a note, researching battery information I found " Service lifeā€¦discharge end-point voltage of 2.0V"
CR2477 data sheet

Also ā€œcut off voltage down to 2.0V at 20Ā°Cā€ at lithium coin batteries

I think repeating the MAC address is redundant and a big waste in such an environment. Apple might fix their software one day to reveal the MAC address from the Bluetooth advertisement? The chip ID could be more useful. People using buggy software stacks can still use this field for identification. And then the chip ID could also be correlated between raw and URL modes.

There could be space for firmware version and other stuff if you encoded bits at non-byte boundaries. Eg. you can chip off 2-3 bits from format id, humidity and each acceleration field. I see how it seems difficult for beginners, so for each format version I would suggest publishing a reference decoder library in a couple of languages. This could be community contributed even.