Query stored data on adafruit.io by time

Moderators: adafruit_support_bill, adafruit

Forum rules
If you're posting code, please make sure your code does not include your Adafruit IO Active Key or WiFi network credentials.
Locked
User avatar
cweiss
 
Posts: 6
Joined: Sat Oct 20, 2018 4:01 pm

Query stored data on adafruit.io by time

Post by cweiss »

Hi there!

I am looking to query the last 24 hours of data from a feed. Im using arduino language and can successfully publish and subscribe via MQTT and adafruitmqtt library.

My goal is to retrieve the last x data values within the 24 hours from my feed via mqtt, but I'm not sure if this is possible. The data points are json strings that are time stamped. Once I get them back into my program, I will split the json and sum values (an interesting problem for later).

I see in the documentation that there is "Get activity by type" and query parameters such as start and end time. I don't understand how to implement it though.

Currently, I am subscribed and retrieve the last value each time and save it to download:

Code: Select all

//from mqtt_2_subs_esp8266 example:

  Adafruit_MQTT_Subscribe *subscription;
  while ((subscription = mqtt.readSubscription(5000))) {
    // Check if its the onoff button feed
    if (subscription == &onoffbutton) {

    download = (char *)onoffbutton.lastread;
If I could get a code snippet or guidance that makes download have the last 24 hours of values instead of lastread that would be perfect.

Thanks!

User avatar
abachman
 
Posts: 352
Joined: Mon Feb 01, 2010 12:48 pm

Re: Query stored data on adafruit.io by time

Post by abachman »

Hi cweiss!


The API endpoint you're looking for is called "Get Feed Data" in our documentation. The "Activity" records are things like, "you created a feed" or "you removed a block from a dashboard". Not particularly interesting in terms of collected data.

I had to write a script last week to do pretty much exactly the same thing you're trying to do, I've posted it on GitHub here: https://gist.github.com/abachman/12df0b ... f6b9695539. It works with Python 3 and only uses the standard library. Near the top of the file are some configuration values you'll need to set up to make it work on your computer with your Adafruit IO account. Everything that's ALL_CAPITAL_LETTERS is intended to be modified before you run the sketch.

Because Adafruit IO only returns 1000 records at a time, we have a pagination system in place for letting HTTP clients know that there's more data available when you make a request that has more than 1000 data points in it. For example, 24 hours of data on a feed that you publish to once per minute would be about 1440 values, or two HTTP requests. If your initial request has start_time and end_time query parameters, then the pagination API will include them in subsequent requests automatically.

The script I linked to handles that automatically to make sure you get all your data.

Also note: the script includes a `time.sleep(1)` line in between requests. make sure that stays in or you will probably end up throttled by IO. We have a higher cap on HTTP GET requests, but we still have a cap.


- Adam

User avatar
cweiss
 
Posts: 6
Joined: Sat Oct 20, 2018 4:01 pm

Re: Query stored data on adafruit.io by time

Post by cweiss »

Thanks for the reply Adam!

Ok, this will work I think. For now I don't actually need this, I can do some querying locally. I will certainly get this implemented down the road and will revisit this forum.

Cheers,

Corey

User avatar
cweiss
 
Posts: 6
Joined: Sat Oct 20, 2018 4:01 pm

Re: Query stored data on adafruit.io by time

Post by cweiss »

Hi Adam!

Configuring this into Arduino C seems pretty daunting, do you have any other ideas or references I can go off of? I guess you are using a raspberry pi or laptop with MQTT? I'm open to using other protocols, not just MQTT, just pulling my daily values off adafruit (about 50 entries) is what I need...

Cheers!

Corey

Locked
Forum rules
If you're posting code, please make sure your code does not include your Adafruit IO Active Key or WiFi network credentials.

Return to “Internet of Things: Adafruit IO and Wippersnapper”