Voting resources, early voting, and poll worker information - VOTE. ... Adafruit is open and shipping.
0

When will my data get deleted?
Moderators: adafruit_support_bill, adafruit

Forum rules
If you're posting code, please make sure your code does not include your Adafruit IO Active Key or WiFi network credentials.
Please be positive and constructive with your questions and comments.

When will my data get deleted?

by jerryJiren on Tue Dec 17, 2019 1:44 am

I understand that my data will get deleted after 30 days on the server. But how can I know exactly when?

I made a water flow meter and I am using a vb.net app to read data and analyze from the JSON files, but I would like to know when will the data be deleted
so I don't miss the information.

For example, if the data would get deleted on December 31st, and I had previously downloaded a JSON file on the 28th, I could lose data If I didn't download another file again.

jerryJiren
 
Posts: 2
Joined: Tue Dec 17, 2019 1:39 am

Re: When will my data get deleted?

by abachman on Tue Dec 17, 2019 11:17 am

Hi jerryJiren,


Deletion of data in IO is more like expiration, and it's on a rolling basis. When a data point is stored, it is given a 30 day time to live (TTL) value. Once that TTL date has passed, the data point is unreachable. If you, for example, downloaded your data every 30 days, to the second, you would be guaranteed to preserve every data point. If you use the Get Feed Data API, for example:
Code: Select all | TOGGLE FULL SIZE
$ curl https://io.adafruit.com/api/v2/abachman/feeds/temperature/data?limit=1
[
  {
    "id": "0EAV600SVAZRBC0MX06TWCJZTD",
    "value": "70.52",
    "feed_id": 606108,
    "feed_key": "temperature",
    "created_at": "2019-12-17T15:07:35Z",
    "created_epoch": 1576595255,
    "expiration": "2020-02-15T15:07:35Z"
  }
]
Each data point has a field named "expiration" that shows the last time at which the data point will be available.

If you want to automate downloading, you'll have to use the Get Feed Data API mentioned above. You could write a script to download the previous day's data and have it run at the beginning of each day. As long as your script always covers 24 hours, you'll never miss a value. I published an example of a (kind of long) Python script that might be helpful in downloading large chunks of data from IO: https://gist.github.com/abachman/12df0b34503edd5692be22f6b9695539.


- adam b.

abachman
 
Posts: 352
Joined: Mon Feb 01, 2010 12:48 pm

Re: When will my data get deleted?

by jerryJiren on Tue Dec 17, 2019 5:15 pm

Thanks, I understand.

I have one last question:

If my device were to send data each day a 3 pm, would a script that downloads JSON files each day at 2 pm miss any information?

jerryJiren
 
Posts: 2
Joined: Tue Dec 17, 2019 1:39 am

Re: When will my data get deleted?

by abachman on Thu Dec 19, 2019 6:11 pm

If my device were to send data each day a 3 pm, would a script that downloads JSON files each day at 2 pm miss any information?


I don't think I understand the scenario 100%. If you send data at 3pm, a download task that ran at 2pm (earlier in the day?) wouldn't see it. A download task that ran at 2pm the next day would see it.

If you ran a job once a week that downloaded the whole feed -- the script I linked to earlier would do it -- you would absolutely never miss a value from your feed due to expiration. You would get _a lot_ of duplicate data, so you'll have to sort out your own system for de-duplicating data based on the unique "id" field that comes with each data point..


- adam b.

abachman
 
Posts: 352
Joined: Mon Feb 01, 2010 12:48 pm

Please be positive and constructive with your questions and comments.