Occasional 404 error and lost data... Moderators: adafruit_support_bill, adafruit

[brubell]

Ok - this is the expected scenario for a feed that hasn't been updated in 2 months (60+ days).

The time to live for the data points are 60 days on an io+ account. The data rolls off as it expires beyond that 60 day threshold. The '7200' value is the last_value saved, and we store that differently for convenience and performance reasons, and do not roll off that value. The historical data does expire though.

Free accounts are provided 30 days of storage, and io+ accounts are 60 days of storage.

[cstanke]

The '7200' value is the last_value saved, and we store that differently for convenience and performance reasons, and do not roll off that value

The underlying issue here is that the "last_value saved” does roll off since the result of reading the feed after 60 days of no updates is a 404 error—as if the feed no longer exists. The result is an incongruity where the feed page shows a last value but that value is not accessible from the feed itself.

Is there no way to maintain a persistent value in a feed that is rarely updated (beyond caching it on the device and handling the 404 error state)? Seems clunky to roll off everything such that feed effectively becomes non-existent after 60 days of no changes.

[jwcooper]

I think in this case, the bug may be that we are using that last_value for that table, and probably shouldn't be, as it's not really indicative of the data records stored within that feed once all of the data rolls off.

The feed itself doesn't 404, does it? I assume calling

Code: Select all | TOGGLE FULL SIZE

/api/v2/{username}/feeds/{feed_key}

would return all of the metadata for that feed.

I would expect calling

Code: Select all | TOGGLE FULL SIZE

/api/v2/{username}/feeds/{feed_key}/data

something like that to return a 404, as there isn't any actual data records. Only the metadata of the last_value stored within that feed.

This is all sort of technical behind the scenes information, but I think it really comes down to is that there is a bug in that table showing data, when there really isn't any data. The value of seeing the last_value may be lost if it's confusing that there isn't actually any data records held within that feed.

We'll file an issue internally and sort out if we should return the last_value always (even if there is no data), or if we should update that UI to show 'no data' once the data records have expired.

[cstanke]

The feed itself doesn't 404, does it? I assume calling

Code: Select all | TOGGLE FULL SIZE

/api/v2/{username}/feeds/{feed_key}

would return all of the metadata for that feed.

Correct, using curl and the API url, the feed itself does indeed return metadata. Until you mentioned that, I'm not sure I would have taken the time to try it because I've only seen this issue using the CircuitPython http client object. For instance (pseudo-code):

Code: Select all | TOGGLE FULL SIZE

from adafruit_io.adafruit_io import IO_HTTP
io = IO_HTTP(ADAFRUIT_IO_USER, ADAFRUIT_IO_KEY, wifi)
apod_delay_feed = io.get_feed('pyportal-01.apod-delay')
io.receive_data(apod_delay_feed['key'])


The io.receive_data call returns the 404 error as shown in the PyPortal image in one of my previous posts (see below).

Granted, catching the AdafruitIO_RequestError (which I wasn't doing) could have helped here. But, a 404 on a request for data, from a valid feed, with a known last value kinda seems weird, no?

We'll file an issue internally and sort out if we should return the last_value always (even if there is no data), or if we should update that UI to show 'no data' once the data records have expired.

I expect what you ultimately do will warrant internal dev discussions, but my 2¢ is that if "last_value" is available but historical data has been purged, the CircuitPython case (at least) shouldn't return a 404 error, it should just offer up the last known value.