API and Dashboard Different Data

[pthacker]

I've created a dashboard with several switches. I use them to control lights in my house.

I have an issue: MQTT data and API data are different. The MQTT data has a switch OFF and the API data has it ON.

Is this a bug or am I doing something wrong?

[pthacker]

Ok here is a little more information. The Feeds shows the garage light switch on

Feeds view with garagelight feed - Switch ON

Feeds.png (18.1 KiB) Viewed 539 times

If you look at the garagelight feed it is OFF - and the dashboard is also OFF

Feed.png (42.91 KiB) Viewed 539 times

I pull down with API data for an iPhone app and the garagelight feed last updated is ON

API.png (87.18 KiB) Viewed 539 times

The time stamp matches the feed (MQTT) but the API shows a different value at the same time stamp.

I need a little help here! Am I interpreting the API incorrectly?

Dave

[pthacker]

Ok, I have a little more information. I pulled down the detailed info on a feed using API DOCS. I looks like the "last updated" is using the value from the first entry not the last entry. I'm guessing here but I used the time stamps to make this assumption.

NOTE - the Switch is in the OFF position in on the dashboard and feed displays. See snapshot below.

details.png (118.07 KiB) Viewed 516 times

[abachman]

Hey pthacker,


Just wanted to ping you here and let you know we're taking a look. I'll post an update when I have more info.


- adam

[pthacker]

Adam,

Thank you. Not urgent.

Dave

[abachman]

No worries, this one was very worthing looking into. The key bit was the /details lookup that had a different value in `last_value` than it did in the last data `value` field. There are two hard problems in computer programming: naming things, cache invalidation, and off by one errors. We totally nailed that second one.

We're aggressively caching in the MQTT layer to reduce DB traffic when writing multiple times to the same feed quickly. The result was that in cases where values repeated, the system thought the `last_value` field hadn't changed because the cached value was the same as the new value, so it didn't update the feed record. BUT, it was only when the new value was a repeat of a previously stored value AND when it was repeated inside a 5 minute window. Because we're storing actual data records separately, they were always up to date and accurate, so one consolation is that the incorrect value was never sent to MQTT subscribers or any API that listed data records, so in most cases, IO clients got correct data in spite of the last_value error.

End result, sometimes inconsistent /{username}/feed/{key} last_value and /{username}/feed/{key}/data/last value. Sorry about that.

Anyhow, fixed that bug and updated IO, you shouldn't see it again but do please check and make sure. I'm running a test script that was failing before the deployment with the same error you were seeing and is working fine now.


- adam

[pthacker]

Adam,

Great troubleshooting! And thanks for the explaination. Years ago I was working on a MIPS processor and had a problem with stale data in a TLB cache line. It was really hard to fix.

I'll try to verify later today. (I'm trying to reconfigure a Pi Zero with a USB WIFI dongle. Its frustrating with the single micro-USB port. I will ONLY buy Zero Ws in the future)

Dave

[pthacker]

Adam,

Problem fixed and verified. Thank you!

Dave