0

Line Chart Bug - May 3rd to now won't display data
Moderators: adafruit_support_bill, adafruit

Please be positive and constructive with your questions and comments.

Line Chart Bug - May 3rd to now won't display data

by sugarleaves on Fri May 03, 2019 9:58 pm

Hello,

I'm experiencing a bug where I switched my feed to sending data via MQTT. Previously it was using HTTP POST. My feed graph now is now not displaying a few hours worth of data, and is only showing the last datapoint (ie. the gap of data not showing is growing). The raw data is shown in the feed correctly with the correct timestamps, so I know it's there.

The incorrect graph looks like:
Image

I've since switched back to HTTP POST but the graph bug is still there. Again, I can still see new raw feed data being added, it's just displaying all wrong.

The page loads with the graph missing that straight line (and thus missing hours of data), and then loads the latest data point it received and ignores everything in between.

Anyone have ideas on how to fix this? I'm tempted to just reset all the data/create a new feed but would rather not lose my history.
Last edited by sugarleaves on Sat May 04, 2019 9:29 am, edited 1 time in total.

sugarleaves
 
Posts: 5
Joined: Mon Apr 29, 2019 8:14 pm

Re: Line Chart Bug - Switched From HTTP to MQTT

by sugarleaves on Sat May 04, 2019 1:27 am

I switched back to MQTT but I'm still having the same problem with the graph only showing the last value sent, even though feed data is clearly present.

sugarleaves
 
Posts: 5
Joined: Mon Apr 29, 2019 8:14 pm

Re: Line Chart Bug - Switched From HTTP to MQTT

by cjdshaw on Sat May 04, 2019 6:50 am

I'm having the same issue since about 5pm UTC on 3rd May. Data shows up to that point, then jumps in a straight line to the latest data when it comes in

Here's a public dashboard showing the problem
https://io.adafruit.com/cjdshaw/dashboards/temp

When you open the dashboard, it only show data up until 5pm on 3rd. If you leave it running, you'll see new data appear on the right (every 2 min), but a big gap where there is data but it's not being drawn

I didn't change anything about my setup at that point, so I don't think it's to do with HTTP / MQTT. You could probably amend the topic title

cjdshaw
 
Posts: 10
Joined: Mon May 08, 2017 4:04 pm

Re: Line Chart Bug - May 3rd to now won't display data

by sugarleaves on Sat May 04, 2019 9:30 am

Thanks for the reply @cjdshaw. I changed the title to more accurately reflect the issue we're experiencing.

sugarleaves
 
Posts: 5
Joined: Mon Apr 29, 2019 8:14 pm

Re: Line Chart Bug - May 3rd to now won't display data

by cjdshaw on Sat May 04, 2019 6:43 pm

Thanks. Even the feed pages are showing the same issue. The graph only goes up until the 3rd, despite there being loads of data in the table. Then new data arriving causes it to draw a straight line
https://io.adafruit.com/cjdshaw/feeds/loft-outside

cjdshaw
 
Posts: 10
Joined: Mon May 08, 2017 4:04 pm

Re: Line Chart Bug - May 3rd to now won't display data

by daveNC on Sat May 04, 2019 6:56 pm

Same here. Bad bug.

daveNC
 
Posts: 40
Joined: Thu Nov 09, 2017 2:32 pm

Re: Line Chart Bug - May 3rd to now won't display data

by sugarleaves on Sat May 04, 2019 11:40 pm

My chart started showing data again around 1am UTC. At the time I was messing around with how I was sending data (currently using HTTP POST). Not sure if that's related or not....

sugarleaves
 
Posts: 5
Joined: Mon Apr 29, 2019 8:14 pm

Re: Line Chart Bug - May 3rd to now won't display data

by cjdshaw on Sun May 05, 2019 6:10 am

Yes, mine's back since about 00:48 UTC. Shame to have a big gap in my long term data, but at least the short term plot is usable again

cjdshaw
 
Posts: 10
Joined: Mon May 08, 2017 4:04 pm

Re: Line Chart Bug - May 3rd to now won't display data

by daveNC on Sun May 05, 2019 6:47 am

Mine has been sporadic in the past 36 hours or so. Back working now.

daveNC
 
Posts: 40
Joined: Thu Nov 09, 2017 2:32 pm

Re: Line Chart Bug - May 3rd to now won't display data

by abachman on Mon May 06, 2019 11:17 am

Hi all!


Just an update on the charting data problem:

We deployed some changes to IO's frontend and backend on Friday around 12pm EDT that fixed the data filtering + paging feature and added the feature of filtering the chart data, too, when you set a filtered date range on feed pages. Good stuff! Super handy! New pagination docs are up on the documentation site if you're curious.

Bad part was we silently and accidentally, in an unrelated part of the code, in a hilarious fit of irony, broke the data aggregator process that actually makes charts work. The timing was completely coincidental to the timing of your original switch from HTTP to MQTT, sugarleaves, which is super lame, but we didn't catch it until Saturday afternoon.

Saturday night we were able to fix the bug and restart the aggregator, which is why chart data started reappearing. Right now we're running through the backlog of data that was missed from Friday to Saturday. That process should finish in a little under 8 hours.

So, we know what caused the missing chart data--and it's just missing chart data, all the actual data is intact--we've fixed the glitch, and we're rebuilding the charts now. The HTTP and MQTT APIs performed fine through the aggregator downtime, so everything was recorded and all devices could communicate with each other fine.

I'm happy to answer any other questions y'all have.


- Adam B.

abachman
 
Posts: 287
Joined: Mon Feb 01, 2010 12:48 pm

Re: Line Chart Bug - May 3rd to now won't display data

by daveNC on Mon May 06, 2019 11:37 am

Thanks for fixing it. Perhaps you can post a change planning calendar now that this service is used by customers.

daveNC
 
Posts: 40
Joined: Thu Nov 09, 2017 2:32 pm

Re: Line Chart Bug - May 3rd to now won't display data

by daveNC on Mon May 06, 2019 3:53 pm

I'm no longer receiving my data. I cannot validate if it is my side; I don't expect it is though since it worked consistently for weeks and I do a total restart if it has any issues.

Anyone else see a current problem?

daveNC
 
Posts: 40
Joined: Thu Nov 09, 2017 2:32 pm

Re: Line Chart Bug - May 3rd to now won't display data

by cjdshaw on Mon May 06, 2019 6:49 pm

All fine here

cjdshaw
 
Posts: 10
Joined: Mon May 08, 2017 4:04 pm

Re: Line Chart Bug - May 3rd to now won't display data

by sugarleaves on Wed May 08, 2019 12:09 am

Thank you for the explanation (and enhancements!) @abachman, my graphs are looking excellent again :)

Perhaps you can post a change planning calendar


Not a bad idea

sugarleaves
 
Posts: 5
Joined: Mon Apr 29, 2019 8:14 pm

Please be positive and constructive with your questions and comments.