0

Server upgrade performed on 2018-04-18
Moderators: adafruit_support_bill, adafruit

Please be positive and constructive with your questions and comments.

Server upgrade performed on 2018-04-18

by jwcooper on Wed Apr 18, 2018 11:34 am

We performed a fairly major server upgrade around 10:10 am CT. There may have been a less than ~5 minute slowdown of mqtt data. No data would have been lost, but the time may be delayed by a slight margin of error.

If you see any further issues or anything else that isn't working after this migration, please let us know.

Thank you!

jwcooper
 
Posts: 663
Joined: Tue May 01, 2012 9:08 pm

Re: Server upgrade performed on 2018-04-18

by pedalermike on Wed Apr 18, 2018 11:53 am

One of my feeds that have been working for nearly a year is now erroring out "2018-04-18 10:47:38 error saving data on pedalermike/feeds/sense-wind, string contains null byte" . Nothing in my client code changed but this error started happening today. The error condition causes my client to time out and then restart which in turn puts the entire app into reset mode.

pedalermike
 
Posts: 11
Joined: Sat Oct 22, 2016 6:17 pm

Re: Server upgrade performed on 2018-04-18

by jwcooper on Wed Apr 18, 2018 12:24 pm

We're looking into the null byte error. Will respond as soon as we can determine a root cause.

Are you sending this data as a group, or individual feeds?

Edit:

We're still looking into why this started happening now, but a quick fix on your end may be to clean up the data being sent for that feed. It looks like a null byte is being sent to IO after the value.

Example (not your actual data):
value="99\u0000"

jwcooper
 
Posts: 663
Joined: Tue May 01, 2012 9:08 pm

Re: Server upgrade performed on 2018-04-18

by jwcooper on Wed Apr 18, 2018 1:09 pm

We've deployed a fix for the null byte issue. Let us know if you see any further issues. Thank you!

jwcooper
 
Posts: 663
Joined: Tue May 01, 2012 9:08 pm

Re: Server upgrade performed on 2018-04-18

by pedalermike on Wed Apr 18, 2018 2:07 pm

Thanks for the quick reply. All is working fine. Good work!

pedalermike
 
Posts: 11
Joined: Sat Oct 22, 2016 6:17 pm

Re: Server upgrade performed on 2018-04-18

by franksmicro on Thu Apr 19, 2018 5:32 pm

I publish several feeds of weather data once a minute to a private feed using MQTT using a Raspberry Pi.

Yesterday morning I note a several hour gap in data. Today it appears that posting stopped around 4:45AM EDT. Since then, only 2 of the per-minute updates have worked (one around 8:55AM and one around 10:05AM). Updates are presently failing. I have not made any changes to my setup.

I also tried manually connecting to the MQTT server (using MQTTTool on my iPhone). Each time I click Connect I see "Status: Disconnected 2". My iPhone is completely independent from my home internet connection. In the past, I have been able to post and subscribe using this tool.

Is this related to the upgrade, or could there be a problem with my account?

Thanks,
Frank

franksmicro
 
Posts: 2
Joined: Thu Apr 19, 2018 4:21 pm

Re: Server upgrade performed on 2018-04-18

by jwcooper on Fri Apr 20, 2018 10:56 am

Are you still seeing this issue? We don't have any reports of server issues from anyone else at this time.

jwcooper
 
Posts: 663
Joined: Tue May 01, 2012 9:08 pm

Re: Server upgrade performed on 2018-04-18

by franksmicro on Fri Apr 20, 2018 6:43 pm

It appears that the problem was being caused by another application that I am using to monitor my data in another location. That application uses an ESP32 to subscribe to my feeds. If it gets disconnected, it immediately tries to connect again. Apparently when the server was down briefly for the upgrade, it kept trying to connect and subscribe. It looks like some sort of rate limiting is done on the connect attempts, and I exceeded that rate trying to reconnect when the server was down briefly. Because it kept trying to reconnect while rate limiting was stopping me, I never got connected with the ESP32 even when the service came back up. And attempting to connect using my ID was also failing for posts from my Raspberry Pi, and from other devices such as my iPhone.

Within minutes of shutting down my ESP32, MQTT posts began working again.

I will add a sleep of a minute or so between connect attempts in my ESP32 code so that it does not hammer the server trying to reconnect if the server is down temporarily.

Thanks for the response, and for the service you provide. Being able to graph posted values and create dashboards with controls to post to MQTT saves a bunch of coding vs. having to run my own MQTT server and develop my own database interface, graphing tools, and web widgets.

Frank

franksmicro
 
Posts: 2
Joined: Thu Apr 19, 2018 4:21 pm

Re: Server upgrade performed on 2018-04-18

by kklicker on Sat Apr 21, 2018 1:45 pm

Is the server down? It is April 21 and I am unable to reach the server. Is this related to the upgrade or something else?

Thanks.

kklicker
 
Posts: 1
Joined: Thu Feb 16, 2017 1:05 pm

Re: Server upgrade performed on 2018-04-18

by approachct on Wed May 02, 2018 9:16 pm

Is there an issue with dashboard line graphics since the upgrade? No matter where I point on the graph the "tool tip" shows a time close to the current time instead of the time where the pointer is. See attached picture. Notice the time in the tool tip and where the point is on the X axis below.
Attachments
Screen Shot 2018-05-02 at 9.10.30 PM.png
Screen Shot 2018-05-02 at 9.10.30 PM.png (80.18 KiB) Viewed 192 times

approachct
 
Posts: 3
Joined: Sat Mar 03, 2018 11:29 am

Please be positive and constructive with your questions and comments.