🎁📬❄️📦 Holiday Shipping Deadlines are approaching! International customers and domestic ground - get your orders in 12/9/2020 📦❄️📬🎁
0

Error with multiple sources for fetch_data() in MatrixPortal
Moderators: adafruit_support_bill, adafruit

Please be positive and constructive with your questions and comments.

Error with multiple sources for fetch_data() in MatrixPortal

by snaxter on Sun Nov 01, 2020 5:26 pm

I'm getting an error "Expected 01 but got 00" from the _check_data() function in the adafruit_esp32spi library. It looks like the socket did not return data after it was sent a command to get data from a URL.

I've attached the code that I've stripped down to three fetch_data() calls that reproduce this problem. I'm pretty sure the order of the calls is important. I'm using two calls to the purple air API and one call to the wunderground (weather.com) API. The wunderground API call works fine on its own and the two purple air calls work fine as long as there is not a call to the wunderground API. Any ordering of all three calls will eventually fail when the URL is changed.

Just guessing here: In tracing the calls through fetch_data() -> adafruit_requests -> adafruit_esp32spi_socket -> adafruit_esp32spi (check_data()), it looks like the code uses socket 0 for the first call(s) to fetch_data() and then will attempt to open socket 1 when a new URL is passed to fetch_data(). The check_data() call is definitely looking for data on socket 1 and not finding any.

I've tried increasing the timeout and adding time.sleep() calls between the fetch_data calls. Do I need to explicitly reset the socket before starting a new fetch_data() call?
Attachments
code.py
(1.93 KiB) Downloaded 1 time

snaxter
 
Posts: 19
Joined: Mon Mar 24, 2008 10:57 pm

Re: Error with multiple sources for fetch_data() in MatrixPo

by tannewt on Mon Nov 02, 2020 4:50 pm

Please try the latest requests version (1.7.4). We fixed a couple issues at the end of last week.

tannewt
 
Posts: 1922
Joined: Thu Oct 06, 2016 8:48 pm

Re: Error with multiple sources for fetch_data() in MatrixPo

by snaxter on Tue Nov 03, 2020 12:09 am

I updated just requests to 1.7.4.

This changed the error, but I'm still getting the error "ESP32 not responding" from the _wait_for_ready() function in esp32spi.py.

On the subsequent attempt to open a different URL, I can see that the new requests code does start off with a connect to socket 1 and then switches to socket 0 (this is new), but then fails getting data back from that socket.

snaxter
 
Posts: 19
Joined: Mon Mar 24, 2008 10:57 pm

Re: Error with multiple sources for fetch_data() in MatrixPo

by tannewt on Tue Nov 03, 2020 1:27 pm

Thanks for trying the latest! Melissa and I are going to look at it more today. So, please keep an eye out for newer versions of adafruit_requests and esp32spi libraries.

tannewt
 
Posts: 1922
Joined: Thu Oct 06, 2016 8:48 pm

Re: Error with multiple sources for fetch_data() in MatrixPo

by snaxter on Sun Nov 08, 2020 5:11 pm

I tried out version 1.75 of requests that was released on Friday and I'm still getting the error "ESP32 not responding" after the second call to fetch_data() for the request to get the second URL. I spent a bit of time trying to debug the error and I think the problem is in the _get_socket() function, where I see the connection getting an "OSError" from the sock.connect() call in that function. It does retry twice, but there is another error that gets thrown before it retries. I can see that the code now tries to use _socket_free.items() if it is retrying. Should there be a socket close somewhere in there if there is a retry?
Attachments
code.py
(1.64 KiB) Downloaded 2 times

snaxter
 
Posts: 19
Joined: Mon Mar 24, 2008 10:57 pm

Re: Error with multiple sources for fetch_data() in MatrixPo

by tannewt on Mon Nov 09, 2020 8:50 pm

Thanks for the example! Did you update ESP32SPI as well?

A connect failure that leads to an OSError should close the socket here: https://github.com/adafruit/Adafruit_Ci ... ts.py#L454

tannewt
 
Posts: 1922
Joined: Thu Oct 06, 2016 8:48 pm

Re: Error with multiple sources for fetch_data() in MatrixPo

by snaxter on Tue Nov 10, 2020 12:42 am

No - I did not update ESP32SPI - I missed that.

Fantastic! It looks like it is working! Thank you so much - so happy to be hacking on...

snaxter
 
Posts: 19
Joined: Mon Mar 24, 2008 10:57 pm

Re: Error with multiple sources for fetch_data() in MatrixPo

by tannewt on Tue Nov 10, 2020 2:29 pm

Great! The ESP32SPI fix ensures the nina-fw closes TLS sockets when send fails. Without it, it leaks memory and subsequent sockets can't fit.

tannewt
 
Posts: 1922
Joined: Thu Oct 06, 2016 8:48 pm

Please be positive and constructive with your questions and comments.