0

ESP32SPI / MatrixPortal wifi connect issues
Moderators: adafruit_support_bill, adafruit

Please be positive and constructive with your questions and comments.

ESP32SPI / MatrixPortal wifi connect issues

by Jrbski on Wed May 05, 2021 10:26 pm

OK, this is a weird one... I have a few different wifi networks, and I'm trying to get my matrixportal to send data to adafruit.io. I had used the matrixportal and adafruit.io previously for the text scroller demo. Now I am having issues connecting to 2 of 3 wifi networks. A unifi and asus router won't work, but a 'hacked' dd-wrt router does work. As an example, I attached the debug from wifi manager for the unifi router. Even though the dd-wrt has a weaker signal, it successfully connects. The only obvious difference is the working ssid is all lower case, and any ssid with uppercase fails. All use WPA2 personal.

I have updated from CP6.1 to CP6.2 and the 5/25 libraries with no change in behavior. I don't recall what version of CP the wifi previously worked on, but the unifi firmware was recently updated.

Any suggestions on what else to try? I may try setting up another 'guest' ssid with all lowercase as a last gasp. Note that the debug returns no error info after the 'retry' message that looks like should be printed out based on the code in github. I do have other ESP32's on the unifi wireless with no issues.

Code: Select all | TOGGLE FULL SIZE
ssID = Unifi
Connecting to WiFi...
ESP32 found and in idle mode
Firmware vers. bytearray(b'1.2.2\x00')
MAC addr: ['0x##', '0x##', '0x##', '0x##', '0x##', '0x##'
        Unifi           RSSI: -63
        RX          RSSI: -63
        JXR         RSSI: -66
        JX              RSSI: -66
        dd-wrt              RSSI: -81

Connecting to AP...
Failed to connect, retrying
 ('No such ssid', b'Unifi')
Connecting to AP...
Failed to connect, retrying
 ('Failed to connect to ssid', b'Unifi')

Jrbski
 
Posts: 11
Joined: Thu Sep 26, 2019 11:27 pm

Re: ESP32SPI / MatrixPortal wifi connect issues

by Jrbski on Thu May 06, 2021 7:44 pm

One update... I did look at the logs on Unifi, and I can see that the MatrixPortal did connect, and get an IP address from dhcp, but then I see a failure, so it looks like the issue is on the esp32 side.

Jrbski
 
Posts: 11
Joined: Thu Sep 26, 2019 11:27 pm

Re: ESP32SPI / MatrixPortal wifi connect issues

by Jrbski on Sat May 08, 2021 7:46 pm

Latest update: I did try adding an all lowercase ssid to the same access point, and that made no difference. I also updated the ESP32 firmware to 1.7.3, with no changes in behaviour. MatrixPortal connects to the access point, gets an IP from DHCP, and then disconnects. I never get a ping response from the ESP while it is connecting, so it seems like the wifi connection is never fully completed.

Jrbski
 
Posts: 11
Joined: Thu Sep 26, 2019 11:27 pm

Re: ESP32SPI / MatrixPortal wifi connect issues

by Jrbski on Wed May 12, 2021 11:47 am

So I found a hint from another website that noted similar wifi issues related to CP 6.1... the difference is that my 'working' AP is also running as a DHCP server, and the AP's that aren't are using a separate DHCP server. The delay in getting DHCP may be causing the ESP32 to give-up. I see there is an 'AP_connect' method that has a time-out setting that I will try to replace the 'connect' method next before I try to roll back to CP 6.0.

Jrbski
 
Posts: 11
Joined: Thu Sep 26, 2019 11:27 pm

Re: ESP32SPI / MatrixPortal wifi connect issues

by Jrbski on Wed May 12, 2021 8:52 pm

I'm pretty much stuck now... the connect_AP delay didn't help, and neither did the rollback to CP 6.0. I enabled all the debugging I can. The only thing I see is that my AP reports the link in low power mode, but that is consistent with other esp's on the network...

Jrbski
 
Posts: 11
Joined: Thu Sep 26, 2019 11:27 pm

Re: ESP32SPI / MatrixPortal wifi connect issues

by adafruit_support_mike on Wed May 12, 2021 11:10 pm

You'll probably need to sniff the network traffic with something like Wireshark to see which signals are being sent and which are being missed.

If the ESP32 can connect to one network but not to the others, you should be able to compare the traffic and see where things stop working.

adafruit_support_mike
 
Posts: 62788
Joined: Thu Feb 11, 2010 2:51 pm

Re: ESP32SPI / MatrixPortal wifi connect issues

by Jrbski on Thu May 13, 2021 12:01 pm

Just to complete the documentation, this is what I see in the Unifi AP logs... I don't see any errors, so it looks like the ESP32 is just sending the disconnect.

Unifi_success_detail.png
details on the connect success log message
Unifi_success_detail.png (7.06 KiB) Viewed 162 times


Unifi_log.png
Log of wifi connect sequence
Unifi_log.png (28.11 KiB) Viewed 162 times

Jrbski
 
Posts: 11
Joined: Thu Sep 26, 2019 11:27 pm

Re: ESP32SPI / MatrixPortal wifi connect issues

by Jrbski on Thu May 13, 2021 7:44 pm

adafruit_support_mike wrote:You'll probably need to sniff the network traffic with something like Wireshark to see which signals are being sent and which are being missed.

If the ESP32 can connect to one network but not to the others, you should be able to compare the traffic and see where things stop working.


So by pure luck, I happened to catch a successful connection to the Unifi AP, so I was able to compare a failure to a successful connection. On the success, you can see that the matrix portal successful connects to adafruit.io. In the failure, it seems to time out processing the DHCP response, and makes another DHCP request. The DHCP packet sent in both cases is the same, so more data that points to the ESP32. Note that there is a delay second ARP request in the 'success' dump, so it looks like that is when the ESP32 is processing the DHCP packet.

The single success seems to point to a timing issue, either on the DHCP response, or the ESP32 setting the IP and/or DNS settings in the DHCP packet. As a note, the .11.22 address is my pi-hole DNS, and I am not using pi-hole on the dd-wrt AP.

Unifi_success_detail.png
TCPdump comparison between a successful and failed association
Unifi_success_detail.png (101.85 KiB) Viewed 156 times

Jrbski
 
Posts: 11
Joined: Thu Sep 26, 2019 11:27 pm

Re: ESP32SPI / MatrixPortal wifi connect issues

by adafruit_support_mike on Thu May 13, 2021 11:23 pm

Hmm.. the failure explicitly declines the address it gets from the DHCP server (line 71 on the right).

That makes sense from the traffic that follows the DHCP config packet. The ESP32 checks to see if any other device responds to the same address (who-has 192.168.11.183 tell 0.0.0.0), sees a reply from 70:85:c2:etc, and sends back a DECLINE packet because the address is already in use.

The part I can't understand is why the version on the left ignores those replies and takes the address anyway.

Do you happen to know which of your devices has the MAC beginning with 70:85:c2?

adafruit_support_mike
 
Posts: 62788
Joined: Thu Feb 11, 2010 2:51 pm

Re: ESP32SPI / MatrixPortal wifi connect issues

by Jrbski on Fri May 14, 2021 11:11 am

adafruit_support_mike wrote:Hmm.. the failure explicitly declines the address it gets from the DHCP server (line 71 on the right).

That makes sense from the traffic that follows the DHCP config packet. The ESP32 checks to see if any other device responds to the same address (who-has 192.168.11.183 tell 0.0.0.0), sees a reply from 70:85:c2:etc, and sends back a DECLINE packet because the address is already in use.

The part I can't understand is why the version on the left ignores those replies and takes the address anyway.

Do you happen to know which of your devices has the MAC beginning with 70:85:c2?


Well 70:85:c2 is a 'mystery device', but you definitely put your finger on the problem... I 'cheated' as a work-around by assigning a static IP for the matrixportal, and that solved the issue for now.

I checked arp tables on pi-hole, and a linux box, and they correctly had the matrixportal on .11.183 but a windows box still had 70:85:c2 in its arp table. It looks like a windows box on the network has a stale arp table linked to an old device. I guess the timing issue is related to how quickly the 'bad' arp response gets to the matrixportal. Unfortunately I was only watching TCPdump on the command line, so I don't know (yet) who was serving the ARP packet with the bad info. Some more detective work to track that part down, but it looks like windows may be 'helping'... NOT!

Thanks for the help!

Jrbski
 
Posts: 11
Joined: Thu Sep 26, 2019 11:27 pm

Re: ESP32SPI / MatrixPortal wifi connect issues

by Jrbski on Fri May 14, 2021 7:21 pm

Final mystery solved! 70:85:c2 = Asrock was a good hint. It turns out that a windows desktop (with an Asrock MB) was in an odd power state after a windows update. It looked like it was in 'sleep' mode, but it was mostly dead... no ping response, no display, not seen by dhcp server, etc. but the nic was 'alive' enough to respond to ARP requests! NMAP intense scan helped be persistent enough to hear an ARP response from it. Disconnect ethernet cable and its gone, plug back in an ARP response is back. I needed to power it off to bring it back.

Now is actually shows up in the DHCP tables, so all is well, and the network is ready for Adabox 018 which in now on the way!

Jrbski
 
Posts: 11
Joined: Thu Sep 26, 2019 11:27 pm

Re: ESP32SPI / MatrixPortal wifi connect issues

by adafruit_support_mike on Fri May 14, 2021 10:39 pm

Jrbski wrote:It turns out that a windows desktop (with an Asrock MB) was in an odd power state after a windows update. It looked like it was in 'sleep' mode, but it was mostly dead... no ping response, no display, not seen by dhcp server, etc. but the nic was 'alive' enough to respond to ARP requests!

Oy.. gotta love network debugging. It's such a fertile source of 'wait 'til you hear this' stories that it's almost worth the therapy.

FWIW after the fact, check your router to see if it keeps a device table. If you're lucky it will have a list of devices it's seen that includes human readable names, IP addresses, and MACs.

I'm glad to hear that line of approach turned out to be useful, and that you were able to find/fix the problem. Happy hacking!

adafruit_support_mike
 
Posts: 62788
Joined: Thu Feb 11, 2010 2:51 pm

Re: ESP32SPI / MatrixPortal wifi connect issues

by Jrbski on Mon May 17, 2021 3:44 pm

adafruit_support_mike wrote:
Jrbski wrote:It turns out that a windows desktop (with an Asrock MB) was in an odd power state after a windows update. It looked like it was in 'sleep' mode, but it was mostly dead... no ping response, no display, not seen by dhcp server, etc. but the nic was 'alive' enough to respond to ARP requests!

Oy.. gotta love network debugging. It's such a fertile source of 'wait 'til you hear this' stories that it's almost worth the therapy.

FWIW after the fact, check your router to see if it keeps a device table. If you're lucky it will have a list of devices it's seen that includes human readable names, IP addresses, and MACs.

I'm glad to hear that line of approach turned out to be useful, and that you were able to find/fix the problem. Happy hacking!


My router does have the full list of devices, and that is where I started the fun! unfortunately, the ethernet card on the desktop was alive enough to break the matixportal, but not enough to register with dhcp on the router. The router showed that the matrixportal owned 11.183, so it was a really weird failure mode.

Jrbski
 
Posts: 11
Joined: Thu Sep 26, 2019 11:27 pm

Re: ESP32SPI / MatrixPortal wifi connect issues

by adafruit_support_mike on Mon May 17, 2021 6:30 pm

Yeah that was an oddball.

adafruit_support_mike
 
Posts: 62788
Joined: Thu Feb 11, 2010 2:51 pm

Please be positive and constructive with your questions and comments.


cron