Board freezing sporadically during `code.py` runs (Matrix Portal M4)

[LauraTe]

Versions:
- FW: '1.2.2\x00'
- SW: 7.3.3

Hardware:
- Matrix Portal M4
- 64x32 RGB LED Matrix with 4mm pitch

Code:
https://github.com/laurate/on-air-off-a ... gn/code.py

Hi all!
I have been using the YouTube On Air project https://learn.adafruit.com/rgb-matrix-a ... n-air-sign as a starting point to implement an On Air / Off Air sign and I repeatedly run into issues with the ESP (issue 1 and 2 are ok, just describing them for context):
1. during initial connection to the network it will repeatedly give errors `Failed to connect to ssid` or `No such ssid` and then connect fine after several retries (RSSI on connection is -60 to -36, depending on location)
2. it will often take a long time for straight-forward requests or time out completely (then on a retry it works fine)
3. every ~10 minutes or so it will completely hang up after a request although it has already received the response. It will still print the response object and status code 200 but then never continue to return and print the response (in my code that's line 217).

Code: Select all

Fetching status from http://192.168.178.23:6006/get_status
code:  200
<Response object at 0x200049a0>
# ... nothing happening until reset

I have tried to add a watchdog for issue 3, but because of issue 2 this has been triggered every ~2 minutes during a 1 hour testrun (setting watchdog to the max timeout of 16 seconds).

It's kind of hard to debug this with only the high-level logs so I've tried to eliminate a few possible causes:
1. I completely disconnected the LED matrix and only executed the ESP-related code, still seeing the same issues
2. we have a mesh network, on disabling the repeater I still saw the same issue
3. I changed the location, had a slightly better RSSI but still the same issues with watchdog being triggered and board becoming unresponsive
4. we have a 2.4 GHz and 5 GHz network with the same ssid, but my understanding is, that the ESP32 only has a 2.4 GHz receiver, so that shouldn't be an issue correct?

I saw that there were some issues in the past with the MatrixPortal hanging (e.g. https://github.com/adafruit/circuitpython/issues/6205) that have been fixed with CircuitPython 7.3.3, are you aware of similar issues in version 7.3.3? Could it be worth it to try the beta 8.0.0?

Also: is there a way to access more low-level logs, to see what could be causing an issue after a successful request?

Thank you so much for reading and please let me know if you need any more info from me!

Best, Laura

[eddtrettel]

Hi Laura,

I had unstable ESP32 (in my case BLE) on the Matrix Portal M4 until I upgraded the ESP32 firmware. Your issue could be related since the ESP32 does all of the networking.

Before you upgrade the firmware, you can check it's current level by followling the "Connect to WiFi" section in https://learn.adafruit.com/adafruit-pyp ... et-connect. You can copy/paste the code there to check on the firmware, etc.

I think the most current firmware level is 1.7.4.0, based upon the article at https://learn.adafruit.com/upgrading-es ... t-firmware. BTW, in that article they reference the "Connect to WiFi" that I've listed above.

In my case I did the firmware upgrade and the ESP32 became very, very stable. I used my Windows (11) computer to do the entire process, but you have have other machines like Linux that will work just as well. Instructions for both are at the link above.

Completely unrelated to your question, I wrote some code for the Matrix Portal M4 on a 32x64 LED matrix and you can get the code, documentation, etc. at viewtopic.php?t=191902 if you are interested.

Good luck with the unstable n/w issue.

E.T.

[LauraTe]

Hi E.T.,

Thanks so much for your detailed response. I'm on FW version 1.2.2, so it definitely sounds like it would be worth it to upgrade.
I will try this as soon as I get a chance and add an update here on how it goes with the networking issues!

[eddtrettel]

Hi Laura,

I forgot to mention that you should backup your Matrix Portal (code, lib, etc) before you do the firmware update. The firmware update will erase all of that.

Also, once the firmware update is done you'll need to re-install CircuitPython via the "UF2" file drag method. There is only a passing mention of this in a sentence somewhere in the firmware update doc, but I missed it at first and thought I had toasted my Matrix Portal M4.

I think you'll be in good shape after the upgrade.

All the best,
E.T.