Matrix Portal UDP server not receiving packets

Moderators: adafruit_support_bill, adafruit

Forum rules
If you're posting code, please make sure your code does not include your Adafruit IO Active Key or WiFi network credentials.
Locked
User avatar
BNNorman
 
Posts: 12
Joined: Tue Jan 03, 2023 7:17 am

Matrix Portal UDP server not receiving packets

Post by BNNorman »

I've been grappling with this on and off for a few weeks and about to give up (or change tack). The objective is to send images (video frames) using UDP to the Adafruit Matrix Portal (MP) and display them. Just about all examples I find pull data from somewhere.

One example of a server I found uses bind() but that doesn't exist in the MP CircuitPython 8.0.0 rc2. Probably because the ESP32 is driven by SPI and not loaded with CP

I can get a UDP server working on a Pimoroni Interstate 75 W (Rpi Pico) and can almost get 25fps. I send the video frames from a python script on my desktop machine which does all the frame resizing and LED brightness compensation.

I can also get an FTP server working on the MP but it is slow due to FTP handshake (about 10fps).

I can get a UDP client working - an example I found which gets time from pool.ntp.org. But this sends a UDP packet to pool.ntp.org and receives a packet back.

The code I'm trying to use is below. It connects to my wifi network ok and reports an IP address 192.168.1.202. Ping to that address gets a response from the MP.

I have read all the code in the adafruit_esp32spi library folder trying to find out what is going on.

I must be missing something OR a UDP server is impossible on the MP. Whch is it?

Can anyone help? Has anyone gotten a UDP server to work on the MP?

Thanks for reading.
Regards
Brian


Code: Select all

# Adafruit CircuitPython 8.0.0-rc.2 on 2023-02-02; Adafruit Matrix Portal M4 with samd51j19
import time
import board
import busio
from digitalio import DigitalInOut
import adafruit_esp32spi.adafruit_esp32spi_socket as socket
from adafruit_esp32spi import adafruit_esp32spi
import os

PORT = 8000

CREATE_AP=False # True= create a new AP (192.168.4.1) otherwise get Ip by dhcp

PORTAL_SSID="Portal"  # only needed if CREATE_AP is True
PORTAL_PASS=None      # open AP

spi = board.SPI()

esp32_cs = DigitalInOut(board.ESP_CS)
esp32_ready = DigitalInOut(board.ESP_BUSY)
esp32_reset = DigitalInOut(board.ESP_RESET)

esp = adafruit_esp32spi.ESP_SPIcontrol(spi, esp32_cs, esp32_ready, esp32_reset)
socket.set_interface(esp)

if CREATE_AP:
    # create AP creates a server with an ip address 192.168.4.1
    esp.create_AP(PORTAL_SSID,PORTAL_PASS)
    print(f"Now an AP: RSSI ={ esp.rssi} IP={esp.pretty_ip(esp.ip_address)} port:{PORT}")
else:
    # connect_AP connects to the existing network
    esp.connect_AP(os.getenv("WIFI_SSID"), os.getenv("WIFI_PASSWORD"))
    print(f"Connected to AP: RSSI ={ esp.rssi} IP={esp.pretty_ip(esp.ip_address)} port:{PORT}")


s = socket.socket(type=socket.SOCK_DGRAM)
esp.start_server(PORT, s.socknum, conn_mode=1,ip=esp.ip_address) # conn_mode=UDP (see ESP_SPIcontrol)


while True:
        
    numbytes_avail = esp.socket_available(s.socknum)
    if numbytes_avail:
        print(numbytes_avail)
        bytes_read = esp.socket_read(s.socknum, numbytes_avail)
        print(bytes_read)
        
    time.sleep(0.1)

User avatar
adafruit_support_mike
 
Posts: 67446
Joined: Thu Feb 11, 2010 2:51 pm

Re: Matrix Portal UDP server not receiving packets

Post by adafruit_support_mike »

It should be possible to do UDP, but that's a risky protocol for video.

UDP doesn't guarantee delivery, and doesn't include any concept of packet sequencing. The 'no guarantee' thing can be okay in local networks with no other traffic, but the lack of sequencing means you can't do fragmentation and reassembly reliably. If any packet is dropped, delayed, or arrives out of order, your video will have holes or be scrambled.

TCP is the protocol for large amounts of data that need to be kept in order.

Instead of using FTP to transfer the data, try using HTTP and simply keeping the socket open. You can send any amount of data you want, for any length of time you want, without having to pay the latency of opening another connection.

User avatar
BNNorman
 
Posts: 12
Joined: Tue Jan 03, 2023 7:17 am

Re: Matrix Portal UDP server not receiving packets

Post by BNNorman »

Thanks for the reply. I understand the implications of using UDP but TCP was slow.

This is an academic exercise i.e. 'Can I do it or have I wasted my money?' for a retired bloke just pottering in his shed.

As I said, I have gotten this to work on an Interstate Pico W. And yes, I could stop there but when faced with a problem , even if self inflicted, I' m the sort of rottweiler that won't let go till I, hopefully, solve it.

I don't care much about dropped frames and if using an Matrix Portal as an access point at 192.168.4.1 and a client sending on that network (no hubs or routers involved) losses should be minimal. Unfortunately when I tried that the Matrix Portal still didn't hear the UDP transmissions.

I did read the NINA wifi code, which I believe, lives on the ESP32 but only found a method to set the ESP32 as a TCP server - though that code did appear to handle UDP as well (a bit confusing).

I presumed, maybe wrongly, HTTP would be slower. I tried adding image uploading to the wsgi_server example but that failed, probably due to my lack of understanding of what is needed (ajax javascript file uploader?). I guess I could use a multi-part form upload but I wanted a system where the frames could be requested & dislayed at a controlled rate.

At the moment, I have reversed the device roles. The Matrix Portal will be a client and my PC the server so that the MP can instigate comms and the PC can deliver an image frame, but I haven't completed that bit yet.

Although I can transfer video frames at 25fps using an Interstate 75W a much lower rate would be good for a simple slideshow - like having an xmas theme playing at, well, xmas.

Thanks again for your input. It often helps to talk to a human about these things.

Brian

User avatar
adafruit_support_mike
 
Posts: 67446
Joined: Thu Feb 11, 2010 2:51 pm

Re: Matrix Portal UDP server not receiving packets

Post by adafruit_support_mike »

BNNorman wrote: Thu Feb 09, 2023 6:19 am Can I do it
Suitable reason for a project right there. ;-)
BNNorman wrote: Thu Feb 09, 2023 6:19 am I presumed, maybe wrongly, HTTP would be slower.
There's a little more latency in setting up a connection, but the goal is to get a socket and keep it open for as long as possible. Latency becomes unbearable when you have to pay it over and over, but gets proportionally cheaper as you continue to use the open connection.

It's basically the 'never underestimate the bandwidth of a station wagon full of tapes' problem. Connection protocols will probably never be able to deliver data faster than a box full of storage media (originally tapes, then semis full of hard drives, now a briefcase full of SD cards) moving between the same endpoints.

A fiber connection can deliver the first byte within milliseconds, but will still be chugging away when the physical media arrive in 48 hours. Physical media can deliver an essentially unlimited amount of data in 48 hours, but the ping response is terrible.
BNNorman wrote: Thu Feb 09, 2023 6:19 am I guess I could use a multi-part form upload but I wanted a system where the frames could be requested & dislayed at a controlled rate.
Just-in-time data sounds good, but is almost always a headache in practice. The better model is just-ahead-of-time, where you buffer N frames of data ahead. For the setup you describe, transmit in chunks that will display within maybe 100ms, and keep a buffer of maybe 300ms. When you start using data from block-1, begin the request for block-4.

Set your server up as a firehose that keeps sending blocks until the client disconnects. That way, if you start using data from block-2 before all of block-4 has arrived, the client can just keep reading and get block-5 without having to make another request.

As long as you can transmit blocks faster than the system consumes them, your client can stay ahead of the game. You'll still have the flexibility to change what you request in real-time, and you always have the option to dump the whole buffer and load fresh data if you need to.

User avatar
BNNorman
 
Posts: 12
Joined: Tue Jan 03, 2023 7:17 am

Re: Matrix Portal UDP server not receiving packets

Post by BNNorman »

Cheers. I have reversed the roles of my PC and Matrix Portal and gotten them talking to each other using UDP and am now adding in my Matrix handling - so not yet tried to display an image/video with this hardware but hope to at some point later today.

Buffering frames would be something to add if this was to be allowed to get out of the paddock but this is more of an in-home educational thing to keep my brain active. Other septuagenarians do crosswords/books/dog walking, I do hobby programming and hardware stuff/toys (Arduino etc).

It would have been nice to get a UDP server running on the Matrix Portal - to prove it can be done. But the darned thing seems to be deaf. And no-one has provided a working example of how to do it, or even if it can be done. Just about all the Matrix Portal examples I have seen pull data from a website.

I will look at using http later - after revisiting my 'Midi keyboard' made of 8 laser TOF sensors and an I2C multiplexor.

Thanks again.

Locked
Forum rules
If you're posting code, please make sure your code does not include your Adafruit IO Active Key or WiFi network credentials.

Return to “Internet of Things: Adafruit IO and Wippersnapper”