Display NOAA Buoy data on a MagTag?

CircuitPython on hardware including Adafruit's boards, and CircuitPython libraries using Blinka on host computers.

Moderators: adafruit_support_bill, adafruit

Please be positive and constructive with your questions and comments.
User avatar
ghulse
 
Posts: 97
Joined: Tue Nov 30, 2021 10:49 am

Re: Display NOAA Buoy data on a MagTag?

Post by ghulse »

danhalbert wrote: Sat Jan 14, 2023 9:03 pm
You can wrap you current code in an unending loop, and just sleep in between:

Code: Select all

import time

while True:
    # do all the work you're doing now.
    time.sleep(5*60)   # sleep for 5 minutes
    # loop around and go back
If this is a battery-operated device, you can use "deep sleep" to not drain the battery when you're sleeping. This guide talks about that: https://learn.adafruit.com/deep-sleep-w ... cuitpython.
I appreciate your continued efforts trying to prevent me from making the code more complicated than it needs to be. :-)

Does the time.sleep line go at the end of the rest of the code?

Also, I'm getting a message after the buoy info is displayed: code stopped by auto-reload. Reloading soon.

And, then finally, I can now dress up the output using fonts and a bmap background, correct? Also, it wouldn't be very difficult to make this work on a PyPortal?

Thanks again for all your help.

User avatar
danhalbert
 
Posts: 4613
Joined: Tue Aug 08, 2017 12:37 pm

Re: Display NOAA Buoy data on a MagTag?

Post by danhalbert »

Does the time.sleep line go at the end of the rest of the code?
Yes, it's the last statement in the `while True:` loop.
Also, I'm getting a message after the buoy info is displayed: code stopped by auto-reload. Reloading soon.
This is not due to your program. This is due to the host computer writing something to CIRCUITPY a number of seconds after your last edit. If this happens often, it might be due to a utility program (such as an indexing or backup program) on your host computer.
And, then finally, I can now dress up the output using fonts and a bmap background, correct? Also, it wouldn't be very difficult to make this work on a PyPortal?
Yes, to both, and you can probably find examples of that in the PyPortal Learn Guide examples, though many of them use the PyPortal library (which is analogous to the MagTag library and shares a lot of code with it).

User avatar
ghulse
 
Posts: 97
Joined: Tue Nov 30, 2021 10:49 am

Re: Display NOAA Buoy data on a MagTag?

Post by ghulse »

Wrapping the code in the time loop crashes the MagTag. The code runs in MU Editor's serial window, but nothing happens on the MagTag display itself. It says:
"File "code.py" , line 113, in module> KeyboardInterrupt.

Code done running. Press any key to enter the REPL . . ."
Ejecting the MagTag from the Mac, then and turning it off and back on again, same thing on display.

But then going back and removing the time loop from the code (unindenting all the lines that were in the loop), the code works as before and displays the NOAA tide info.

Should all the import statements be in this time loop? How about the Try block that gets info from secrets.py file?

Does this block of code get put in the time loop?

Code: Select all

print("Connecting to %s"%secrets["ssid"])
wifi.radio.connect(secrets["ssid"], secrets["password"])
print("Connected to %s!"%secrets["ssid"])

pool = socketpool.SocketPool(wifi.radio)
requests = adafruit_requests.Session(pool, ssl.create_default_context())

# print("Fetching and parsing NOAA data from Cambridge buoy from", noaa_url)
response = requests.get(noaa_url)

html_text = response.text

User avatar
danhalbert
 
Posts: 4613
Joined: Tue Aug 08, 2017 12:37 pm

Re: Display NOAA Buoy data on a MagTag?

Post by danhalbert »

ghulse wrote: Sun Jan 15, 2023 12:48 pm Should all the import statements be in this time loop? How about the Try block that gets info from secrets.py file?

Does this block of code get put in the time loop?
[...]
Put the imports and the wifi initalization code above the `while True:` loop. Put the fetching of the Buoy data and its display inside the loop. Then put a sleep at the end of that code, so that it will sleep before it goes around again and fetches the data again.

User avatar
ghulse
 
Posts: 97
Joined: Tue Nov 30, 2021 10:49 am

Re: Display NOAA Buoy data on a MagTag?

Post by ghulse »

danhalbert wrote: Sun Jan 15, 2023 3:37 pm
Put the imports and the wifi initalization code above the `while True:` loop. Put the fetching of the Buoy data and its display inside the loop. Then put a sleep at the end of that code, so that it will sleep before it goes around again and fetches the data again.
That's how I had it to start with. But when I update the code the Magtag screen seems frozen. And there's a message: "Code stopped by auto-reload. Reloading soon." The MagTag remains unresponsive when I disconnect from the Mac.

When I plug back into Mac, this message appears on the MagTag screen:

File "adafruit_requests.py", line 710, in request
File "adafruit_requests.py", line 554, in _get_socket
OSError: -2

And, yet, if I check the MU Editor serial output, it shows normal output from the NOAA buoy. And it updates every five minutes just like it's supposed to.

Here's the entire code:

Code: Select all

# SPDX-FileCopyrightText: 2020 Brent Rubell for Adafruit Industries
#
# SPDX-License-Identifier: MIT

import time
import ipaddress
import ssl
import wifi
import socketpool
import adafruit_requests


# URLs to fetch from
noaa_url = "https://www.ndbc.noaa.gov/rss/ndbc_obs_search.php?lat=38.574N&lon=76.069W"

# Get wifi details and more from a secrets.py file
try:
    from secrets import secrets
except ImportError:
    print("WiFi secrets are kept in secrets.py, please add them there!")
    raise

print("Connecting to %s" % secrets["ssid"])
wifi.radio.connect(secrets["ssid"], secrets["password"])
print("Connected to %s!" % secrets["ssid"])

pool = socketpool.SocketPool(wifi.radio)
requests = adafruit_requests.Session(pool, ssl.create_default_context())

while True:

    # print("Fetching and parsing NOAA data from Cambridge buoy from", noaa_url)
    response = requests.get(noaa_url)

    html_text = response.text

    # -- Find the substring between the start and end strings
    ##-- Find the substring between the start and end strings
    start_string = "<description>"
    end_string = "</description>"
    start_index = html_text.find(start_string)
    start_index = html_text.find(start_string, start_index + len(start_string))
    end_index = html_text.find(end_string, start_index + len(start_string))

    desc = html_text[start_index : end_index + start_index]
    rows = desc.split("\n")

    ##-- set variables
    date = [i for i in rows if "EST" in i.upper()][0]
    date = date.strip()
    date = date.replace("<strong>", "")
    date = date.replace(" EST</strong><br />", "")
    print(f"Station CAMM2 - Cambridge, MD - {date}")

    direction = [i for i in rows if "direction" in i.lower()][0]
    direction = direction.strip()
    direction = direction.replace("<strong>Wind Direction:</strong> ", "")
    direction = direction.replace("<br />", "")
    direction = direction.replace("&#176;", chr(176))
    # print(f"Direction: {direction}")

    speed = [i for i in rows if "speed" in i.lower()][0]
    speed = speed.strip()
    speed = speed.replace("<strong>Wind Speed:</strong> ", "")
    speed = speed.replace("<br />", "")
    # print(f"Wind speed: {speed}")

    gusts = [i for i in rows if "gust" in i.lower()][0]
    gusts = gusts.strip()
    gusts = gusts.replace("<strong>Wind Gust:</strong> ", "")
    gusts = gusts.replace("<br />", "")
    # print(f"Gusting: {gusts}")
    print(f"Wind from the {direction}")
    print(f"{speed}, gusting to {gusts}")

    pressure = [i for i in rows if "pressure" in i.lower()][0]
    pressure = pressure.strip()
    pressure = pressure.replace("<strong>Atmospheric Pressure:</strong> ", "")
    pressure = pressure.replace("<br />", "")
    print(f"Pressure: {pressure}")

    trend = [i for i in rows if "tendency" in i.lower()]
    if len(trend) == 0:
        trend = ""
        print(f"Trend: {trend}--")
    else:
        trend = trend[0]
        trend = trend.strip()
        trend = trend.replace("<strong>Pressure Tendency:</strong> ", "")
        trend = trend.replace("<br />", "")
        print(f"Trend: {trend}")

    air_temp = [i for i in rows if "air" in i.lower()][0]
    air_temp = air_temp.strip()
    air_temp = air_temp.replace("<strong>Air Temperature:</strong> ", "")
    air_temp = air_temp.replace("&#176;", chr(176))
    ##--remove celsius part
    separator = "F"
    air_temp = air_temp.split(separator, 1)[0] + separator
    # print(f"Air Temperature: {air_temp}")

    water_temp = [i for i in rows if "water" in i.lower()][0]
    water_temp = water_temp.strip()
    water_temp = water_temp.replace("<strong>Water Temperature:</strong> ", "")
    water_temp = water_temp.replace("&#176;", chr(176))
    ##-- remove celsius part
    separator = "F"
    water_temp = water_temp.split(separator, 1)[0] + separator
    print(f"Air Temperature: {air_temp} - Water Temperature {water_temp}")

    time.sleep(5*60)   # sleep for 5 minutes

User avatar
danhalbert
 
Posts: 4613
Joined: Tue Aug 08, 2017 12:37 pm

Re: Display NOAA Buoy data on a MagTag?

Post by danhalbert »

I tried this myself with your program, with the MagTag plugged into my Linux computer. It printed the buoy info on the host computer, waited 5 minutes and printed the buoy info again. I did not see the errors you are seeing. It is possible the host computer is writing occasionally to the CIRCUITPY drive.

The MagTag does not display what is being printed in real time. This is because it's an e-ink display, not a regular display, and it can easily wear out if you change it too often.

You probably want to display the buoy info on the MagTag screen, instead of just printing it. Then you could plug the MagTag into a USB power adapter, or use a battery, and have it not tethered to the host computer. This is the primary use case for the MagTag. I suggest you look through the MagTag projects and pick a simple one. I am not very familiar with the MagTag library, which could help you do the whole thing. I will try to find someone else to pick this up.

User avatar
ghulse
 
Posts: 97
Joined: Tue Nov 30, 2021 10:49 am

Re: Display NOAA Buoy data on a MagTag?

Post by ghulse »

Thanks for all your help. My first goal was to get some basic code working on the MagTag before I try to figure out how to display it nicely. But since the buoy info has to be updated so frequently, the PyPortal may be a better choice for this project. So I'll put the NOAA tide project back on the MagTag, which is what I've been using it for the last couple of years. I believe the tide code only has to connect to the internet once a day, which is why the MagTag can go several weeks on a single charge.

You may recall helping me get the moon phase code (originally written for the Adafruit Matrix Portal) working on the PyPortal. That still works great by the way.

Cheers, and thanks again.

User avatar
danhalbert
 
Posts: 4613
Joined: Tue Aug 08, 2017 12:37 pm

Re: Display NOAA Buoy data on a MagTag?

Post by danhalbert »

You're welcome! Glad to help.

User avatar
Hingeway
 
Posts: 2
Joined: Fri Aug 13, 2021 2:23 pm

Re: Display NOAA Buoy data on a MagTag?

Post by Hingeway »

Here’s an article on getting buoy data: Retrieving Data from National Data Buoy Center API
https://medium.com/@holtan.chase/retrie ... 4d262c7ea7

Although the article says there is an API, it’s really just a text file of tabular data. The most recent metrics are the top data line.

Buoy data files are available using this URL format:
https://www.ndbc.noaa.gov/data/realtime ... ATION#.txt
for Cambridge, MD:
https://www.ndbc.noaa.gov/data/realtime2/CAMM2.txt

I tried the article’s Python code on Windows and it worked fine. The file downloaded was well over 900K. Whoa! Too much for a little microcontroller board. My PyPortal crashed with a MemoryError when I ran the equivalent code on it.

I modified the code to only capture the top few lines of the file and then stop the retrieval. It works fine on my PyPortal and only requires a couple hundred bytes of memory for the data.

Code: Select all

# Gets and parses a file with buoy metrics.
# * Only the most recent data line is parsed.
#   The remaining portion of file is ignored and won't consume memory.

import time
import board
from digitalio import DigitalInOut
from adafruit_esp32spi import adafruit_esp32spi
import adafruit_esp32spi.adafruit_esp32spi_socket as socket
import adafruit_requests as requests

# If you are using a board with pre-defined ESP32 Pins, like the PyPortal:
esp32_cs    = DigitalInOut(board.ESP_CS)
esp32_ready = DigitalInOut(board.ESP_BUSY)
esp32_reset = DigitalInOut(board.ESP_RESET)

# SPI Configuration
spi = board.SPI()
esp = adafruit_esp32spi.ESP_SPIcontrol(spi, esp32_cs, esp32_ready, esp32_reset)
requests.set_socket(socket, esp)

# Wifi credentials are stored in secrets.py file
try:
    from secrets import secrets
except ImportError:
    print("WiFi credentials are kept in secrets.py, please add them there!")
    raise

# Various buoy file URLs
#FILE_URL = "https://www.ndbc.noaa.gov/data/realtime2/FTPC1.txt"       # San Francisco, Golden Gate
#FILE_URL = "https://www.ndbc.noaa.gov/data/realtime2/46026.txt"       # San Francisco, Outside Bay
#FILE_URL = "https://www.ndbc.noaa.gov/data/realtime2/51211.txt"       # Pearl Harbor entrance
#FILE_URL = "https://www.ndbc.noaa.gov/data/realtime2/46025.txt"       # Santa Monica, CA
FILE_URL = "https://www.ndbc.noaa.gov/data/realtime2/CAMM2.txt"       # Cambridge, MD

TOP_PORTION      = 320     # Top portion of file to retrieve, so RAM is not overwhelmed
REFRESH_INTERVAL = 60      # Number of seconds between refreshes from file

while True:
    while not esp.is_connected:
        print("\nConnecting to AP... ", end="")
        try:
            esp.connect_AP(secrets["ssid"], secrets["password"])
            print("connected")
        except ConnectionError as e:
            print("could not connect to AP retrying: ", e)
            continue

    file_portion = b''    # Stores the obtained file chunks

    # Accumulate the chunks until a little past the first data line.
    # When examined during testing the first data line ended at byte 282.
    print("\nFetching text from", FILE_URL)
    response = requests.get(FILE_URL)
    for file_chunk in response.iter_content(chunk_size=64):
        if len(file_portion) <= TOP_PORTION:
            file_portion = file_portion + file_chunk 
        else:
            # Break out of iterator with a hack
            # Doesn't appear to be a friendly way to stop iterator.
            # A simple break leaves the response waiting to time out (3 minutes for me).
            # Even a response.close() waits to time out.
            # Retrieving a chunk_size of zero seems to do the job.
            file_chunk in response.iter_content(chunk_size=0)
            break
    response.close()

    # The most recent data is on the second file line, prior lines are headers
    # (::) is my rendition of a buoy.
    file_lines = file_portion.decode('utf-8').split(u'\u000A')
    parts = file_lines[2].split()
    date_time = "{}-{}-{} {}:{}".format( parts[0], parts[1], parts[2], parts[3], parts[4] )
    print( "{} \n(::) Air Pres: {} hPa, Water Temp: {} C".format(date_time,parts[12],parts[14]) )

    print("Sleeping... ", end="")
    sleep_start = time.monotonic()
    time.sleep(REFRESH_INTERVAL)
    sleep_end = time.monotonic()
    slept = round(sleep_end - sleep_start,0)
    print("slept for %d seconds" % slept)

User avatar
ghulse
 
Posts: 97
Joined: Tue Nov 30, 2021 10:49 am

Re: Display NOAA Buoy data on a MagTag?

Post by ghulse »

Wow, thanks for posting this. I've been working on this for several weeks. Just today I finally got the code to display data from the Cambridge buoy on the pyportal. But I still hadn't figured out how to make it refresh every 10 minutes or so. I'll check out your code right now. Thanks again for posting.

User avatar
Hingeway
 
Posts: 2
Joined: Fri Aug 13, 2021 2:23 pm

Re: Display NOAA Buoy data on a MagTag?

Post by Hingeway »

I updated my program for retrieving buoy data because of a reliability problem. The PyPortal didn’t have a problem but when I tried my Feather M4 it produced a MemoryError on the line: file_chunk in response.iter_content(chunk_size=0)

I never liked that hack because I didn’t understand why it worked. Anyway, I discovered there are better data sources.

This directory contains many NDBC data sources:
https://www.ndbc.noaa.gov/data/

The latest_obs subdirectory has files with just the latest observations. There are .rss and .txt versions.
https://www.ndbc.noaa.gov/data/latest_obs/

Latest observations for CAMM2 (note lowercase name in URL):
https://www.ndbc.noaa.gov/data/latest_obs/camm2.txt

The data appears to be the same as displayed by the NDBC widgets.
https://www.ndbc.noaa.gov/widgets/
https://www.ndbc.noaa.gov/widgets/stati ... tion=CAMM2

The big advantage of this data source is that a micro-controller can read the entire file into memory without a problem.

The metrics in the file can vary. Local time can even be missing. GMT seems to be guaranteed unless zero data was collected. A file may even have a Wave Summary section with wave related metrics.

The program is modified to use the smaller file with selected metrics from the latest observations. This version requires additional parsing of the file to get the data into variables. I ran the program for 12 hours on a PyPortal and Feather M4. Neither had any problem retrieving the data.

Code: Select all

# Gets and parses a file with buoy metrics.
# * The file contains only the latest observations for a station.
# * This is the data displayed by the NDBC widget.
# * Data Sources: https://www.ndbc.noaa.gov/data/latest_obs/

import busio
import time
import board
import neopixel
import re
from digitalio import DigitalInOut
from adafruit_esp32spi import adafruit_esp32spi
from adafruit_esp32spi import adafruit_esp32spi_wifimanager

# Wifi credentials are stored in secrets.py file
try:
    from secrets import secrets
except ImportError:
    print("WiFi credentials are kept in secrets.py, please add them there!")
    raise

# Latest observations files for buoys
# * Unlike the 5- and 45-day files these file URLs are lowercase -- uppercase file name won't work.
#FILE_URL = "https://www.ndbc.noaa.gov/data/latest_obs/13002.txt"       # Atlas moored buoy
#FILE_URL = "https://www.ndbc.noaa.gov/data/latest_obs/ftpc1.txt"       # San Francisco, Golden Gate
#FILE_URL = "https://www.ndbc.noaa.gov/data/latest_obs/46026.txt"       # San Francisco, outside bay
#FILE_URL = "https://www.ndbc.noaa.gov/data/latest_obs/51211.txt"       # Pearl Harbor entrance
#FILE_URL = "https://www.ndbc.noaa.gov/data/latest_obs/46025.txt"       # Santa Monica, CA
FILE_URL = "https://www.ndbc.noaa.gov/data/latest_obs/camm2.txt"        # Cambridge, MD

REFRESH_INTERVAL = 360      # Number of seconds between refreshes from file

# Extract some key metrics from the file lines.
# * The first two lines are treated as a fixed header with remaining lines being variable.
#   Time will tell if this is a valid assumption.
# * Some files contain more metrics than the ones parsed by this function.
#   Local time can even be missing. GMT can be missing if there is no data at all.
#   A file may even have a Wave Summary section with wave related metrics.
def parse_file(file_lines):

    # Probably would pass some type of structure for data collection, globals will do for now
    global station_id
    global lat_degrees
    global lat_minutes
    global lat_compass
    global lon_degrees
    global lon_minutes
    global lon_compass
    global local_time
    global local_ampm
    global local_timezone
    global gmt_time
    global gmt_timezone
    global gmt_date
    global wind_direction_compass
    global wind_direction_degrees
    global wind_speed
    global wind_speed_measure
    global gust_speed
    global gust_speed_measure
    global pressure_sealevel
    global air_temperature
    global air_temperature_measure
    global water_temperature
    global water_temperature_measure

    # Station identifier expected in line 0
    parts = file_lines[0].split()
    if parts[0] == "Station":
        station_id = parts[1]

    # Lat/Lon expected in line 1
    parts = file_lines[1].split()
    lat_degrees = parts[0]
    lat_minutes = parts[1]
    lat_compass = parts[2]
    lon_degrees = parts[3]
    lon_minutes = parts[4]
    lon_compass = parts[5]

    # Remaining portion with metrics is variable.
    for i in range(2,len(file_lines)):

        parts = file_lines[i].split()
        if len(parts) == 0:
            # There's nothing here to see, move along folks.
            continue

        # Local time
        if parts[1] == "am" or parts[1] == "pm" :
            local_time = parts[0]
            local_ampm = parts[1]
            local_timezone = parts[2]
            continue

        # GMT/UTC time
        if parts[1] == "GMT":
            gmt_time = parts[0]
            gmt_timezone = parts[1]
            gmt_date = parts[2]
            continue

        if parts[0] == "Wind:":
            wind_direction_compass = parts[1]
            regex = re.compile("[(),°]")
            wind_direction_degrees = regex.split(parts[2])[1]
            wind_speed = parts[3]
            wind_speed_measure = parts[4]
            continue

        if parts[0] == "Gust:":
            gust_speed = parts[1]
            gust_speed_measure = parts[2]
            continue

        if parts[0] == "Pres:":
            pressure_sealevel = parts[1]
            continue

        if parts[0] == "Air":
            air_temperature = parts[2]
            air_temperature_measure = parts[3]
            continue

        if parts[0] == "Water":
            water_temperature = parts[2]
            water_temperature_measure = parts[3]
            continue

#end parse_file()

# Get the Posix timestamp for the date and time arguments.
# * GMT date and time are the expected arguments. DST is hardcoded as -1 since it doesn't apply to GMT.
# * Date expected format is: "DD/MM/YY", example "02/06/23"
# * Time expected format is: "HHMM", example "1830" or "0120"
# * Year is expected to be in 21st century because time.mktime can only handle dates after Jan 1, 2000.
def get_gmt_timestamp( gmt_date, gmt_time ):
    # Parse date
    parts = gmt_date.split('/')
    gmt_month = int(parts[0])
    gmt_day   = int(parts[1])
    gmt_year  = 2000 + int(parts[2])
    # Parse time
    gmt_hour   = int(gmt_time[0:2])
    gmt_minute = int(gmt_time[2:])
    
    gmt_struct = time.struct_time( (gmt_year, gmt_month, gmt_day, gmt_hour, gmt_minute, 0, 0, -1, -1,) )

    return time.mktime(gmt_struct)
#end get_gmt_timestamp()

# If you are using a board with pre-defined ESP32 Pins, like the PyPortal:
esp32_cs    = DigitalInOut(board.ESP_CS)
esp32_ready = DigitalInOut(board.ESP_BUSY)
esp32_reset = DigitalInOut(board.ESP_RESET)
spi = board.SPI()

# Feather M4 with AirLift
#esp32_cs = DigitalInOut(board.D13)
#esp32_ready = DigitalInOut(board.D11)
#esp32_reset = DigitalInOut(board.D12)
#spi = busio.SPI(board.SCK, board.MOSI, board.MISO)

# ESP32 Wifi coprocessor SPI Configuration
esp = adafruit_esp32spi.ESP_SPIcontrol(spi, esp32_cs, esp32_ready, esp32_reset)

# Wifi manager maintains the connection
status_light = neopixel.NeoPixel( board.NEOPIXEL, 1, brightness=0.2 )
wifi = adafruit_esp32spi_wifimanager.ESPSPI_WiFiManager(esp, secrets, status_light)

# Optional: Debug information on retrieval
#esp._debug = True

while True:

    # Retrieve the entire file of latest observations
    print("\nFetching file from", FILE_URL)
    response = wifi.get(FILE_URL)

    # File contains a Unicode degree symbol that must be replaced
    new_content = response.content.replace(b'\xb0',b'°')

    # Convert to list
    file_lines = new_content.decode('utf-8').split(u'\u000A')

    # Initialize collection variables
    # * Any metric not in the file retains the Missing Measurement designator
    station_id = "Missing"
    lat_degrees = "MM"
    lat_minutes = "MM"
    lat_compass = "MM"
    lon_degrees = "MM"
    lon_minutes = "MM"
    lon_compass = "MM"
    local_time = "MM"
    local_ampm = "MM"
    local_timezone = "MM"
    gmt_time = "MM"
    gmt_timezone = "MM"
    gmt_date = "MM"
    wind_direction_compass = "MM"
    wind_direction_degrees = "MM"
    wind_speed = "MM"
    wind_speed_measure = ""
    gust_speed = "MM"
    gust_speed_measure = ""
    pressure_sealevel = "MM"
    air_temperature = "MM"
    air_temperature_measure = ""
    water_temperature = "MM"
    water_temperature_measure = ""

    # Parse out the data to the global collection variables
    parse_file( file_lines )

    # Do something with the parsed data
    print()
    print("Station is", station_id)
    print("Latitude: {} {} {}, Longitude: {} {} {}".format(lat_degrees, lat_minutes, lat_compass, lon_degrees, lon_minutes, lon_compass))
    print("Local time is {} {} {}".format(local_time, local_ampm, local_timezone))
    print("{} is {} {}".format(gmt_timezone, gmt_date, gmt_time))
    print("Wind is {} {} at bearing {} ({}°)".format(wind_speed, wind_speed_measure, wind_direction_compass, wind_direction_degrees))
    print("Gust is {} {}".format(gust_speed, gust_speed_measure))
    print("Pressure at sea level: {} inHg".format(pressure_sealevel))
    print("Air temperature: {} {}".format(air_temperature, air_temperature_measure))
    print("Water temperature: {} {}".format(water_temperature, water_temperature_measure))
    
    # Convert GMT date/time to timestamp
    # * Useful when storing data in database
    gmt_timestamp = get_gmt_timestamp(gmt_date,gmt_time)
    print()    
    print("GMT timestamp: {}".format(gmt_timestamp))
    print("GMT struct_time: {}".format(time.localtime(gmt_timestamp)))

    # Take a rest
    print()        
    print("Sleeping... ", end="")
    sleep_start = time.monotonic()
    time.sleep(REFRESH_INTERVAL)
    sleep_end = time.monotonic()
    slept = round(sleep_end - sleep_start,0)
    print("slept for %d seconds" % slept)

User avatar
ghulse
 
Posts: 97
Joined: Tue Nov 30, 2021 10:49 am

Re: Display NOAA Buoy data on a MagTag?

Post by ghulse »

I've been using the rss version of the Cambridge buoy data, but even that has a lot of extraneous characters that have to be parsed out. The .txt version is much better. Thanks for mentioning that.

My current version works pretty well, though I'm still tweaking how the info gets displayed on the PyPortal screen. Here's what it looks like today.

Image

Of concern, my version still crashes about once a day. (Usually "Runtime error: ESP32 timed out on SPI select" or "OutOfRetries: repeated socket failures"). I'm using Try-Except to handle when data is missing. Not sure if this is the best way or not.

I suspect my version of the code is amateurish compared to Hingeway's. I'm not familiar with global variables, so it might take me a while to understand his code. But here's what I have currently.

Code: Select all

# Gets and parses NOAA buoy data
# then displays on the pyportal

import time
import board
import busio
import displayio
from digitalio import DigitalInOut
import adafruit_requests as requests
import adafruit_esp32spi.adafruit_esp32spi_socket as socket
from adafruit_esp32spi import adafruit_esp32spi
from adafruit_bitmap_font import bitmap_font
from adafruit_display_text import label
import gc

# Get wifi details and more from a secrets.py file
try:
    from secrets import secrets
except ImportError:
    print("WiFi secrets are kept in secrets.py, please add them there!")
    raise

# Set text, font, and color and !
background_image = open("images/buoy_background.bmp", "rb")
font = bitmap_font.load_font("fonts/Arial-12.bdf")
color = 0xFFFFFF

display = board.DISPLAY
noaa_url = 'https://www.ndbc.noaa.gov/data/latest_obs/camm2.rss'
# If you are using a board with pre-defined ESP32 Pins:
esp32_cs = DigitalInOut(board.ESP_CS)
esp32_ready = DigitalInOut(board.ESP_BUSY)
esp32_reset = DigitalInOut(board.ESP_RESET)

spi = busio.SPI(board.SCK, board.MOSI, board.MISO)
esp = adafruit_esp32spi.ESP_SPIcontrol(spi, esp32_cs, esp32_ready, esp32_reset)

requests.set_socket(socket, esp)

print("Connecting to AP...")
while not esp.is_connected:
    try:
        esp.connect_AP(secrets["ssid"], secrets["password"])
    except OSError as e:
        print("could not connect to AP, retrying: ", e)
        continue

while True:
    # open rss feed and start parsing text
    print("Getting data from NOAA...")
    r = requests.get(noaa_url)
    #try:
    #    r = requests.get(noaa_url)
    #except (ValueError, RuntimeError, ConnectionError, OSError) as e:
    #    print("Failed to get data, retrying\n", e)
    #    esp.reset()
    #    continue
    text = ""
    gc.collect()
    text = r.text
    r.close()

    # Find the substring between the start and end strings
    # Ned was here!
    start_string = '<description>'
    end_string = "</description>"
    start_index = text.find(start_string)
    start_index = text.find(start_string, start_index + len(start_string))
    end_index = text.find(end_string, start_index + len(start_string))

    desc = text[start_index: end_index+start_index]
    rows = desc.split('\n')

    # set variables
    try:
        date = [i for i in rows if 'EST' in i.upper()][0]
        date = date.strip()
        date = date.replace("<strong>", "")
        date = date.replace(" EST</strong><br />", "")
        date = (date[-8:])
        date = "last updated " + date
    except:
        date = '--'

    try:
        direction = [i for i in rows if 'direction' in i.lower()][0]
        direction = direction.strip()
        direction = direction.replace("<strong>Wind Direction:</strong> ", "")
        direction = direction.replace("<br />", "")
        direction = direction.replace("&#176;", chr(176))
        direction = "from " + direction
    except:
        direction = "from " + '--'

    try:
        speed = [i for i in rows if 'speed' in i.lower()][0]
        speed = speed.strip()
        speed = speed.replace("<strong>Wind Speed:</strong> ", "")
        speed = speed.replace("<br />", "")
        speed = speed + ", gusting "
    except:
        speed = '--'

    try:
        gusts = [i for i in rows if 'gust' in i.lower()][0]
        gusts = gusts.strip()
        gusts = gusts.replace("<strong>Wind Gust:</strong> ", "")
        gusts = gusts.replace("<br />", "")
    except:
        gusts = '--'

    try:
        pressure = [i for i in rows if 'pressure' in i.lower()][0]
        pressure = pressure.strip()
        pressure = pressure.replace("<strong>Atmospheric Pressure:</strong> ", "")
        pressure = pressure.replace("<br />", "")
        pressure = (pressure[-10:-1])
    except:
        pressure = '--'

    try:
        trend = [i for i in rows if 'tendency' in i.lower()]
        trend = trend[0]
        trend = trend.strip()
        trend = trend.replace("<strong>Pressure Tendency:</strong> ", "")
        trend = trend.replace("<br />", "")
        trend = (trend[-8:-1])
        trend = "trend " + trend
    except:
        trend = '--'
        trend = "trend " + trend

    try:
        air_temp = [i for i in rows if 'air' in i.lower()][0]
        if len(air_temp) == 0:
            air_temp = '--'
        else:
            air_temp = air_temp.strip()
            air_temp = air_temp.replace("<strong>Air Temperature:</strong> ", "")
            air_temp = air_temp.replace("&#176;", chr(176))
            # remove celsius part
            separator = 'F'
            air_temp = air_temp.split(separator, 1)[0] + ' ' + separator
            air_temp = 'Air: ' + air_temp
    except:
        air_temp = '--'

    try:
        water_temp = [i for i in rows if 'water' in i.lower()][0]
        water_temp = water_temp.strip()
        water_temp = water_temp.replace("<strong>Water Temperature:</strong> ", "")
        water_temp = water_temp.replace("&#176;", chr(176))
        # remove celsius part
        separator = 'F'
        water_temp = water_temp.split(separator, 1)[0] + ' ' + separator
        water_temp = 'Water : ' + water_temp
    except:
        water_temp = '--'

    # Create the text label
    text_area = label.Label(font, text=date, color=color)
    text_area2 = label.Label(font, text=direction, color=color)
    text_area3 = label.Label(font, text=speed, color=color)
    text_area4 = label.Label(font, text=gusts, color=color)
    text_area5 = label.Label(font, text=pressure, color=color)
    text_area6 = label.Label(font, text=trend, color=color)
    text_area7 = label.Label(font, text=air_temp, color=color)
    text_area8 = label.Label(font, text=water_temp, color=color)

    # Set the locations
    text_area.x = 82 # date
    text_area.y = 40

    text_area2.x = 50 # direction
    text_area2.y = 146

    text_area3.x = 50 # speed
    text_area3.y = 167

    text_area4.x = 190 # gusts
    text_area4.y = 167

    text_area5.x = 50 # pressure
    text_area5.y = 218

    text_area6.x = 155 # trend
    text_area6.y = 218

    text_area7.x = 50 # air temp
    text_area7.y = 95

    text_area8.x = 148 # water temp
    text_area8.y = 95

    # Setup the file as the bitmap data source
    bitmap = displayio.OnDiskBitmap("images/buoy_background.bmp")

    # Create a TileGrid to hold the bitmap
    tile_grid = displayio.TileGrid(bitmap, pixel_shader=bitmap.pixel_shader)

    # create group to hold text areas
    text_group = displayio.Group()
    text_group.append(tile_grid)
    text_group.append(text_area)
    text_group.append(text_area2)
    text_group.append(text_area3)
    text_group.append(text_area4)
    text_group.append(text_area5)
    text_group.append(text_area6)
    text_group.append(text_area7)
    text_group.append(text_area8)

    # Show it
    display.show(text_group)

    time.sleep(5*60)   # sleep for 5 minutes
Attachments
pyportal.jpg
pyportal.jpg (101.34 KiB) Viewed 94 times

User avatar
ghulse
 
Posts: 97
Joined: Tue Nov 30, 2021 10:49 am

Re: Display NOAA Buoy data on a MagTag?

Post by ghulse »

I simplified this project to display exactly as NOAA has it on this feed . . .

https://www.ndbc.noaa.gov/data/latest_obs/camm2.txt

As such it's easy to change to another buoy. At night, the GMT date shifts to the EST field for some reason.

Code: Select all

# Gets and parses NOAA buoy data
# then displays on the pyportal

import time
import board
import busio
import displayio
from digitalio import DigitalInOut
import adafruit_requests as requests
import adafruit_esp32spi.adafruit_esp32spi_socket as socket
from adafruit_esp32spi import adafruit_esp32spi
from adafruit_bitmap_font import bitmap_font
from adafruit_display_text import label
import gc

# Get wifi details and more from a secrets.py file
try:
    from secrets import secrets
except ImportError:
    print("WiFi secrets are kept in secrets.py, please add them there!")
    raise

# Set text, font, and color and !
#background_image = open("images/buoy_background.bmp", "rb")
font = bitmap_font.load_font("fonts/Arial-12.bdf")
#font2 = bitmap_font.load_font("fonts/cq-mono-30.bdf")
color = 0xFFFFFF

display = board.DISPLAY
noaa_url = 'https://www.ndbc.noaa.gov/data/latest_obs/camm2.txt'
# If you are using a board with pre-defined ESP32 Pins:
esp32_cs = DigitalInOut(board.ESP_CS)
esp32_ready = DigitalInOut(board.ESP_BUSY)
esp32_reset = DigitalInOut(board.ESP_RESET)

spi = busio.SPI(board.SCK, board.MOSI, board.MISO)
esp = adafruit_esp32spi.ESP_SPIcontrol(spi, esp32_cs, esp32_ready, esp32_reset)

requests.set_socket(socket, esp)

print("Connecting to AP...")
while not esp.is_connected:
    try:
        esp.connect_AP(secrets["ssid"], secrets["password"])
    except OSError as e:
        print("could not connect to AP, retrying: ", e)
        continue

while True:
    # open rss feed and start parsing text
    print("Getting data from NOAA...")
    r = requests.get(noaa_url)

    # File contains a Unicode degree symbol that must be replaced
    new_content = r.content.replace(b'\xb0',b'°')

    # Convert to list, split every \n
    lines = new_content.decode('utf-8').split(u'\u000A')
    r.close()

    # set variables
    station = lines[0]
    coordinates = lines[1]
    blank_line = lines[2]
    updated = lines[3]
    gmt_time = lines [4] 
    wind = lines[5]
    gust = lines[6]
    pressure = lines[7]
    air_temp = lines[8]
    water_temp = lines[9]

    # Create the text label
    text_area0 = label.Label(font, text=station, color=color)
    text_area1 = label.Label(font, text=coordinates, color=color)
    text_area2 = label.Label(font, text=blank_line, color=color)
    text_area3 = label.Label(font, text=updated, color=color)
    text_area4 = label.Label(font, text=gmt_time, color=color)
    text_area5 = label.Label(font, text=wind, color=color)
    text_area6 = label.Label(font, text=gust, color=color)
    text_area7 = label.Label(font, text=pressure, color=color)
    text_area8 = label.Label(font, text=air_temp, color=color)
    text_area9 = label.Label(font, text=water_temp, color=color)

    # Set the locations
    text_area0.x = 30 # station
    text_area0.y = 20

    text_area1.x = 30 # buoy coordinates
    text_area1.y = 40

    text_area2.x = 30 # blank line
    text_area2.y = 60

    text_area3.x = 30 # updated
    text_area3.y = 80

    text_area4.x = 30 # gmt time
    text_area4.y = 100

    text_area5.x = 30 # wind
    text_area5.y = 120

    text_area6.x = 30 # gust
    text_area6.y = 140

    text_area7.x = 30 # pressure
    text_area7.y = 160

    text_area8.x = 30 # air temp
    text_area8.y = 180

    text_area9.x = 30 # water temp
    text_area9.y = 200

    # Setup the file as the bitmap data source (for when using background imnage
    #bitmap = displayio.OnDiskBitmap("images/buoy_background.bmp")

    # Create a TileGrid to hold the bitmap
    #tile_grid = displayio.TileGrid(bitmap, pixel_shader=bitmap.pixel_shader)

    # create group to hold text areas
    text_group = displayio.Group()
    #text_group.append(tile_grid)
    text_group.append(text_area0)
    text_group.append(text_area1)
    text_group.append(text_area2)
    text_group.append(text_area3)
    text_group.append(text_area4)
    text_group.append(text_area5)
    text_group.append(text_area6)
    text_group.append(text_area7)
    text_group.append(text_area8)
    text_group.append(text_area9)

    # Show it
    display.show(text_group)

    time.sleep(5*60)   # sleep for 5 minutes
Image
Attachments
noaa_buoy.jpg
noaa_buoy.jpg (70.45 KiB) Viewed 71 times

Locked
Please be positive and constructive with your questions and comments.

Return to “Adafruit CircuitPython”