Run code.py from SD card

CircuitPython on hardware including Adafruit's boards, and CircuitPython libraries using Blinka on host computers.

Moderators: adafruit_support_bill, adafruit

Please be positive and constructive with your questions and comments.
User avatar
mikeysklar
 
Posts: 13979
Joined: Mon Aug 01, 2016 8:10 pm

Re: Run code.py from SD card

Post by mikeysklar »

Cool. Nice setup you have there with the SD init stub code and your conditional menu for importing different types of code.

As for the delay, let's figure that out.

Are you printing anything at the top of program.py to see when it starts? It is not clear to me if the code you are importing is slow to load of if there is a 75 second time out before it even tried to load. Can you share the program.py code?

What size and model of SD card are you using?

User avatar
blakebr
 
Posts: 960
Joined: Tue Apr 17, 2012 6:23 pm

Re: Run code.py from SD card

Post by blakebr »

Mike,

Thanks for the compliment.

I do print stuff very near the top of the ~700 line imported program as you can see from below. The problem waits for 75 seconds before it prints.

I tried importing a one line program. (print("One Line Of Code!")) It ran in less than 5 seconds.
I also tried putting /sd/lib/ at the start of the .path list. No change.

I am using a 32GB SD card. I use this device to try different things. Some for a final project, some to learn stuff. I decided to explore putting the library and CP stuff on the SD card because the RPi Pico and RPi Pico W have so little flash memory.

The startup credits are no longer accurate, but they are what I started with, so credit must be given.

Bruce

Code: Select all

"""
  Internet Clock using Maker Pi Pico and CircuitPython
  Items:
  – Maker Pi Pico
    https://my.cytron.io/p-maker-pi-pico
  – ESP8266 ESP-01 WiFi Serial Transceiver Module
    https://my.cytron.io/p-esp-01-wifi-serial-transceiver-module-esp8266
*  – 4 In 1 MAX7219 Dot Matrix Display Module
*    https://my.cytron.io/p-4-in-1-max7219-dot-matrix-display-module
  – USB Micro B Cable
    https://my.cytron.io/p-usb-micro-b-cable
  Libraries required from bundle (https://circuitpython.org/libraries):
  – adafruit_espatcontrol
*  – adafruit_max7219
  – adafruit_framebuf.mpy
  – adafruit_requests.mpy
  – simpleio.mpy
  – font5x8.bin (in examples folder)
  References:
  – https://tutorial.cytron.io/2019/10/23/dot-matrix-clock-with-ntp-server-using-esp32/
  Last update: 3 Jan 2022
"""

# Intrinsic Libraries
import rtc
import sys
import json
import time
import math
import busio
import busio as busio
import board
from   board import *
from   struct import unpack
import random
import storage
import builtins
import digitalio
from   digitalio import *
import supervisor
from   neopixel_write import neopixel_write
import microcontroller
try: # for FRAM
    from digitalio import DigitalInOut
    from busio import I2C, SPI
except ImportError:
    pass

# Extrinsic Libraries
import simpleio
import adafruit_ntp
import adafruit_fram
import adafruit_ds3231
from   adafruit_bme280 import advanced as adafruit_bme280
import adafruit_sdcard
import adafruit_ssd1306
import adafruit_bus_device
import adafruit_requests as requests
import adafruit_espatcontrol.adafruit_espatcontrol_socket as socket
from   adafruit_espatcontrol import adafruit_espatcontrol as espatcontrol
try:    from   sparkfun_alphanumeric_bbb import QwiicAlphanumeric as Qwiic
except: from   sparkfun_alphanumeric     import QwiicAlphanumeric as Qwiic

# Get wifi details and more from a secrets.py file
try:
    from secrets import secrets
except ImportError:
    print("All secret keys are kept in secrets.py, please add them there!")
    raise
print(sys.path)
# Import the devices and I/O ports on the Maker Pi Pico Board as MPPio
if not False: import Maker_Pi_Pico_Base as MPPio
else: MPPio = builtins.__import__(board.board_id)

prev_seconds =  None
prev_minutes =  None
prev_hours   =  None # Not Used
prev_mday    =  None
prev_year    =  None
am_pm        =  "24" # "24"=24 hour clock, "12"=AM/PM
request_time =  True
display_date =  True
internet     =  None # Not Tested
rset         = not True
OLED         = not True
BME280       =  True
RTC          =  True
Fram         =  True
CPU          =  True
TickTock     =  True
First        =  True

if  False: # if True show Pin Assignments
    print("Microcontroller Pin Assignments:")
    board_pins = []
    for pin in dir(microcontroller.pin):
        if isinstance(getattr(microcontroller.pin, pin), microcontroller.Pin):
            pins = []
            for alias in dir(board):
                if getattr(board, alias) is getattr(microcontroller.pin, pin):
                    pins.append("board.{}".format(alias))
            if len(pins) > 0:
                board_pins.append(", ".join(pins))
    for pins in sorted(board_pins):
        print(pins)
    time.sleep(10)

DayOfWeek0 = ["Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Beerday"]
Month      = ["Null","January","February","March","April","May","June",
           "July","August","September","October","November","December"]
DTG_API = "http://worldtimeapi.org/api/ip"
#led = digitalio.DigitalInOut(LED)
#led.direction = digitalio.Direction.OUTPUT
VMon = DigitalInOut(board.VOLTAGE_MONITOR)
VMon.direction = Direction.INPUT
SMPS = DigitalInOut(board.SMPS_MODE)
SMPS.direction = Direction.INPUT
VBUS = DigitalInOut(board.VBUS_SENSE)
VBUS.direction = Direction.INPUT

if  False:
    print(microcontroller.cpu.uid)
    print(board.board_id)
    print(dir(MPPio))
    print(MPPio.GROVE_1)
    while  False: pass
# on-board buzzer
BUZZER = MPPio.BUZZER # board.GP18
if not True:
    try:
        # on-board default SPI for SD card
        SCK          = MPPio.SCK   # board.GP10
        MOSI         = MPPio.MOSI  # board.GP11
        MISO         = MPPio.MISO  # board.GP12
        SD_CS        = MPPio.SD_CS # board.GP15
        spi          = busio.SPI(SCK, MOSI, MISO)
        cs           = digitalio.DigitalInOut(SD_CS)
        cs.direction = digitalio.Direction.OUTPUT
        sd           = adafruit_sdcard.SDCard(spi, cs)
        vfs          = storage.VfsFat(sd)
    except(OSError) as er:
        print("SD Card Mount Fail...", er)
        pass
    try:
        storage.mount(vfs, '/sd')
    except:
        print("storage.mount(vfs, '/sd') Failure")
        pass

if  True: # Test write/read to SD card
    with open("/sd/test.txt", "w") as f:
        print("Write line to file  /sd/test.txt:")
        f.write("Hello CircuitPython!\r\n")
    with open("/sd/test.txt", "r") as f:
        print("Read line from file /sd/test.txt:")
        print(f.readline())
    time.sleep(1)

# RGB Pin
RGB = digitalio.DigitalInOut(MPPio.NEOPIXEL) # GP28)
RGB.direction = digitalio.Direction.OUTPUT
# RGB Colors
C = 12
pixel_off     = bytearray([0, 0, 0])
pixel_black   = bytearray([0, 0, 0])
pixel_green   = bytearray([C, 0, 0])
pixel_red     = bytearray([0, C, 0])
pixel_blue    = bytearray([0, 0, C])
pixel_magenta = bytearray([0, C, C])
pixel_cyan    = bytearray([C, 0, C])
pixel_yellow  = bytearray([C, C, 0])
pixel_white   = bytearray([C, C, C])
neopixel_write(RGB, pixel_yellow)
# Create access to the three Buttons
BUTTON = [MPPio.BUTTON_20, MPPio.BUTTON_21, MPPio.BUTTON_22]
BUTTON_nn = [20, 21, 22] # aka [0, 1, 2]
for btns in range(3):
    BUTT = DigitalInOut(BUTTON[btns])
    BUTT.direction = Direction.INPUT
    BUTT.pull = Pull.UP
    BUTTON_nn[btns] = BUTT
# Define access to the six Grove Ports
GROVE   = [MPPio.GROVE_1, MPPio.GROVE_2, MPPio.GROVE_3, MPPio.GROVE_4, MPPio.GROVE_5, MPPio.GROVE_6]
GROVE_n = [0, 1, 2, 3, 4, 5, 6]
for ports in range(6):   # set up each ports
    try:
        _i2c = busio.I2C(*GROVE[ports]) # defaults to input
        print("GROVE_"+str(ports+1)+" - I2C ports: ", end='')
        while not _i2c.try_lock(): pass
        print([hex(x) for x in _i2c.scan()])
        _i2c.unlock()
        GROVE_n[ports+1] = _i2c
    except(ValueError, RuntimeError, espatcontrol.OKError) as er:
        print("GROVE#"+str(ports+1)+" -", er)
        pass
#while True: pass
# GROVE_n[x] port usages defined
I2C_Port = 4
if  RTC:      rtc = adafruit_ds3231.DS3231(GROVE_n[I2C_Port])
if Fram:   fram50 = adafruit_fram.FRAM_I2C(GROVE_n[I2C_Port], address=0x50, write_protect=False,
           wp_pin=None)#, max_size=0x4000)
if BME280: bme280 = adafruit_bme280.Adafruit_BME280_I2C(GROVE_n[I2C_Port], address=0x76)
if (OLED):
    oled = adafruit_ssd1306.SSD1306_I2C(128, 64, GROVE_n[I2C_Port], addr=0x3c)
    oled.fill(0)
    Left = 0 # 10=0-5 lines, 9=0-6 Lines, 8=0-7 7=0-8 Lines
    Line = 9 # Pixels per line of text (lower number >>> more lines)
    oled.text('______NTP  Time______'     , Left, Line*0, 1)
    oled.text('GP20: Reboot'              , Left, Line*1, 1)
    oled.text('0x70: Year'                , Left, Line*2, 1)
    oled.text('0x71: Month Day.Ord'       , Left, Line*3, 1)
    oled.text('0x72: Hour:Min:Sec'        , Left, Line*4, 1)
    oled.text('0x73: Day Of Week'         , Left, Line*5, 1)
    if Line <= 9: oled.text(board.board_id, Left, Line*6, 1)
    S = ((str(sys.implementation.version[0]))+":"+(str(sys.implementation.version[1]))+
        ':'+(str(sys.implementation.version[2])))
    if Line <= 8: oled.text('CircuitPython: ' + S, Left, Line*7, 1)
    if Line <= 7: oled.text('CircuitPython: ' + S, Left, Line*8, 1)
    oled.show()

if board.board_id == "raspberry_pi_pico_w":
    # RPi PicoW WiFi Chip CYW43439KUBG
    print("Setup Raspberry Pi Pico W")
    WL_ON  = microcontroller.pin.GPIO23 # board.GP23
    WL_D   = microcontroller.pin.GPIO24 # board.GP24
    WL_CS  = microcontroller.pin.GPIO25 # board.GP25
    WL_CLK = microcontroller.pin.GPIO29 # board.GP29

# initial comm
    print("My MAC addr:", [hex(i) for i in wifi.radio.mac_address])
# wifi scan
    print("Available WiFi networks:")
    for network in wifi.radio/start_scanning_networks():
        s = " "*len(str(network.ssid, "utf-8"))
        print("\t%s%s\t\tRSSI: %d\tChannel: %d" % (str(network.ssid, "utf-8"), s, network.rssi,
             network.channel))
    wifi.radio.stop_scanning_networks()
# connect and IP
    wifi.radio.connect('adafruit', 'DEADBEEFC0')
    print("Hello World", wifi.radio.ipv4_address)
elif board.board_id == "raspberry_pi_pico":
    print("Setup ESP8266 - ESP-01 Module")
    # Initialize UART/ESP-01 connection to the ESP8266 WiFi Module.
    TX = MPPio.ESP_TX # GP16
    RX = MPPio.ESP_RX # GP17
    BUFFER_SIZE = 4096
    BAUDRATE = 115200
    DEBUG = False
    uart = busio.UART(TX, RX, receiver_buffer_size=BUFFER_SIZE)
    # Use large buffer as we're not using hardware flow control.
    esp = espatcontrol.ESP_ATcontrol(uart, default_baudrate=BAUDRATE, debug=DEBUG)
    requests.set_socket(socket, esp)
else:
    print("*** No WiFi Module.")
    raise
C = 64.0
display0 = Qwiic(GROVE_n[I2C_Port], address=0x70, brightness=1.0/C)
display0.clear()
display0.print("0x70")
display1 = Qwiic(GROVE_n[I2C_Port], address=0x71, brightness=1.0/C)
display1.clear()
display1.print("0x71")
display2 = Qwiic(GROVE_n[I2C_Port], address=0x72, brightness=1.0/C)
display2.clear()
display2.print("0x72")
display3 = Qwiic(GROVE_n[I2C_Port], address=0x73, brightness=1.0/C)
display3.clear()
display3.print("0x73")
print("########","########")
X = 1
print("Scanning for WiFi Access Points")
print("Seq\tECN\tSSID\t\t  dBm\tMAC Address\t\tChan\tFoffs\tFcal")
#aps = []
for ap in sorted(esp.scan_APs()):
#    aps.append(ap)
    S = "." * (18 - len(str(ap[1])))
    print("{0:2}\t{1:}\t{2:}{3:}{4:}\t{5:}\t{6:}\t{7:}\t{8:}".format(X,ap[0],ap[1],S,ap[2],ap[3],ap[4],ap[5],ap[6]))
    X += 1
print("  ECN = 0:OPEN  1:WEP  2:WPA_PSK  3:WPA2_PSK  4:WPA_WPA2_PSK  5:WPA2_Enterprise")
print("########","########")
#time.sleep(1800)
'''
print("")
print(aps)
print("")
for X in aps:
    print("ssid:",aps, "rssi:",aps)
    print("")
'''
'''
import wifi
networks = []
for network in wifi.radio.start_scanning_networks():
    networks.append(network)
wifi.radio.stop_scanning_networks()
networks = sorted(networks, key=lambda net: net.rssi, reverse=True)
for network in networks:
    print("ssid:",network.ssid, "rssi:",network.rssi)
'''
time.sleep(0.25)
print("Resetting ESP8266 * ESP-01 module.")
esp.soft_reset()
time.sleep(0.25)
print("Socket Disconnect.")
esp.socket_disconnect()
time.sleep(0.25)
#
time.sleep(0.25)
NOTE_C4, NOTE_F4, NOTE_G4, NOTE_A4 = 261, 349, 392, 440
#######################################
#######################################
Square = lambda x: x**2
CtoF   = lambda x: x*1.8+32
FtoC   = lambda x: (x-32)/1.8
KtoC   = lambda x: x-273.15
KtoF   = lambda x: CtoF(KtoC(x)) # (x-273.15)*1.8+32
#######################################
def play_melody(notes, duration):
  for i in range(len(notes)):
    simpleio.tone(BUZZER, notes[i], 1/duration[i])
#######################################
def B_Fifth():
  C = 6
  _notes = [NOTE_A4, 0, NOTE_A4, 0, NOTE_A4, 0, NOTE_C4]
  _duration = [C, C*2, C, C*2, C, C*2, C/4]
  play_melody(_notes, _duration)
#######################################
def ds3231_ReadTime1():
  if not RTC: return(None)
  try:
    return(rtc.datetime)
  except:
    return(None)
#######################################
def ds3231_WriteTime1(t):
  if not RTC: return(None)
  try:
    rtc.datetime = t
    return(ds3231_ReadTime1())
  except:
    return(None)
#######################################
def Board_UID():
  print(f"\t\b\bCircuit Python ID: %s" % board.board_id)
  UID = (microcontroller.cpu.uid) # (str(network.ssid, "utf-8"))
  #print('\t\t\t\b:', UID)
  UID_ = ''
  #print(bytes(microcontroller.cpu.uid.decode()))
  for x in range(8):
    if (len(hex(UID[x])[2:4]) < 2): UID_ += "0"
    UID_ += hex(UID[x])[2:4]
    if x%2: UID_ += " "
  print(f"    Microcontroller UID: %s" % UID_.upper())
#######################################
def Z_L(hours, am_pm):
  if (am_pm != "24"):
    if hours >= 12:
      hours -= 12
      am_pm = "PM"
      if not hours: hours = 12
    else:
      am_pm = "AM"
  return(hours, am_pm)
#######################################
def ds3231_rtc(am_pm):
  if not RTC: return(None)
  global e, f
#  global a, b, c, d, e, f, g, h, i, j, k, now
  now = ds3231_ReadTime1()
  a = now.tm_year  # Year
  b = now.tm_mon   # Month
  c = now.tm_mday  # MDay
  d = now.tm_hour  # Hours_24
  e = now.tm_min   # Minute
  f = now.tm_sec   # Second
  g = now.tm_wday  # Day of Week 1-7
  h = now.tm_yday  # yday -1=N/A
  i = now.tm_isdst # is DST -1=N/A
  #print(a, b, c, d, e, f, g, h, i, "\n", now)
  hours = d
  hours, am_pm = Z_L(hours, am_pm)
  print("  {} {}, {} {}:{:02}:{:02} {} {} DS3231".format(Month[b], c, a, hours, 
    e, f, am_pm, DayOfWeek0[g]))
  #print RTC DS3231 temperature
  r = rtc.temperature
  print(f"\t\b\b\b\b DS3231 Temperature: %0.2f°C\t%0.2f°F" % (r, CtoF(r)))
  return(now)
#######################################
def CPU_Temp():
  #print(microcontroller.cpus[n].temperature)
  print(f"    CPU: Temperature and Device Status")
  for i in range(2):
    cpu_C = microcontroller.cpus[i].temperature
    print(f"\t\t CPU(%d): %3.2f°C\t%3.2f°F" % (i, cpu_C, CtoF(cpu_C)))
  print(f"\tVOLTAGE_MONITOR: %s" % str(VMon.value))
  print(f"\t      SMPS_MODE: %s" % str(SMPS.value))
  print(f"\t     VBUS_SENSE: %s" % str(VBUS.value))
#######################################
def BME_280():
  if not BME280: return
  # Local Weather Source hPa/mB https://forecast.weather.gov/MapClick.php?lat=38.97&lon=-76.49
  t_off_set = -2.25
  h_off_set =  2.75
  bme280.sea_level_pressure = 1010.0 # Annapolis, MD
  t = bme280.temperature + t_off_set
  h = bme280.humidity    + h_off_set
  d = dewpoint(t, h)   # Calculated
  i = heat_index(t, h) # Calculated
  p = bme280.pressure
  a = bme280.altitude
  s = ''
  if d < 10: s = ' '
  print(f"\t\b\b\b\bBME280: Environment Readings")
  print(f"\t    Temperature: %0.2f°C\t%0.2f°F" % (t, CtoF(t)))
  print(f"\t       Humidity: %0.2f%%\t\t%0.2f%%" % (h, h))
  print(f"\t      Dew Point: %s%0.2f°C\t%0.2f°F" % (s, d, CtoF(d)))
  print(i)
  print(f"\t\b\b\b\bBarometric Pressure: %0.2f hPa\t%0.2f mmHg" % (p, p/33.8638867))
  print(f"\t       Altitude: %0.2f Meters\t%0.2f Feet" % (a, a*3.28))
  print(f"\t      Constants: Pi = " + str(math.pi) + "\te = " + str(math.e))
#######################################
def dewpoint(C, H): # Calculated
  c1 =  17.62
  c2 = 243.15
  gamma = ((c1 * C) / (c2 + C)) + math.log(H / 100.0)
  dp = (c2 * gamma) / (c1 - gamma)
  return(dp)
#######################################
def heat_index(t, h): # aka Chill Factor, Feels Like
  # Formula Source https://www.wpc.ncep.noaa.gov/html/heatindex_equation.shtml
  T  = CtoF(t) # Convert to Fahrenheit
  RH = h        # RH Relative Humidity, HI Heat Index
  HI = 0.5 * (T + 61.0 + ((T-68.0)*1.2) + (RH*0.094))
  s  = "Standard"
  if HI > 80:
    c1,c2,c3,c4,c5 = -42.379, 2.04901523, 10.14333127, 0.22475541, 0.00683783
    c6, c7, c8, c9 = 0.05481717, 0.00122874, 0.00085282, 0.00000199
    Ts  = Square(T)
    RHs = Square(RH)
    HI  = c1 + c2*T + c3*RH - c4*T*RH - c5*Ts - c6*RHs + c7*Ts*RH + c8*T*RHs - c9*Ts*RHs
    s   = "detailed "
  ST = ("\t     Heat Index: %0.2f°C\t%0.2f°F %s" % (FtoC(HI), HI, s))
  F_ = False
  if RH < 13 and T > 85: # Subrtact from HI
    HI -= ((13-RH)/4)*sqrt((17-abs(T-95.))/17)
    s = s + "adjusted - low"
    F_ = True
  if RH > 85 and (T >= 80 and T <= 87): # Add to HI
    HI += ((RH-85)/10) * ((87-T)/5)
    s = s + "adjusted - high"
    F_ = True
  if  F_: ST = ("\t     Heat Index: %0.2f°C\t%0.2f°F %s" % (FtoC(HI), HI, s))
  return(ST)
#######################################
def fram_test():
  if  Fram:
    print("\t\t\t\b\b\b\b\bFRAM: ",end='')
    print(len(fram50))
    if not True:
      __ = time.monotonic()
      values = list(range(128))  # or bytearray or tuple
      fram50[0:128] = values
      print("\t\b\b\b\b",end='')
      print(fram50[ 0:32])
      print("\t\b\b\b\b",end='')
      print(fram50[32:64])
      print("\t\b\b\b\b",end='')
      print(fram50[64:96])
      print("\t\b\b\b\b",end='')
      print(fram50[96:128])
      print("\t\b\b\b\b",end='')
      print(fram50[128:144])
      print("\t\b\b\b\b",end='')
      print(fram50[65], "#", fram50[65][0], "#", chr(fram50[65][0]))
      print(time.monotonic() - __)
#######################################
#######################################
#######################################
#######################################
#######################################
############################################################
C = 4
_notes = [NOTE_A4, NOTE_F4, NOTE_G4, NOTE_C4, 0, NOTE_C4, NOTE_G4, NOTE_A4, NOTE_F4]
_duration = [C, C, C, C,  C/2,  C, C, C, C]
play_melody(_notes, _duration)
while True:
    if request_time:
        for i in range(5): # Try five times
            print("\nTry #" + str(i+1) + " of Connect/Get Data loop.")
            while not esp.is_connected:
                try:
                    print("Connecting...")
                    esp.connect(secrets)
                    print("Connected!...")
                    time.sleep(0.25)
                    p = esp.ping(secrets["modem_ip"])
                    print("Ping round trip time: " + str(p) + "ms")
                    internet = True
                except (ValueError, RuntimeError, espatcontrol.OKError) as er:
                    print("Failed, retrying\n", er)
                    internet = False
                pass
            try:
                #led.value = True
                time.sleep(0.1)
                p = esp.ping(secrets["modem_ip"])
                print("Ping round trip time: " + str(p) + "ms")
                if p == 'None': supervisor.reload() # Nonems
                print("Getting DTG from: " + DTG_API)
                response = requests.get(DTG_API)
                json = response.json()
                #                    UnixTime  -  SysTime  +       TZ Offset  +       DST Offset
                time_offset = (json["unixtime"]-time.time()+json["raw_offset"]+(json["dst_offset"]))
                #print(json, "Test") #################
                print("Got ... DTG")
                Day_of_week = json['day_of_week']
                #led.value = False
                print(json["utc_datetime"], "UTC")
                print(json["datetime"], json["abbreviation"])
                print(json["dst_from"], "DST Starts")
                print("\t\t\t ", json["abbreviation"], "Time Zone")
                print(json["dst_until"], "DST Stops")
                request_time = False
                internet = True
                time.sleep(0.1)
                break
            except (ValueError, RuntimeError, espatcontrol.OKError) as er:
                print("Failed, Retrying,\nError: ", er)
                print("Resetting ESP8266 * ESP-01 module.")
                esp.soft_reset()
                print("Socket Disconnect.")
                esp.socket_disconnect()
                B_Fifth()
                print("End of try #" + str(i+1) + " Connect/Get Data loop.")
                time.sleep(10)
                internet = False
            pass
        pass
    try:
        current_time = time.time() + time_offset
    except:
        supervisor.reload()
    now = time.localtime(current_time)
    seconds = now.tm_sec
    if seconds != prev_seconds: # Skip the following code until seconds changes
        prev_seconds = seconds
        print('\f')
        _ = 25
        R = random.randrange(_)
        G = random.randrange(_)
        B = random.randrange(_)
        RND = bytearray([R, G, B])
        neopixel_write(RGB, RND)
        year     = now.tm_year
        month    = now.tm_mon
        mday     = now.tm_mday
        yday     = now.tm_yday
        hours_24 = now.tm_hour
        minutes  = now.tm_min
        seconds  = now.tm_sec
        isdst    = now.tm_isdst
        wday     =(now.tm_wday+1)%7
        #Check if we should reboot
        if(not BUTTON_nn[0].value):
            B_Fifth()
            supervisor.reload()
        #Display Date, Time, Day on 14 segment displays
        S = ""
        if seconds < 10: S = "0" # Leading Zero for numbers less than 10
        if (not seconds % 2) : S = ":" + S # flash colon (OFF for Odd numbers)
        if (not seconds % 2) and rset: S = S[1:] # len(S)-1
        hours = hours_24
        hours, am_pm = Z_L(hours, am_pm)
        if am_pm != "24":
            S0 = S + str(seconds) + am_pm # Seconds AM/PM
        else:
            S0 = S + str(seconds) + DayOfWeek0[wday][0:2].upper() # Seconds & first two letters of day
        display3.print(S0)
        if(prev_minutes != minutes):
            prev_minutes = minutes
            S = ":"
            if (minutes < 10): S = S +  "0"
            S1 = str(hours) + S + str(minutes)
            display2.print(S1)
        if(prev_mday != mday):
            prev_mday = mday
            S = ""
            if (mday < 10): S = " "
            S2 = str(month) + S + str(mday)
            display1.print(S2)
        if(prev_year != year):
            prev_year = year
            S3 = str(year)
            display0.print(S3)
        #Create REPL string
        if(hours_24 == 2 and minutes == 0 and seconds == 0): request_time = True # 2:00 AM
        hours = hours_24
        hours, am_pm = Z_L(hours, am_pm)
        #led.value = True
        if CPU: Board_UID()
        #print("000000000111111111122222222223333333333")
        #print("123456789012345678901234567890123456789")
        ds3231_rtc(am_pm) # Get time from DS3231 RTC clock chip and print to REPL.
        s = " " * int((len(DayOfWeek0[wday])+0.5)/2)
        print(" {}{}, {}/{}/{} {}:{:02}:{:02} {} {}, {:03}".format(s, DayOfWeek0[wday],
              month, mday, year, hours, minutes, seconds, am_pm, json["abbreviation"], yday))
        # Show the CPU temperatures
        if CPU: CPU_Temp()
        #led.value = False
        #Play Seconds and Hourly Alerts
        oneKHz = 1000
        if   minutes == 0 and seconds == 0:
            if TickTock: simpleio.tone(BUZZER, oneKHz, 0.25) # Long Tick at each minute
        elif minutes == 59 and seconds >= 55:
            if TickTock: simpleio.tone(BUZZER, oneKHz, 0.05) # Medium Tick for seconds 55 through 59
        elif seconds == 0:
            if TickTock: simpleio.tone(BUZZER, oneKHz, 0.01) # Short Tick at each minute
        else:
            if TickTock: simpleio.tone(BUZZER, oneKHz, 0.0025) # Tick at each second
        # Go Show the Board Environment
        BME_280()
        fram_test()
        #Check if we are connected to WiFi
        interval = 30
        if (minutes%interval == 0 or minutes == 0) and seconds == 0 and display_date == True:
            print(str(interval) + " Minute Time Check.")
            display_date = False
            prev_minutes = minutes
            if not esp.is_connected: # Check if WiFi is still working
                for i in range(3): # Try 3 times
                    try:
                        print("Connecting...")
                        esp.connect(secrets)
                        print("Connected!...")
                        time.sleep(0.25)
                        p = esp.ping(secrets["modem_ip"])
                        print("Ping round trip time: " + str(p) + "ms")
                        request_time = True
                        internet = True
                        break
                    except (ValueError, RuntimeError, espatcontrol.OKError) as er:
                        print("Failed, retrying\t", er)
                        internet = False
                    # end try
                # end for
            # end if
        else:
            if minutes == prev_minutes and display_date == False:
                display_date = True
        if RTC:
            if((seconds != f) or (minutes != e)): # Adjust DS3231 if off by +/-1 second or more
                rset = True
                if(seconds > f):
                    tm = time.struct_time((year, month, mday, hours_24, minutes, (seconds+1 )%60, Day_of_week, yday, isdst))
                if(seconds < f):
                    tm = time.struct_time((year, month, mday, hours_24, minutes, (seconds+59)%60, Day_of_week, yday, isdst))
                ds3231_WriteTime1(tm)
                print("    DS3231 reset.", minutes, seconds, e, f)
            else:
                rset = False
    if (OLED):
        offset = 1
        flip = not minutes % 2
        oled.fill(not flip)
        oled.text('______NTP Time______0', Left, Line*0, flip)
        oled.text('GP20: Reboot', Left, Line*1, flip)
        oled.text('0x70: '+ S3, Left, Line*2, flip)
        if   mday >= 11 and mday <= 13: ordinal = "th"
        elif mday %  10 == 0: ordinal = "th"
        elif mday %  10 == 1: ordinal = "st"
        elif mday %  10 == 2: ordinal = "nd"
        elif mday %  10 == 3: ordinal = "rd"
        elif mday %  10 >= 4: ordinal = "th"
        else:                 ordinal = "wtf"
        oled.text('0x71: '+ Month[month]+" "+str(mday)+ordinal, Left, Line*3, flip)
        sec = seconds + offset
        s = ''
        if sec < 10 or sec == 60: s = '0'
        oled.text('0x72: ' + S1+':' + s + str(sec%60), Left, Line*4, flip)
        oled.text('0x73: ' + DayOfWeek0[wday] , Left, Line*5, flip)
        if Line <= 9: oled.text('_____Local Time_____6', Left, Line*6, flip)
        if Line <= 8: oled.text('_____Local Time_____7', Left, Line*7, flip)
        if Line <= 7: oled.text('_____Local Time_____8', Left, Line*8, flip)
        oled.show()
        First = False
    pass
pass
############################################################


User avatar
mikeysklar
 
Posts: 13979
Joined: Mon Aug 01, 2016 8:10 pm

Re: Run code.py from SD card

Post by mikeysklar »

Okay, this is helpful. Thank you for the walk thru.

Your 700 line program is bringing in a ton of libraries so that is going to be slow coming off an SD card. You could add prints in between the imports at the top of your big program to get an idea of how long the different libraries are taking to load

There are at least four optimizations to consider:

1) Reduce the /sd/lib contents to only the libraries you are using. That is still a lot, but I've seen multiple issues with os.listdir() being particularly pokey so reducing the contents of the /sd/lib directory could speed things up.

2) The SDcard itself is a slow model. What model is your 32GB card? The difference can be from 2MB/s to 90MB/s. The number printed on the card makes a big difference. Maybe use a different card if it is on the slower side.
Screenshot from 2022-09-28 13-07-03.png
Screenshot from 2022-09-28 13-07-03.png (169.03 KiB) Viewed 135 times
https://www.sdcard.org/developers/sd-st ... eed-class/

3) Normally with CircuitPython SDIO would be an option, but the RP2040 has its own PIO system which will probably means SPI bus for SD card uses.

https://learn.adafruit.com/adafruit-mic ... cuitpython

https://learn.adafruit.com/adafruit-mic ... io-3119669

4) The SD card SPI bus speed can be adjusted.

https://docs.circuitpython.org/projects ... t/api.html

User avatar
blakebr
 
Posts: 960
Joined: Tue Apr 17, 2012 6:23 pm

Re: Run code.py from SD card

Post by blakebr »

Mike,

Thanks for the reply.

I am using the Samsung 32GB EVO Select Micro SD HC I U1, Ultra-fast read write speeds: Up to 95MB/S Read and 20MB/S Write Speeds; Uhs Speed Class U1 and Speed Class 10 (Performance may vary based on host device, interface, usage conditions, and other factors). Operating Voltage: 2.7-3.6V.

I like your other suggestions. I will try them one by one this evening and get back to you.

Bruce

User avatar
blakebr
 
Posts: 960
Joined: Tue Apr 17, 2012 6:23 pm

Re: Run code.py from SD card

Post by blakebr »

Mike,

I put 7 print statements along the library import section of my code. Indeed it is the loading of the libraries that is slowing things down.

It intrigued me that loading the program took the same time if /lib or /libbbb was in play. So I commented out the append /sd/lib statement in code.py. With /libbbb the program crashed, because /lib was not available and /sd/lib was not in the sys.path. With /lib available the program ran at regular speed. So just by appending /sd/lib the loading of the libraries was slowed down. With /lib and /sd/lib available and /sd/lib at the end or beginning of the sys.path statement the libraries were loaded slowly from /sd/lib. Why is /sd/lib taking priority over /lib? We have a clue!

In other words:
/lib Normal
/libbbb Crash, No Libraries Found
/lib & /sd/lib Slow
/libbbb & /sd/lib Slow

Bruce

User avatar
blakebr
 
Posts: 960
Joined: Tue Apr 17, 2012 6:23 pm

Re: Run code.py from SD card

Post by blakebr »

Mike,

We still don't know why:
RESET/POWER is slow
Import code.py & import program.py from REPL is fast.

Bruce

User avatar
blakebr
 
Posts: 960
Joined: Tue Apr 17, 2012 6:23 pm

Re: Run code.py from SD card

Post by blakebr »

Mike,

By changing the baudrate:

Default: 74 Seconds
660_000: 119 Seconds
1_320_000: 73 Seconds
2_640_000: 52 Seconds
5_280_000: 40 Seconds
10_560_000: 35 Seconds

Hmmmm

Can you find the baudrate limits?
Is baudrate for bits or bytes?

Bruce

User avatar
blakebr
 
Posts: 960
Joined: Tue Apr 17, 2012 6:23 pm

Re: Run code.py from SD card

Post by blakebr »

Mike,

These numbers are with BAUDRATE = 10_560_000 # bps aka 84.48 MBps
OK... Interesting fact having /sd/lib in the path slows down importing internal libraries by a bunch. See below for code. They load in 7 or 8 seconds with /sd/lib, 1 second without /sd/lib. What is happening? All libraries, internal and external load in 2 or 3 seconds without /sd/lib. With /sd/lib 33 seconds.

Bruce

Code: Select all

tt = time.time()
# Intrinsic Libraries
import rtc
import sys
import json
import math
import busio
import busio as busio
import board
from   board import *
from   struct import unpack
import random
import storage
print(time.time() - tt) ############
import builtins
import digitalio
from   digitalio import *
import supervisor
from   neopixel_write import neopixel_write
import microcontroller
try: # for FRAM
    from digitalio import DigitalInOut
    from busio import I2C, SPI
except ImportError:
    pass
# Extrinsic Libraries
print(time.time() - tt, 'intrinsic') ############

User avatar
mikeysklar
 
Posts: 13979
Joined: Mon Aug 01, 2016 8:10 pm

Re: Run code.py from SD card

Post by mikeysklar »

Nice job figuring out the baud-rate plays a role in regards to library loading time.

You have shrunk the load time from 75 seconds to 33 seconds. Is that correct?

During your testing with and without /sd/lib were you moving the libraries to local flash as well or using the full pathname /sd/lib to access them? There is usually some caching involved when adding directories to a PATH variable so it can be expensive the first time. You would also be seeing the difference of pulling in files from fast flash versus SD card media.

Do you have an Adafruit Feather RP2040 (8MB flash)? I would be curious if all the slowdowns and complexity went away just due to having adequate builtin flash.

https://www.adafruit.com/product/4884

User avatar
blakebr
 
Posts: 960
Joined: Tue Apr 17, 2012 6:23 pm

Re: Run code.py from SD card

Post by blakebr »

Mike,

Thank you. Yes, I have shrunk the load time by 1/2.

During testing the library files existed in /lib and /sd/lib simultaneously.
In all my testing there are two situations I think we should focus on.
1.
Without /sd/lib in the path the internal libraries, time, sys, board, etc. takes 1 second to load.
With /sd/lib at the front or rear of the path it takes 8 seconds to load.
With /sd/lib at the front and rear of the path it takes 15 seconds to load.
Having /sd/lib on an SD card should have absolutely no impact on load times of internal libraries. I think there may be a bug in the path management of CircuitPython.
2.
Without /sd/lib in the path all the libraries take 3 seconds to load.
With /sd/lib at the front or rear of the path it takes 33 seconds to load.
With /sd/lib at the front and rear of the path it takes 59 seconds to load.
Again, I think this points to a bug in CircuitPython's management of the path function.

The RPi Pico has 243KB free of 0.98MB on its flash drive.
Yes, I have a Feather RP2040, however I would prefer to use an ItsyBitsy RP2040. It fits a breadboard better. What do you propose I try?

P.S, I miss the green star by my icon.

Bruce

User avatar
mikeysklar
 
Posts: 13979
Joined: Mon Aug 01, 2016 8:10 pm

Re: Run code.py from SD card

Post by mikeysklar »

Bruce,

I think you are correct that this is something to bring to the attention of the CircuitPython github repo.

In terms of sharing a minimal viable code example what would you recommend? Is it necessary to even have an /sd card present or just "/sd/lib" being added to the PATH (no card connected) is enough to slow things?

Do you see a significant delay just by running a Pico RP2040 with the append PATH (no /sd present). I wanted to verify that the CircuitPython code is slow due to walking the directory.

Code: Select all

import adafruit_sdcard
import busio
import digitalio
import board
import storage
import sys
# Connect to the card and mount the filesystem.
#spi = busio.SPI(board.GP10, MOSI=board.GP11, MISO=board.GP12) 
#cs = digitalio.DigitalInOut(board.GP15) 
#sdcard = adafruit_sdcard.SDCard(spi, cs)
#vfs = storage.VfsFat(sdcard)
#storage.mount(vfs, '/sd')
sys.path.append('/sd/lib')
I did look into the CircuitPython source and this stood out. The default SPI for the SD card baudrate is 8MHz. I assume the baudrate you had been changing was for the REPL serial. Is that correct?

Code: Select all

STATIC mp_obj_t sdcardio_sdcard_make_new(const mp_obj_type_t *type, size_t n_args, size_t n_kw, const mp_obj_t *all_args) {
    enum { ARG_spi, ARG_cs, ARG_baudrate, NUM_ARGS };
    static const mp_arg_t allowed_args[] = {
        { MP_QSTR_spi, MP_ARG_OBJ, {.u_obj = mp_const_none } },
        { MP_QSTR_cs, MP_ARG_OBJ, {.u_obj = mp_const_none } },
        { MP_QSTR_baudrate, MP_ARG_INT, {.u_int = 8000000} },
    };
Note that during detection and configuration, a hard-coded low baudrate is used. Data transfers use the specified baurate (rounded down to one that is supported by the microcontroller)
The RP2040 ItsyBitsy would be fine to use for testing. It also has 8MB of flash.

User avatar
blakebr
 
Posts: 960
Joined: Tue Apr 17, 2012 6:23 pm

Re: Run code.py from SD card

Post by blakebr »

Mike,

The one thing I could test fast. I commented out the initialization of the SD card. Added /sd and /sd/lib to the path. Everything loaded in 3 seconds. So there must be an SD card established and it must have a /sd/lib folder.

I looked at the adafruit_sdcard.py library. It does its setup with a clock rate of 1_320_000 MHz, then near the end of the __init__ function it does a second setup with the user specified data rate if one is given. If none are given the 1_230_000 MHz rate stays in effect. What are the roundoff data rates? Are they multiples of the 1_320_000 MHz data rate?

This is 100% a guess. When a library is requested ALL areas are searched and an array is created for the library being searched for. Then foo.py, foo.mpy, foo.txt are selected by some priority. What that priority is I have no idea. I bet this is so an internal library can be replaced by one in flash /lib memory or on an SD card /sd/lib.

Bruce

User avatar
mikeysklar
 
Posts: 13979
Joined: Mon Aug 01, 2016 8:10 pm

Re: Run code.py from SD card

Post by mikeysklar »

I took a moment to review your comments on the adafruit_sdcard.py library.

The slow init SPI data rate is 250 kHz
The default after card setup is 1.3 MHz

When I look at an example of people trying to get more performance out of a RP2040 based SDcard setup they are setting their SPI bus to 24, 30 and 50 MHz. Quite a bit faster than the default adafruit_sdcard setting of 1.3 MHz which is probably in place for maximum compatibility.

https://community.element14.com/product ... -benchmark

Maybe you can try initializing setting the SPI data rate with these values and see how it effects the initial "indexing (guess)" load speed for your code.

50MHz

Code: Select all

sdcard = adafruit_sdcard.SDCard(spi, cs, 50000000)
30MHz

Code: Select all

sdcard = adafruit_sdcard.SDCard(spi, cs, 30000000)
24MHz

Code: Select all

sdcard = adafruit_sdcard.SDCard(spi, cs, 24000000)

User avatar
blakebr
 
Posts: 960
Joined: Tue Apr 17, 2012 6:23 pm

Re: Run code.py from SD card

Post by blakebr »

Mike,

It looks like 21MHz is the maximum. 30MHz no response from SD card

Code: Select all

#BAUDRATE = 26_400_000 # bps aka 253.44 MBps, 26 Seconds
_BAUDRATE = 24_000_000 # bps aka 168.96 MBps, 26 Seconds
_BAUDRATE = 21_120_000 # bps aka 168.96 MBps, 26 Seconds
#BAUDRATE = 10_560_000 # bps aka  84.48 MBps, 35 Seconds
#BAUDRATE =  5_280_000 # bps aka  42.24 MBps, 40 Seconds
#BAUDRATE =  2_640_000 # bps aka  21.12 MBps, 52 Seconds
#BAUDRATE =  1_320_000 # bps aka  10.56 MBps, 73 Seconds DEFAULT
#BAUDRATE =    660_000 # bps aka   5.28 MBps, 119 Seconds
Bruce

User avatar
mikeysklar
 
Posts: 13979
Joined: Mon Aug 01, 2016 8:10 pm

Re: Run code.py from SD card

Post by mikeysklar »

Thank you, Bruce.

It looks like performance in this case was plateauing @21,120,000 assuming that the two uncommented lines were run correctly. eg. 24 MHz would not run if 21.12 MHz was still uncommented.

I think it is time to bundle the minimum viable hardware / code necessary for a CircuitPython issue.

Locked
Please be positive and constructive with your questions and comments.

Return to “Adafruit CircuitPython”