M4 Matrix Portal 64x64 animated display with sound?

Please tell us which board you are using.
For CircuitPython issues, ask in the Adafruit CircuitPython forum.

Moderators: adafruit_support_bill, adafruit

Please be positive and constructive with your questions and comments.
Locked
User avatar
Astrovore
 
Posts: 2
Joined: Fri Jul 29, 2022 1:39 pm

M4 Matrix Portal 64x64 animated display with sound?

Post by Astrovore »

I have an M4 matrix portal connected to a 64x64 LED matrix. I'm currently coding in CircuitPython 7.x.

I started with the code from this tutorial on sprite-based animation, adapted for a 64x64 display and my own 64x64 sprites: https://learn.adafruit.com/pixel-art-ma ... on-display.

I have the small STEMMA-QT speaker breakout board that I would like to play sound over while the display is animating. My naive approach of simply adding the playback to the main loop gave me incredibly choppy audio playback thanks to (I assume) the time.sleep call used to set the animation framerate. Fair enough.

My next approach was to use the asyncio library to allow the sound to keep playing while the animation waits for its next frame. This gave potentially slightly better audio outbut but it's still pretty awful. It seems that re-rendering the entire 64x64 display 10 times a second simply requires too much processor time to allow audio playback over an analog pin at the same time.

Is this conclusion correct? My current thought is that offloading sound playback to something like the FX sound board (https://www.adafruit.com/product/2133) is the best solution, but if I'm missing something and doing full-screen animations alongside smooth audio playback is possible without extra hardware I'm all ears!

For reference, here's my code in full

Code: Select all

import time
import os
import asyncio
import audioio
import board
import displayio
from adafruit_matrixportal.matrix import Matrix
from audiocore import WaveFile
import adafruit_nunchuk

displayio.release_displays()

BACKGROUND_BMP = displayio.OnDiskBitmap("/bmps/momo.bmp")
IDLE_BMP = displayio.OnDiskBitmap("/bmps/idle.bmp")
SPEAKING_BMP = displayio.OnDiskBitmap("/bmps/speaking.bmp")

VOICE_WAV = "/voice/voice.wav"

class FullscreenSprite:
    def __init__(self, sprite_bitmap, background_bitmap, frame_delay):
        self.group = displayio.Group()
        self.current_frame = 0
        self.frame_delay = frame_delay
        self.sprite_bitmap = sprite_bitmap
        self.frame_count = int(sprite_bitmap.width / 64)
        self.background_bitmap = background_bitmap

        colors = displayio.ColorConverter()
        colors.make_transparent(0)
        self.background = displayio.TileGrid(
             background_bitmap,
             pixel_shader=colors,
             tile_width=64,
             tile_height=64
         )
        self.sprite = displayio.TileGrid(
            sprite_bitmap,
            pixel_shader=colors,
            tile_width=64,
            tile_height=64
        )
        self.group.append(self.background)
        self.group.append(self.sprite)

    async def advance_frame(self):
        self.current_frame = self.current_frame + 1
        if self.current_frame >= self.frame_count:
            self.current_frame = 0
        self.sprite[0] = self.current_frame
        await asyncio.sleep(self.frame_delay)


async def audio_player():
    speaker = audioio.AudioOut(board.A0)
    wav = WaveFile(open(VOICE_WAV, "rb"))
    speaker.play(wav, loop=True)

async def animate_face():
    matrix = Matrix(bit_depth=4, width=64, height=64)

    idle_sprite = FullscreenSprite(IDLE_BMP, BACKGROUND_BMP, 0.1)

    current_sprite = idle_sprite
    matrix.display.show(current_sprite.group)

    while True:
        await current_sprite.advance_frame()

async def main():
    animation_task = asyncio.create_task(animate_face())
    audio_task = asyncio.create_task(audio_player())

    await asyncio.gather(animation_task, audio_task)


asyncio.run(main())

User avatar
mikeysklar
 
Posts: 13936
Joined: Mon Aug 01, 2016 8:10 pm

Re: M4 Matrix Portal 64x64 animated display with sound?

Post by mikeysklar »

Good analysis on your part on the code looks super nice.

The WAV audio files you are using. Do you think it would make a difference if they were lower quailty:

8-bit unsigned, mono, 16Hz sample rate?

This might require additional HW as you suggest (Audio FX board), but I wanted to rule out that the audio quality wasn't slowing things down or naturally choppy due to a format mismatch.

User avatar
Astrovore
 
Posts: 2
Joined: Fri Jul 29, 2022 1:39 pm

Re: M4 Matrix Portal 64x64 animated display with sound?

Post by Astrovore »

Thanks for the advice. I'd tried lower sample rates without luck but that was at 16 bit encoding.

I decided to get a sound board as it also gives me more storage as well as the option for stereo and a larger built-in amp.

Powering it from the 3v pin on the portal and controlling it over UART worked really well too, so it's a great solution for me.

Locked
Please be positive and constructive with your questions and comments.

Return to “Itsy Bitsy Boards”