Improving M4 SDfat read/write speeds

[moonie223]

Hello!

I would like to try and log ~50-200KB/sec of data to a SD card, using an itsybitsy m4.

Using the examples as they are (bench.ino) I only manage around 130kb/sec, using a class 10 SD card. I feel like even with a 12MHz clock 1000+kb/sec should be possible. The sdfat library has an example with a due that manages some really nice speeds, much better than what the current library manages.

I see that lots has changed in the BSP since the SDfat-adafruit fork has been updated. I wonder if there are any quick changes that can be implemented, even if it means running a large read/write buffer. I feel like if I smash my face into the keyboard for a few weeks I might be able to make something work but I felt like I should ask here first if anyone had any ideas.

Thank you!

[mikeysklar]

@moonie223,

Were you running this SdFat's bench.ino?

https://github.com/greiman/SdFat/blob/m ... /bench.ino

I see that the SPI_CLOCK is set with:

Code: Select all

#define SPI_CLOCK SD_SCK_MHZ(50)

I'm wondering if it can be any faster on todays M4's. The comments suggest bumping this clock speed up until it stops working.

Also the Config file has a few performance options:

https://github.com/greiman/SdFat/blob/m ... atConfig.h

Code: Select all

 * If CHECK_FLASH_PROGRAMMING is zero, overlap of single sector flash
 * programming and other operations will be allowed for faster write
 * performance.
#define CHECK_FLASH_PROGRAMMING 1

set to '1' on these:

Code: Select all

//------------------------------------------------------------------------------
/**
 * Set USE_SEPARATE_FAT_CACHE nonzero to use a second 512 byte cache
 * for FAT16/FAT32 table entries.  This improves performance for large
 * writes that are not a multiple of 512 bytes.
 */
#ifdef __arm__
#define USE_SEPARATE_FAT_CACHE 1
#else  // __arm__
#define USE_SEPARATE_FAT_CACHE 0
#endif  // __arm__
//------------------------------------------------------------------------------
/**
 * Set USE_EXFAT_BITMAP_CACHE nonzero to use a second 512 byte cache
 * for exFAT bitmap entries.  This improves performance for large
 * writes that are not a multiple of 512 bytes.
 */
#ifdef __arm__
#define USE_EXFAT_BITMAP_CACHE 1
#else  // __arm__
#define USE_EXFAT_BITMAP_CACHE 0
#endif  // __arm__

set to '1'

Code: Select all

/**
 * Set MAINTAIN_FREE_CLUSTER_COUNT nonzero to keep the count of free clusters
 * updated.  This will increase the speed of the freeClusterCount() call
 * after the first call.  Extra flash will be required.
 */
#define MAINTAIN_FREE_CLUSTER_COUNT 0

Always start with a freshly formatted card.

Make sure you call sd.begin to read the config.

Code: Select all

sd.begin(SD_CONFIG);

[moonie223]

I am pretty sure the non-adafruit fork of SDfat will not work on adafruit's BSP. If you look in the issues/pull requests you can see where the split occurred.

https://github.com/greiman/SdFat/pull/142

When that pull/fork was made I don't think the adafruit BSP SPI libraries used DMA by default.

Also, I believe M4 really only supports 12MHz SPI, 24MHz is pretty iffy.

https://learn.adafruit.com/adafruit-fea ... h?view=all

Around the MAX SPI/MAX QSPI setting. If I recall it's something to do with how fast the micro can toggle the clock pin. Due to clock dividers 12MHz or 24MHz are the next closest clocks available.

https://github.com/arduino/ArduinoCore- ... -361976681

I've also got a SAME54 xplained pro board, and I've managed to make raw SDIO access work using the examples. After integrating a systick counter and other functions (I think they are working right?) I think I am topping out at ~1.3MB/s of write speed using a 50MHz clock speed and a 32kb write buffer.

https://www.alkgrove.com/articles/SAMD5 ... ticle.html

According to this (and microchip?) the speeds I am seeing are about as good as they get.

I am starting to wonder if those due access times are correct, otherwise I have no idea how I get so much less speed out of 4 times the bandwidth on double or more the clock rate. Those write times (3965kb/s) were supposedly done with (edit) 42MHz and 32kb write buffers...

[moonie223]

So I think I managed to make the new updated non-fork of SDfat work with SAMD board support package. Blocks were changed to sectors, and syncBlocks was changed to syncDevice. With some more small changes I've managed to get ~1.3mb/s write speeds, and it still works with tinyUSB MSC.

I've left the clock set at 50MHz. I believe the BSP ties this to the 48MHz clock generator. I am not actually 100% sure what clock speed is being used still, if there's a divider, but it works!

What a complicated and convoluted way to make a USB sd card reader!

[mikeysklar]

@moonie223,

Thanks for sharing this. I've had another post related to the same issue recently and it is encouraging that you can see such high write performance on the TinyUSB MSC.

Did you post your code, changes or patch anywhere for others to experiment with? I'm sure the first question is how reliable is the data read/write at the higher clock speed.

[moonie223]

I would have, but I didn't really change any code. I just removed the adafruit fork of SDfat and "installed" Bill Greiman's newest version instead. The examples included in SDfat work out of the box. The examples included in Tinyusb will not directly work, only because of the few small changes in names of a few things.

Code: Select all

// Callback invoked when received READ10 command.
// Copy disk's data to buffer (up to bufsize) and
// return number of copied bytes (must be multiple of block size)
int32_t msc_read_cb (uint32_t lba, void* buffer, uint32_t bufsize)
{
  return sd.card()->readSectors(lba, (uint8_t*) buffer, bufsize/512) ? bufsize : -1;
}

// Callback invoked when received WRITE10 command.
// Process data in buffer to disk's storage and 
// return number of written bytes (must be multiple of block size)
int32_t msc_write_cb (uint32_t lba, uint8_t* buffer, uint32_t bufsize)
{
  digitalWrite(LED_BUILTIN, HIGH);

  return sd.card()->writeSectors(lba, buffer, bufsize/512) ? bufsize : -1;
}

// Callback invoked when WRITE10 command is completed (status received and accepted by host).
// used to flush any pending cache.
void msc_flush_cb (void)
{
  sd.card()->syncDevice();

  // clear file system's cache to force refresh
  //sd.cacheClear();

  changed = true;

  digitalWrite(LED_BUILTIN, LOW);
}

Those are the big chunks. readBlocks/writeBlocks has changed to readSectors/writeSectors, and syncBlocks was changed to syncDevice.

Further up in the code I have changed to the new classes for exFat, as well as the new file class that has more features.

I have not tried spi flash support. I do not like the lack of wear leveling, so I won't be using it that way. I will probably use XIP or just use it as static flash for storing graphic data.

Code: Select all

// File system on SD Card
SdFs sd;

FsFile root;
FsFile file;

I also forgot, but I pulled all references to serial from the example. If usb serial is initialized MSC will not enumerate and just crashes everything. I still need to work on selectively enabling and disabling that, I guess.

In the background SDfat is using it's base arduino SPI driver still, I have not made a custom spi driver interface just yet. I still may change some stuff around and see if I can use callbacks to manage DMA writes, because I think the code just uses DMA and sits waiting till it's transfer is completed at the moment.

With the example running I was able to copy a lot of photos back and forth with no corruption. The SDfat bench.ino test will work, and is where I'm getting the write speeds from.

Code: Select all

Type any character to start
FreeStack: 156848
Type is FAT32
Card size: 3.90 GB (GB = 1E9 bytes)

Manufacturer ID: 0X2
OEM ID: TM
Product: SA04G
Version: 1.1
Serial number: 0X193D961E
Manufacturing date: 7/2013

FILE_SIZE_MB = 20
BUF_SIZE = 32768 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
1188.64,887496,24743,27581
1205.84,400051,24742,27187

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
1351.35,24261,24260,24261
1351.26,24263,24260,24262

Done
Type any character to start

With a smaller buffer/file size those reads are pretty quick!

Code: Select all

Type any character to start
FreeStack: 189104
Type is FAT32
Card size: 3.90 GB (GB = 1E9 bytes)

Manufacturer ID: 0X2
OEM ID: TM
Product: SA04G
Version: 1.1
Serial number: 0X193D961E
Manufacturing date: 7/2013

FILE_SIZE_MB = 1
BUF_SIZE = 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
1295.34,10706,378,394
856.90,137067,378,597

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
1314.06,382,380,389
1340.48,382,380,381

Done
Type any character to start