Adafruit is open and shipping! Let's build back better, together!
0

Benchmarks for CircuitPython?
Moderators: adafruit_support_bill, adafruit

Please be positive and constructive with your questions and comments.

Benchmarks for CircuitPython?

by kevinjwalters on Sat Oct 10, 2020 7:50 am

Are there any benchmarks for CircuitPython? I noticed something in HackSpace recently (edition 34, page 67) and I was trying to reproduce the numbers in it which didn't look quite right to me for the two M4-based procesors mentioned, nRF52840 (64MHz) and SAMD51 (120MHz).

hackspace-34-p67-benchmarktable.png.png
CircuitPython FP benchmark table from HackSpace Edition 34 page 67
hackspace-34-p67-benchmarktable.png.png (46.41 KiB) Viewed 194 times


Here's some very simple floating point tests, primary use here being to compare performance across boards. The speed difference here looks appropriate given the clock speeds of the two M4 boards. ALL running CircuitPython 5.3.1.

Feather M4 Express

Code: Select all | TOGGLE FULL SIZE
Adafruit CircuitPython 5.3.1 on 2020-07-13; Adafruit Feather M4 Express with samd51j19
>>>
>>> import simple_benchmarks
>>> simple_benchmarks.run()
platform: MicroChip SAMD51
version: 3.4.0
modules: {'simple_benchmarks': <module 'simple_benchmarks' from 'simple_benchmarks.py'>}
001AdditionA: 1.511477s (330802.226067Hz)
002MultiplicationA: 1.517326s (329526.972771Hz)
003DivisionA: 1.584962s (315464.997292Hz)
>>> simple_benchmarks.run()
platform: MicroChip SAMD51
version: 3.4.0
modules: {'simple_benchmarks': <module 'simple_benchmarks' from 'simple_benchmarks.py'>}
001AdditionA: 1.511183s (330866.622925Hz)
002MultiplicationA: 1.517409s (329509.019852Hz)
003DivisionA: 1.584792s (315498.876572Hz)


CLUE (alpha hardware)

Code: Select all | TOGGLE FULL SIZE
Adafruit CircuitPython 5.3.1 on 2020-07-13; Adafruit CLUE nRF52840 Express with nRF52840
>>>
>>> import simple_benchmarks
>>> simple_benchmarks.run()
platform: nRF52840
version: 3.4.0
modules: {'simple_benchmarks': <module 'simple_benchmarks' from 'simple_benchmarks.py'>}
001AdditionA: 3.610917s (138468.933105Hz)
002MultiplicationA: 3.594544s (139099.681854Hz)
003DivisionA: 3.706118s (134912.061691Hz)
>>> simple_benchmarks.run()
platform: nRF52840
version: 3.4.0
modules: {'simple_benchmarks': <module 'simple_benchmarks' from 'simple_benchmarks.py'>}
001AdditionA: 3.612694s (138400.876999Hz)
002MultiplicationA: 3.606740s (138629.317284Hz)
003DivisionA: 3.705397s (134938.311577Hz)


Circuit Playground Express (CPX)

Code: Select all | TOGGLE FULL SIZE
Adafruit CircuitPython 5.3.1 on 2020-07-13; Adafruit CircuitPlayground Express with samd21g18
>>>
>>> import simple_benchmarks
>>> simple_benchmarks.run()
platform: Atmel SAMD21
version: 3.4.0
modules: {'simple_benchmarks': <module 'simple_benchmarks' from 'simple_benchmarks.py'>}
001AdditionA: 9.364471s (53393.297195Hz)
002MultiplicationA: 10.113869s (49437.060356Hz)
003DivisionA: 13.884789s (36010.622978Hz)
>>> simple_benchmarks.run()
platform: Atmel SAMD21
version: 3.4.0
modules: {'simple_benchmarks': <module 'simple_benchmarks' from 'simple_benchmarks.py'>}
001AdditionA: 9.364391s (53393.750191Hz)
002MultiplicationA: 10.113922s (49436.793327Hz)
003DivisionA: 13.884777s (36010.656357Hz)


Code is in https://github.com/kevinjwalters/circui ... chmarks.py

kevinjwalters
 
Posts: 834
Joined: Sun Oct 01, 2017 3:15 pm

Re: Benchmarks for CircuitPython?

by tannewt on Mon Oct 12, 2020 4:51 pm

We don't maintain or track any ourselves. Generally, it can be a bit of a trap because there is always performance to gain at an increasing cost.

I've tweeted at Hackspace about how they got their numbers: https://twitter.com/tannewt/status/1315754129206042624

tannewt
 
Posts: 2010
Joined: Thu Oct 06, 2016 8:48 pm

Re: Benchmarks for CircuitPython?

by kevinjwalters on Fri Jan 08, 2021 8:42 am

The ESP32-S2 numbers also looked odd in that HackSpace benchmark but I've only just got a board with that chip to puts its floating point through its paces:

FeatherS2 (ESP32-S2)

Code: Select all | TOGGLE FULL SIZE
sys.version: 3.4.0
sys.modules: {}
os.sysname: esp32s2
os.nodename: esp32s2
os.release: 6.0.0
os.version: 6.0.0 on 2020-11-16
os.machine: FeatherS2 with ESP32S2
001AdditionA: 1.4886s,1.3293s,1.4772s,1.3329s,1.4849s (14059Hz)
002MultiplicationA: 1.3424s,1.5280s,1.3663s,1.3788s,1.3277s (14403Hz)
003DivisionA: 1.9622s,1.9824s,1.9818s,1.9905s,1.9863s (10098Hz)
004ArchimedesPiA: 1.3958s,1.3141s,1.4156s,1.2751s,1.4064s (147Hz)


There's something curious going on with the addition and multiplication, it's almost like they are getting scheduled onto two different units running at different speeds.

micro:bit V1

Code: Select all | TOGGLE FULL SIZE
sys.platform: microbit
sys.version: 3.4.0
os.sysname: microbit
os.nodename: microbit
os.release: 1.0.1
os.version: micro:bit v1.0.1+b0bf4a9 on 2018-12-13; MicroPython v1.9.2-34-gd64154c73 on 2017-09-01
os.machine: micro:bit with nRF51822
001AdditionA: 20.8069s,20.8069s,20.8070s,20.8069s,20.8070s (961Hz)
002MultiplicationA: 22.4981s,22.4982s,22.4982s,22.4982s,22.4982s (888Hz)
003DivisionA: 35.0681s,35.0681s,35.0682s,35.0682s,35.0682s (570Hz)
004ArchimedesPiA: 18.7679s,18.7679s,18.7679s,18.7680s,18.7679s (11Hz)



micro:bit V2

I'd assumed the nRF52833 would have very similar performance to nRF52840 as they are both down as "64 MHz Arm Cortex-M4 with FPU" but apparently not

Code: Select all | TOGGLE FULL SIZE
sys.platform: microbit
sys.version: 3.4.0
sys.modules: {}
os.sysname: microbit
os.nodename: microbit
os.release: 2.0.0-beta.3
os.version: micro:bit v2.0.0-beta.3+d6c01d0 on 2020-12-21; MicroPython v1.13 on 2020-12-21
os.machine: micro:bit with nRF52833
001AdditionA: 7.6184s,7.6231s,7.6225s,7.6213s,7.6215s (2624Hz)
002MultiplicationA: 7.2355s,7.2386s,7.2354s,7.2372s,7.2362s (2764Hz)
003DivisionA: 9.2329s,9.2380s,9.2330s,9.2333s,9.2348s (2166Hz)
004ArchimedesPiA: 5.6066s,5.6113s,5.6098s,5.6072s,5.6111s (36Hz)

kevinjwalters
 
Posts: 834
Joined: Sun Oct 01, 2017 3:15 pm

Re: Benchmarks for CircuitPython?

by kevinjwalters on Sat Jan 09, 2021 12:36 pm

6.1.0-rc0 has made the FeatherS2 more consistent and faster! Or perhaps time.monotonic_ns() was buggy on ESP32-S2 6.0.0 and these have been fixed??

FeatherS2 (ESP32-S2)

Code: Select all | TOGGLE FULL SIZE
sys.version: 3.4.0
sys.modules: {}
os.sysname: esp32s2
os.nodename: esp32s2
os.release: 6.1.0
os.version: 6.1.0-rc.0 on 2021-01-06
os.machine: FeatherS2 with ESP32S2
001AdditionA: 0.9988s,0.9986s,0.9985s,0.9986s,0.9985s (20028Hz)
002MultiplicationA: 1.0196s,1.0194s,1.0193s,1.0193s,1.0194s (19619Hz)
003DivisionA: 1.5855s,1.5853s,1.5852s,1.5854s,1.5853s (12616Hz)
004ArchimedesPiA: 1.3531s,1.3614s,1.3529s,1.3530s,1.3699s (147Hz)
005MixedBigIntA: 2.7977s,2.9087s,2.7851s,2.9026s,2.8022s (1268Hz)

kevinjwalters
 
Posts: 834
Joined: Sun Oct 01, 2017 3:15 pm

Re: Benchmarks for CircuitPython?

by kevinjwalters on Thu Jan 14, 2021 10:27 am


kevinjwalters
 
Posts: 834
Joined: Sun Oct 01, 2017 3:15 pm

Please be positive and constructive with your questions and comments.