0

Build for board removing modules DEF missing
Moderators: adafruit_support_bill, adafruit

Please be positive and constructive with your questions and comments.

Build for board removing modules DEF missing

by Kangus on Tue Mar 30, 2021 12:10 pm

While reading : Adding Frozen Modules I found
# Make room for frozen libs.

CIRCUITPY_BITMAPTOOLS = 0

Which stops the module bitmaptools from being added to the board build. I tried to find a list of module names and their DEF labels but found nothing.
Here's what I have so far but it is missing a lot:
(and no, the Core Modules Matrix does not have the LABEL listed with each modules name.)

Code: Select all | TOGGLE FULL SIZE
MODULE                        DEF LABEL (or what-ever they are called)
_bleio            ??????????????????
_pixelbuf         ??????????????????
adafruit_bus_device     ???????????????????
aesio                               ?
alarm                               CIRCUITPY_ALARM
analogio                 CIRCUITPY_ANALOGIO
audiobusio         CIRCUITPY_AUDIOBUSIO
audiocore         CIRCUITPY_AUDIOCORE
audiomp3         CIRCUITPY_AUDIOMP3
audiomixer                     CIRCUITPY_AUDIOMIXER
audiopwmio         CIRCUITPY_AUDIOPWMIO
binascii                 ?????????????????
bitbangio         CIRCUITPY_BITBANGIO
bitmaptools         CIRCUITPY_BITMAPTOOLS
bitops            ??????????????????
board            CIRCUITPY_BOARD
busio            CIRCUITPY_BUSIO
countio            CIRCUITPY_COUNTIO
camera            CIRCUITPY_CAMERA
canio            CIRCUITPY_CANIO
digitalio                 CIRCUITPY_DIGITALIO         
displayio         CIRCUITPY_DISPLAYIO         
errno            ???????????????????
fontio                              ?
framebufferio              CIRCUITPY_FRAMEBUFFERIO
gamepad         CIRCUITPY_GAMEPAD
gameshift         CIRCUITPY_GAMEPADSHIFT
gnss            CIRCUITPY_GNSS
ipaddress         CIRCUITPY_IPADDRESS
i2cperipheral         CIRCUITPY_I2CPERIPHERAL
json            ?????????????????????
math            CIRCUITPY_MATH
microcontroller      CIRCUITPY_MICROCONTROLLER
memorymonitor      CIRCUITPY_MEMORYMONITO
network            CIRCUITPY_NETWORK
msgpack         CIRCUITPY_MSGPACK
neopixel_write      CIRCUITPY_NEOPIXEL_WRITE
nvm            CIRCUITPY_NVM
os               CIRCUITPY_OS
pulseio            CIRCUITPY_PULSEIO
pwmio            CIRCUITPY_PWMIO
random            CIRCUITPY_RANDOM
re               ??????????????
rgbmatrix         CIRCUITPY_RGBMATRIX
rotaryio                 CIRCUITPY_ROTARYIO
rtc               CIRCUITPY_RTC
ps2io            CIRCUITPY_PS2IO
rp2pio            CIRCUITPY_RP2PIO
samd            CIRCUITPY_SAMD
sdoio            CIRCUITPY_SDIOIO
sdcardio                 CIRCUITPY_SDCARDIO
sharpdisplay              CIRCUITPY_SHARPDISPLAY
socket                             CIRCUITPY_SOCKET  ???? I added this, wild guess
socketpool         CIRCUITPY_SOCKETPOO
ssl               CIRCUITPY_SSL
storage            CIRCUITPY_STORAGE
struct            CIRCUITPY_STRUCT
supervisor         CIRCUITPY_SUPERVISO
terminalio         CIRCUITPY_TERMINALIO
time            CIRCUITPY_TIME
touchio            CIRCUITPY_TOUCHIO            
ulab            CIRCUITPY_ULAB
usb_hid            CIRCUITPY_USB_HID
usb_midi         CIRCUITPY_USB_MIDI
vectorio                 CIRCUITPY_VECTORIO
watchdog         CIRCUITPY_WATCHDO
________________________________________            
Need the module names
               MICROPY_PY_ASYNC_AWAIT
               CIRCUITPY_AESIO
               CIRCUITPY_AUDIOBUSIO_I2SOUT
               CIRCUITPY_AUDIOBUSIO_PDMIN
               CIRCUITPY_AUDIOIO
               CIRCUITPY_BLEIO_HCI
               CIRCUITPY_BLEIO
               CIRCUITPY_BUSDEVICE
               CIRCUITPY_BUILTINS_POW3
               CIRCUITPY_ESPIDF
               CIRCUITPY_FREQUENCYIO
               CIRCUITPY__EVE
               CIRCUITPY_DUALBANK
               CIRCUITPY_PIXELBUF
               CIRCUITPY_WIFI
               CIRCUITPY_STAGE
               CIRCUITPY_UHEAP
               CIRCUITPY_PEW
               CIRCUITPY_USTACK
               CIRCUITPY_BITBANG_APA102
               CIRCUITPY_REQUIRE_I2C_PULLUPS
               CIRCUITPY_SERIAL_BLE
               CIRCUITPY_BLE_FILE_SERVICE
               CIRCUITPY_SERIAL_UART
               SUPEROPT_GC

Kangus
 
Posts: 38
Joined: Sat May 19, 2018 10:01 pm

Re: Build for board removing modules DEF missing

by tannewt on Tue Mar 30, 2021 1:22 pm

The mapping is done here: https://github.com/adafruit/circuitpyth ... fig.h#L317

The module name you see in CircuitPython is the MP_QSTR_ portion.

Almost all defs are CIRCUITPY_<module name>. If it starts with _ then it'll be omitted. Modules that mirror CPython and are inherited from MicroPython likely won't follow this pattern.

tannewt
 
Posts: 2428
Joined: Thu Oct 06, 2016 8:48 pm

Re: Build for board removing modules DEF missing

by Kangus on Wed Mar 31, 2021 12:06 am

I searched my fork for MICROPY_PY_ASYNC_AWAIT and of course that is not in the circuitpy_mpconfig.h

At any rate I spent the afternoon removing modules in groups:
Problem modules - remove any one of them and the build breaks.

#CIRCUITPY_DISPLAYIO = 0
#CIRCUITPY_BITBANGIO=0
#CIRCUITPY_NEOPIXEL_WRITE=0
#CIRCUITPY_RTC=0

With the below removed there is 388600 flash used

# Make room for frozen libs.
CIRCUITPY_FRAMEBUFFERIO = 0
CIRCUITPY_BITMAPTOOLS = 0
#CIRCUITPY_DISPLAYIO = 0
CIRCUITPY_FREQUENCYIO = 0
CIRCUITPY_I2CPERIPHERAL = 0
CIRCUITPY_MSGPACK = 0
MICROPY_PY_ASYNC_AWAIT = 0
CIRCUITPY_AUDIOBUSIO=0
CIRCUITPY_AUDIOCORE=0
CIRCUITPY_AUDIOMP3=0
CIRCUITPY_AUDIOMIXER=0
CIRCUITPY_AUDIOPWMIO=0
#CIRCUITPY_BITBANGIO=0
CIRCUITPY_BITMAPTOOLS=0
CIRCUITPY_COUNTIO=0
CIRCUITPY_CAMERA=0
CIRCUITPY_CANIO=0
#
CIRCUITPY_GAMEPAD=0
CIRCUITPY_GAMEPADSHIFT=0
CIRCUITPY_GNSS=0
CIRCUITPY_IPADDRESS=0
CIRCUITPY_I2CPERIPHERAL=0
CIRCUITPY_MATH=0
#
CIRCUITPY_NETWORK=0
CIRCUITPY_MSGPACK=0
#CIRCUITPY_NEOPIXEL_WRITE=0
CIRCUITPY_PULSEIO=0
CIRCUITPY_PWMIO=0
CIRCUITPY_RANDOM=0
CIRCUITPY_RGBMATRIX=0
CIRCUITPY_ROTARYIO=0
#CIRCUITPY_RTC=0
CIRCUITPY_PS2IO=0
#
CIRCUITPY_SHARPDISPLAY=0
CIRCUITPY_SOCKET=0
CIRCUITPY_SOCKETPOO=0
CIRCUITPY_SSL=0
CIRCUITPY_STRUCT=0
CIRCUITPY_TERMINALIO=0
#
CIRCUITPY_TOUCHIO=0
CIRCUITPY_ULAB=0
CIRCUITPY_USB_HID=0
CIRCUITPY_USB_MIDI=0
CIRCUITPY_VECTORIO=0
#
CIRCUITPY_AESIO=0
CIRCUITPY_AUDIOBUSIO_I2SOUT=0
CIRCUITPY_AUDIOBUSIO_PDMIN=0
CIRCUITPY_AUDIOIO=0
CIRCUITPY_BLEIO_HCI=0
CIRCUITPY_BLEIO=0
CIRCUITPY_BUSDEVICE=0
CIRCUITPY_BUILTINS_POW3=0
CIRCUITPY_ESPIDF=0
CIRCUITPY__EVE=0
CIRCUITPY_DUALBANK=0
CIRCUITPY_PIXELBUF=0
CIRCUITPY_WIFI=0
CIRCUITPY_STAGE=0
CIRCUITPY_UHEAP=0
CIRCUITPY_PEW=0
CIRCUITPY_USTACK=0
#
CIRCUITPY_BITBANG_APA102=0
CIRCUITPY_REQUIRE_I2C_PULLUPS=0
CIRCUITPY_SERIAL_BLE=0
CIRCUITPY_BLE_FILE_SERVICE=0
CIRCUITPY_SERIAL_UART=0
SUPEROPT_GC=0
#

If you figure out why I can't remove those four let me know, Thanks

Kangus
 
Posts: 38
Joined: Sat May 19, 2018 10:01 pm

Re: Build for board removing modules DEF missing

by tannewt on Wed Mar 31, 2021 11:29 am

Please post the compilation errors for each.

tannewt
 
Posts: 2428
Joined: Thu Oct 06, 2016 8:48 pm

Re: Build for board removing modules DEF missing

by tannewt on Wed Mar 31, 2021 11:50 am


tannewt
 
Posts: 2428
Joined: Thu Oct 06, 2016 8:48 pm

Re: Build for board removing modules DEF missing

by Kangus on Sat Apr 03, 2021 10:39 pm

I spent Saturday redoing my circuitpython install making sure I had the latest repo, I still can not change anything in the \\wsl\Ubuntu-20.04\home\kevin\circuitpython\ports\raspberrypi\boards\adafruit_feather_rp2040\mpconfigboard.mk without breaking the build

build.txt
The build.txt is my attempt to understand setting up circuitpython to build and add to the repo.
(3.68 KiB) Downloaded 1 time

Kangus
 
Posts: 38
Joined: Sat May 19, 2018 10:01 pm

Re: Build for board removing modules DEF missing

by danhalbert on Sun Apr 04, 2021 9:28 am

Not all the CIRCUITPY_* names correspond to individual modules. Please post your error messages. It's not possible to turn off everything without breaking the build in some cases. For instance, CIRCUITPY_BITBANGIO is needed at a low level to do the RGB LED when it's a DotStar.

It could be a goal to document the module names along with more details of what needs what in circuitpy_mpconfig.mk. You are welcome to open an issue asking for what.

danhalbert
 
Posts: 2430
Joined: Tue Aug 08, 2017 12:37 pm

Re: Build for board removing modules DEF missing

by Kangus on Sun Apr 04, 2021 12:18 pm

In "circuitpython\ports\raspberrypi\boards\adafruit_feather_rp2040\mpconfigboard.mk":
Code: Select all | TOGGLE FULL SIZE
USB_VID = 0x239A
USB_PID = 0x80F2
USB_PRODUCT = "Feather RP2040"
USB_MANUFACTURER = "Adafruit"

CHIP_VARIANT = RP2040
CHIP_FAMILY = rp2

EXTERNAL_FLASH_DEVICES = "GD25Q64C"


I want to disable CIRCUITPY_DISPLAYIO = 0

In "circuitpython\ports\raspberrypi\mpconfigport.mk":

Code: Select all | TOGGLE FULL SIZE
# Define an equivalent for MICROPY_LONGINT_IMPL, to pass to $(MPY-TOOL) in py/mkrules.mk
# $(MPY-TOOL) needs to know what kind of longint to use (if any) to freeze long integers.
# This should correspond to the MICROPY_LONGINT_IMPL definition in mpconfigport.h.

ifeq ($(LONGINT_IMPL),NONE)
MPY_TOOL_LONGINT_IMPL = -mlongint-impl=none
endif

ifeq ($(LONGINT_IMPL),MPZ)
MPY_TOOL_LONGINT_IMPL = -mlongint-impl=mpz
endif

ifeq ($(LONGINT_IMPL),LONGLONG)
MPY_TOOL_LONGINT_IMPL = -mlongint-impl=longlong
endif

# All raspberrypi ports have longints.
LONGINT_IMPL = MPZ

CIRCUITPY_RP2PIO ?= 1
CIRCUITPY_NEOPIXEL_WRITE ?= $(CIRCUITPY_RP2PIO)
CIRCUITPY_FRAMEBUFFERIO ?= 1
CIRCUITPY_FULL_BUILD ?= 1
CIRCUITPY_AUDIOMP3 ?= 1
CIRCUITPY_BITOPS ?= 1
CIRCUITPY_PWMIO ?= 1
CIRCUITPY_RGBMATRIX ?= 1
CIRCUITPY_ROTARYIO ?= 1

# Things that need to be implemented.
# Use PWM interally
CIRCUITPY_FREQUENCYIO = 0
CIRCUITPY_I2CPERIPHERAL = 0
CIRCUITPY_NVM = 1
# Use PIO interally
CIRCUITPY_PULSEIO ?= 1
CIRCUITPY_WATCHDOG ?= 1

# Audio via PWM
CIRCUITPY_AUDIOIO = 0
CIRCUITPY_AUDIOBUSIO ?= 1
CIRCUITPY_AUDIOCORE ?= 1
CIRCUITPY_AUDIOPWMIO ?= 1

# These libraries require Cortex M4+ for fancy math instructions.
CIRCUITPY_AUDIOMIXER ?= 0

INTERNAL_LIBM = 1

USB_SERIAL_NUMBER_LENGTH = 16

# Number of USB endpoint pairs.
USB_NUM_EP = 8

INTERNAL_FLASH_FILESYSTEM = 1


Where is ?=1 documented?

The Error: First I build with the default settings, clean build no issues, Then I add CIRCUITPY_DISPLAYIO = 0 to mpconfigboard.mk and when I build I get error: array subscript 0

Code: Select all | TOGGLE FULL SIZE
kevin@Aorus:~/circuitpython/ports/raspberrypi$ make V=1 clean BOARD=adafruit_feather_rp2040
rm -rf build-adafruit_feather_rp2040
kevin@Aorus:~/circuitpython/ports/raspberrypi$ make -j6 BOARD=adafruit_feather_rp2040
Use make V=1, make V=2 or set BUILD_VERBOSE similarly in your environment to increase build verbosity.
{'sku': ['GD25Q64C']}
   text    data     bss     dec     hex filename
    244       0       0     244      f4 build-adafruit_feather_rp2040/boot2.elf
QSTR updated

628936 bytes used, 415544 bytes free in flash firmware space out of 1044480 bytes (1020.0kB).
13928 bytes used, 248216 bytes free in ram for stack and heap out of 262144 bytes (256.0kB).

Converting to uf2, output size: 1253888, start address: 0x10000000
Wrote 1253888 bytes to build-adafruit_feather_rp2040/firmware.uf2
kevin@Aorus:~/circuitpython/ports/raspberrypi$ make V=1 clean BOARD=adafruit_feather_rp2040
rm -rf build-adafruit_feather_rp2040
kevin@Aorus:~/circuitpython/ports/raspberrypi$ make -j6 BOARD=adafruit_feather_rp2040
Use make V=1, make V=2 or set BUILD_VERBOSE similarly in your environment to increase build verbosity.
{'sku': ['GD25Q64C']}
   text    data     bss     dec     hex filename
    244       0       0     244      f4 build-adafruit_feather_rp2040/boot2.elf
QSTR updated
../../shared-module/framebufferio/FramebufferDisplay.c: In function 'common_hal_framebufferio_framebufferdisplay_construct':
../../shared-module/framebufferio/FramebufferDisplay.c:283:17: error: array subscript 0 is outside array bounds of 'primary_display_t[0]' [-Werror=array-bounds]
  283 |     if (self == &displays[0].framebuffer_display) {
      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../../shared-module/framebufferio/FramebufferDisplay.c:34:
../../shared-module/displayio/__init__.h:69:26: note: while referencing 'displays'
   69 | extern primary_display_t displays[CIRCUITPY_DISPLAY_LIMIT];
      |                          ^~~~~~~~
../../shared-module/framebufferio/FramebufferDisplay.c: In function 'common_hal_framebufferio_framebufferdisplay_set_rotation':
../../shared-module/framebufferio/FramebufferDisplay.c:283:17: error: array subscript 0 is outside array bounds of 'primary_display_t[0]' [-Werror=array-bounds]
  283 |     if (self == &displays[0].framebuffer_display) {
      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../../shared-module/framebufferio/FramebufferDisplay.c:34:
../../shared-module/displayio/__init__.h:69:26: note: while referencing 'displays'
   69 | extern primary_display_t displays[CIRCUITPY_DISPLAY_LIMIT];
      |                          ^~~~~~~~
cc1: all warnings being treated as errors
make: *** [../../py/mkrules.mk:55: build-adafruit_feather_rp2040/shared-module/framebufferio/FramebufferDisplay.o] Error 1
make: *** Waiting for unfinished jobs....
kevin@Aorus:~/circuitpython/ports/raspberrypi$


If I add CIRCUITPY_FRAMEBUFFER = 0 to mpconfigboard.mk I lose the framebuffer error and get:

Code: Select all | TOGGLE FULL SIZE
kevin@Aorus:~/circuitpython/ports/raspberrypi$ make -j16 BOARD=adafruit_feather_rp2040
Use make V=1, make V=2 or set BUILD_VERBOSE similarly in your environment to increase build verbosity.
{'sku': ['GD25Q64C']}
   text    data     bss     dec     hex filename
    244       0       0     244      f4 build-adafruit_feather_rp2040/boot2.elf
QSTR updated
/home/kevin/gcc-arm-none-eabi-10-2020-q4-major/bin/../lib/gcc/arm-none-eabi/10.2.1/../../../../arm-none-eabi/bin/ld: build-adafruit_feather_rp2040/shared-bindings/rgbmatrix/RGBMatrix.o: in function `rgbmatrix_rgbmatrix_make_new':
RGBMatrix.c:(.text.rgbmatrix_rgbmatrix_make_new+0x2c): undefined reference to `allocate_display_bus_or_raise'
collect2: error: ld returned 1 exit status
make: *** [Makefile:286: build-adafruit_feather_rp2040/firmware.elf] Error 1


And finally I add CIRCUITPY_RGBMATRIX = 0 to mpconfigboard.mk and get a clean build!

I take it there is no dependency checking during build?
A database a a couple thousand lines of code should fix this.

This is now fixed with the inclusion of additional mpconfigboard statements I'll post an issue requesting documentation on the dependencies of modules
Last edited by Kangus on Sun Apr 04, 2021 12:40 pm, edited 1 time in total.

Kangus
 
Posts: 38
Joined: Sat May 19, 2018 10:01 pm

Re: Build for board removing modules DEF missing

by danhalbert on Sun Apr 04, 2021 12:32 pm

CIRCUITPY_FRAMEBUFFERIO should probably not be mentioned by itself in mpconfigport.mk; that is an error, because it's interdependent with DISPLAYIO. We'll investigate that more -- thanks

`?=` is a Gnu Make feature: See the bottom of the page here: https://www.gnu.org/software/make/manua ... ml#Flavors

For now set both CIRCUITPY_DISPLAYIO and CIRCUITPY_FRAMEBUFFERIO = 0. I think that should fix this error.

danhalbert
 
Posts: 2430
Joined: Tue Aug 08, 2017 12:37 pm

Re: Build for board removing modules DEF missing

by Kangus on Sun Apr 04, 2021 12:45 pm

And finally I add CIRCUITPY_RGBMATRIX = 0 to mpconfigboard.mk and get a clean build!



The link to GNU make helps but this is approaching Zephyr territory, the build system eclipses the programming effort in terms of the amount of time spent learning. Such is life in the 2021 era

Kangus
 
Posts: 38
Joined: Sat May 19, 2018 10:01 pm

Re: Build for board removing modules DEF missing

by danhalbert on Sun Apr 04, 2021 2:26 pm

We have been trying to make things better; the enabling/disabling controls are a lot smoother than they used to be, but there are dependencies between them that need to be considered, and sometimes we forget.

danhalbert
 
Posts: 2430
Joined: Tue Aug 08, 2017 12:37 pm

Please be positive and constructive with your questions and comments.