Ampy with ESP32 Feather V2

[crcibernetica]

I'm trying to automate library and file uploads to ESP32 Feather V2 boards with CircuitPython 8.0.3. Ampy gives me a "could not enter raw repl" error. Ampy still works with Micropython installed on the same board but not with CircuitPython.
Does CircuitPython support raw_repl mode?
Are there any other tools for uploading files over USB serial using python?

[crcibernetica]

The idea is to create a script to install all the libraries and boilerplate files for a class of 20 to 30. I have tried all the tips regarding adding delays to the amp commands but I always get the 'could not enter raw repl' error.

[jerryn]

You may want to look into using rshell : https://github.com/dhylands/rshell

[crcibernetica]

Thanks! Both ampy and rshell use pyboard.py to communicate with the devices and give the same error message. Something with communication using raw repl seems to have been broken in CircuitPython. Both ampy and rshell work fine with micropython installed on the same devices.

[jerryn]

I am able to get rshell to work with an esp32 v2 but I had to install rshell from the repository.
An update to allow it to work with Circuitpython has been merged, but not released
Here is what I did.

Code: Select all

git clone https://github.com/dhylands/rshell.git
cd rshell 
pip install .


jerryneedell@Mac-mini rshell % rshell -p /dev/tty.usbserial-21430              
Using buffer-size of 32
Connecting to /dev/tty.usbserial-21430 (buffer-size 32)...
Trying to connect to REPL  connected
Retrieving sysname ... ESP32
Testing if ubinascii.unhexlify exists ... Y
Retrieving root directories ... /.fseventsd/ /.metadata_never_index/ /.Trashes/ /settings.toml/ /code.py/ /lib/ /boot_out.txt/
Setting time ... Mar 12, 2023 21:38:29
Evaluating board_name ... pyboard
Retrieving time epoch ... Jan 01, 2000
Welcome to rshell. Use Control-D (or the exit command) to exit rshell.

/Users/jerryneedell/projects/rshell> ls /pyboard
lib/          code.py       boot_out.txt  settings.toml
/Users/jerryneedell/projects/rshell> repl
Entering REPL. Use Control-X to exit.
>

Adafruit CircuitPython 8.1.0-beta.0 on 2023-03-01; Adafruit Feather ESP32 V2 with ESP32
>>> 
>>> 
>>> 
>>> 
>>>  

edited to add: this PR has been merged into rshell, but a new release has not been made.
https://github.com/dhylands/rshell/pull/181

[crcibernetica]

I followed those instructions and get the following error:
NOTE that where it says BANNED is really \n raw (with no space after the 'n')

Code: Select all

C:\Users\bentl\Documents\Thonny\rshell>rshell -p COM3
Using buffer-size of 32
Connecting to COM3 (buffer-size 32)...
Trying to connect to REPL  connected
Retrieving sysname ... b'OK\r\BANNED REPL; CTRL-B to exit\r\n>'
Traceback (most recent call last):
  File "C:\Users\bentl\AppData\Local\Programs\Thonny\lib\site-packages\rshell\main.py", line 1390, in connect
    ip_address = socket.gethostbyname(port)
socket.gaierror: [Errno 11001] getaddrinfo failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\bentl\AppData\Local\Programs\Thonny\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\bentl\AppData\Local\Programs\Thonny\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\bentl\AppData\Local\Programs\Thonny\Scripts\rshell.exe\__main__.py", line 7, in <module>
  File "C:\Users\bentl\AppData\Local\Programs\Thonny\lib\site-packages\rshell\command_line.py", line 4, in main
    rshell.main.main()
  File "C:\Users\bentl\AppData\Local\Programs\Thonny\lib\site-packages\rshell\main.py", line 3091, in main
    real_main()
  File "C:\Users\bentl\AppData\Local\Programs\Thonny\lib\site-packages\rshell\main.py", line 3053, in real_main
    connect(args.port, baud=args.baud, wait=args.wait, user=args.user, password=args.password)
  File "C:\Users\bentl\AppData\Local\Programs\Thonny\lib\site-packages\rshell\main.py", line 1396, in connect
    connect_serial(port, baud=baud, wait=wait)
  File "C:\Users\bentl\AppData\Local\Programs\Thonny\lib\site-packages\rshell\main.py", line 1420, in connect_serial
    dev = DeviceSerial(port, baud, wait)
  File "C:\Users\bentl\AppData\Local\Programs\Thonny\lib\site-packages\rshell\main.py", line 1751, in __init__
    Device.__init__(self, pyb)
  File "C:\Users\bentl\AppData\Local\Programs\Thonny\lib\site-packages\rshell\main.py", line 1504, in __init__
    self.sysname = self.remote_eval(sysname)
  File "C:\Users\bentl\AppData\Local\Programs\Thonny\lib\site-packages\rshell\main.py", line 1639, in remote_eval
    return eval(self.remote(func, *args, **kwargs))
  File "C:\Users\bentl\AppData\Local\Programs\Thonny\lib\site-packages\rshell\main.py", line 1617, in remote
    self.pyb.enter_raw_repl()
  File "C:\Users\bentl\AppData\Local\Programs\Thonny\lib\site-packages\rshell\pyboard.py", line 203, in enter_raw_repl
    raise PyboardError('could not enter raw repl')
rshell.pyboard.PyboardError: could not enter raw repl

[jerryn]

I wish I had more to offer. I realize that “It works for me” is not very helpful to you, but I don’t know what else to suggest.

I don’t understand where the “raw REPL” comes into play at this point. I dont see and reference to it in my example.

Are you able to connect to the REPL at all with something like Putty? I am not on a Windows system and I don’t know if that is relevant…

Just to be clear, I do not represent Adafruit.

[jerryn]

A Ha! I had been using 8.1.0-beta -- If I revert to 8.0.3 I can reproduce your failure

Code: Select all

jerryneedell@Mac-mini esp32v2 % rshell -p /dev/tty.usbserial-21430
Using buffer-size of 32
Connecting to /dev/tty.usbserial-21430 (buffer-size 32)...
Trying to connect to REPL  connected
Retrieving sysname ... b'OK\r\BANNED REPL; CTRL-B to exit\r\n>'
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.10/site-packages/rshell/main.py", line 1390, in connect
    ip_address = socket.gethostbyname(port)
socket.gaierror: [Errno 8] nodename nor servname provided, or not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/homebrew/bin/rshell", line 8, in <module>
    sys.exit(main())
  File "/opt/homebrew/lib/python3.10/site-packages/rshell/command_line.py", line 4, in main
    rshell.main.main()
  File "/opt/homebrew/lib/python3.10/site-packages/rshell/main.py", line 3091, in main
    real_main()
  File "/opt/homebrew/lib/python3.10/site-packages/rshell/main.py", line 3053, in real_main
    connect(args.port, baud=args.baud, wait=args.wait, user=args.user, password=args.password)
  File "/opt/homebrew/lib/python3.10/site-packages/rshell/main.py", line 1396, in connect
    connect_serial(port, baud=baud, wait=wait)
  File "/opt/homebrew/lib/python3.10/site-packages/rshell/main.py", line 1420, in connect_serial
    dev = DeviceSerial(port, baud, wait)
  File "/opt/homebrew/lib/python3.10/site-packages/rshell/main.py", line 1751, in __init__
    Device.__init__(self, pyb)
  File "/opt/homebrew/lib/python3.10/site-packages/rshell/main.py", line 1504, in __init__
    self.sysname = self.remote_eval(sysname)
  File "/opt/homebrew/lib/python3.10/site-packages/rshell/main.py", line 1639, in remote_eval
    return eval(self.remote(func, *args, **kwargs))
  File "/opt/homebrew/lib/python3.10/site-packages/rshell/main.py", line 1617, in remote
    self.pyb.enter_raw_repl()
  File "/opt/homebrew/lib/python3.10/site-packages/rshell/pyboard.py", line 203, in enter_raw_repl
    raise PyboardError('could not enter raw repl')
rshell.pyboard.PyboardError: could not enter raw repl

but with 8.1.0-beta.0

Code: Select all

jerryneedell@Mac-mini esp32v2 % rshell -p /dev/tty.usbserial-21430
Using buffer-size of 32
Connecting to /dev/tty.usbserial-21430 (buffer-size 32)...
Trying to connect to REPL  connected
Retrieving sysname ... ESP32
Testing if ubinascii.unhexlify exists ... Y
Retrieving root directories ... /.fseventsd/ /.metadata_never_index/ /.Trashes/ /settings.toml/ /code.py/ /lib/ /boot_out.txt/
Setting time ... Mar 13, 2023 06:01:51
Evaluating board_name ... pyboard
Retrieving time epoch ... Jan 01, 2000
Welcome to rshell. Use Control-D (or the exit command) to exit rshell.
/Users/jerryneedell/projects/esp32v2> repl
Entering REPL. Use Control-X to exit.
>

Adafruit CircuitPython 8.1.0-beta.0 on 2023-03-01; Adafruit Feather ESP32 V2 with ESP32
>>> 
>>> 

Can you try 8.1.0-beta.0?

edited to add: I opened an issue in the CircuitPython repository to report this see: https://github.com/adafruit/circuitpython/issues/7720

[jerryn]

BTW -- there is also a pending PR to implement the same fix in ampy https://github.com/scientifichackers/ampy/pull/111 -- it has not been merged but you can implement it locally if you prefer to use ampy.

I tested it with the feather esp32v2 and CP 8.1.0-beta.0 and it does work with the PR applied.

For this test I just did a

Code: Select all

pip install ampy

to get the released version of ampy then I manually implemented the change in the PR to the installed ampy/files.py. Was not able to use the current master branch from the ampy repository and I have not had a chance to look into it. There was some issue with the "progressbar" module with the master branch.

I prefer to use rshell.

[jerryn]

ah -- OK -- to use the current ampy master branch, you also have to implement the fix in issue#117 https://github.com/scientifichackers/ampy/issues/117

So with PR #111 and issue #117 applied, ampy now works with CP 8.1.0-beta.0 on the feather esp32-v2

Code: Select all

jerryneedell@Mac-mini ampy % ampy -p /dev/tty.usbserial-21430 -b 115200 ls
/.Trashes
/.fseventsd
/.metadata_never_index
/boot_out.txt
/code.py
/lib
/settings.toml
/test.test
jerryneedell@Mac-mini ampy % ls
LICENSE			adafruit_ampy.egg-info	build			setup.cfg		tests
README.md		ampy			requirements.txt	setup.py
jerryneedell@Mac-mini ampy % ampy -p /dev/tty.usbserial-21430 -b 115200 put requirements.txt 
requirements.txt : [####################] 100%
jerryneedell@Mac-mini ampy % ampy -p /dev/tty.usbserial-21430 -b 115200 ls                  
/.Trashes
/.fseventsd
/.metadata_never_index
/boot_out.txt
/code.py
/lib
/requirements.txt
/settings.toml
/test.test

whew!

[crcibernetica]

Thank you so much for your help!
I installed 8.1.0-beta.0 and noticed that there definitely is a difference in behavior from 8.0.3. Now the double CTRL-C starts the user REPL (before CTRL-C wouldn't do anything). Also, CTRL-D now returns the message "soft reboot", before it just returned "raw repl".

Code: Select all

C:\Users\bentl\Documents\Thonny>rshell -p COM3
Using buffer-size of 32
Connecting to COM3 (buffer-size 32)...
Trying to connect to REPL  connected
Retrieving sysname ... ESP32
Testing if ubinascii.unhexlify exists ... Y
Retrieving root directories ... /.fseventsd/ /.metadata_never_index/ /.Trashes/ /settings.toml/ /code.py/ /lib/ /boot_out.txt/
Setting time ... Mar 13, 2023 07:09:02
Evaluating board_name ... pyboard
Retrieving time epoch ... Jan 01, 2000
Welcome to rshell. Use the exit command to exit rshell.
C:\Users\bentl\Documents\Thonny> ls /pyboard
lib/          code.py       boot_out.txt  settings.toml

What version of python are you using? I am using 3.10.9. In order to get rshell to even start I had to apply a hack to remedy the error "module collections has no attribute callable".

In py3k_compat.py, change the 8th line from which should be return isinstance(x, collections.Callable) to return isinstance(x, collections.abc.Callable)

Hacks like this are not very convenient so I'm guessing that maybe there is a python version incompatibility.
Thanks!

[jerryn]

I am also using 3.10.9

Code: Select all

jerryneedell@Mac-mini ampy % python
Python 3.10.9 (main, Dec 15 2022, 17:11:09) [Clang 14.0.0 (clang-1400.0.29.202)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

[jerryn]

In py3k_compat.py, change the 8th line from which should be return isinstance(x, collections.Callable) to return isinstance(x, collections.abc.Callable)

Where is this file? I am not familiar with it.

I am using a Mac (MacOS Ventura 13.2.1) and python is installed via homebrew so there may be some differences from your environment.

[jerryn]

And FYI - this explains why it did not work with 8.0.3
https://github.com/adafruit/circuitpython/pull/7652

It will be in 8.0.4 when that is released.

[crcibernetica]

I am using the Thonny 4.0.2 installed python. I work mainly with educators and the Thonny install gives a full python install with a graphical package manager and CircuitPython.

I will try a separate manual python install and test.

py3k_compat.py is in the pyreadline module in my Python310 install.

Good news regarding 8.0.4!

You have been a great help!