Black Lives Matter - Action and Equality.
0

CircuitPython on custom board periodically disconnects/recon
Moderators: adafruit_support_bill, adafruit

Please be positive and constructive with your questions and comments.

CircuitPython on custom board periodically disconnects/recon

by vputz on Tue Jul 30, 2019 3:30 pm

Just followed the excellent guides and got UF2 and CircuitPython running on my little serial interface board. I'm over the moon! Even talks over the UART to the old serial device that is its raison d'etre.

But... periodically, and I can't fathom it, it disconnects entirely and reconnects as if I've unplugged and replugged it. I don't see any other indications. Any idea what may have gone wrong with the build? This board has worked fine when it was flashed with the regular sam-ba bootloader and compiled code. I'll eventually try different boards, etc, but I was wondering if this was a known weirdness if something gets borked in the CircuitPython definitions. It happens when it's totally idle (I don't have to necessarily be working in the repl or even running code).

vputz
 
Posts: 73
Joined: Mon May 03, 2010 5:43 am

Re: CircuitPython on custom board periodically disconnects/r

by tannewt on Tue Jul 30, 2019 4:09 pm

How much time elapses before it disconnects? What is it connected to? What version of CircuitPython is it?

Have you tried any other CircuitPython boards doing the same thing? Seems like it may be the host computer doing the disconnect.

tannewt
 
Posts: 1644
Joined: Thu Oct 06, 2016 8:48 pm

Re: CircuitPython on custom board periodically disconnects/r

by vputz on Wed Jul 31, 2019 12:24 pm

The time is pretty variable actually; sometimes it can go for a minute or two, other times it'll disconnect and reconnect within 10-20 seconds.

I've tried connecting via a powered USB 3.1 hub as well as directly to the desktop PC. It's CircuitPython version "5.0.0-alpha.0-92-gf9d314b26-dirty" (modified for my board although by far most of the pins are marked as disconnected except for UART TX/RX, which is working in the CircuitPython REPL; I can read from the serial device).

I haven't tried any other CircuitPython boards yet; I also need to reboot this board into the UF2 bootloader and see if that disconnects at all, as well as trying this on another PC or my laptop. I guess my questions until I can get time for that are:

1) Is there any reason something in the customization process would make it unstable like that?
2) Is it possible that since the USB device ID/vendor ID were registered on this computer as a different device (ie this same board, but when it was under the Arduino framework and/or USB hid device) that there's some crosstalk there? Any well-known way to fully purge USB devices with given ID numbers from the Windows registry to start "fresh"?

vputz
 
Posts: 73
Joined: Mon May 03, 2010 5:43 am

Re: CircuitPython on custom board periodically disconnects/r

by tannewt on Wed Jul 31, 2019 1:26 pm

Try your modifications on the 4.1.x branch. We just merged in a newer version of TinyUSB into master that may be unstable.

1) I don't think a new board would make it unstable unless the physical usb connection was unstable. Testing with an alternative software implementation could rule this out.
2) We generally use a different PID for Arduino vs CircuitPython to prevent descriptor caching issues. I don't know of a way to purge the cache.

tannewt
 
Posts: 1644
Joined: Thu Oct 06, 2016 8:48 pm

Re: CircuitPython on custom board periodically disconnects/r

by vputz on Thu Aug 01, 2019 12:56 pm

OK, the plot thickens :)

First: I'm unable to compile with the 4.1.x branch due to tinyusb; not sure what's wrong with my setup, but:

Code: Select all | TOGGLE FULL SIZE
golem% make BOARD=orbotron_9001
Use make V=1, make V=2 or set BUILD_VERBOSE similarly in your environment to increase build verbosity.
GEN build-orbotron_9001/genhdr/moduledefs.h
QSTR updated
In file included from ../../lib/tinyusb/src/tusb_option.h:70:0,
                 from ../../lib/tinyusb/src/device/usbd_control.c:27:
../../supervisor/shared/usb/tusb_config.h:110:50: error: expected declaration specifiers or '...' before numeric constant
 #define CFG_TUSB_MEM_ALIGN          ATTR_ALIGNED(4)
                                                  ^
../../lib/tinyusb/src/device/usbd_control.c:54:22: note: in expansion of macro 'CFG_TUSB_MEM_ALIGN'
 CFG_TUSB_MEM_SECTION CFG_TUSB_MEM_ALIGN uint8_t _usbd_ctrl_buf[CFG_TUD_ENDOINT0_SIZE];
                      ^~~~~~~~~~~~~~~~~~
../../lib/tinyusb/src/device/usbd_control.c: In function 'start_control_data_xact':
../../lib/tinyusb/src/device/usbd_control.c:78:12: error: '_usbd_ctrl_buf' undeclared (first use in this function); did you mean 'usbd_edpt_busy'?
     memcpy(_usbd_ctrl_buf, _control_state.buffer, xact_len);
            ^~~~~~~~~~~~~~
            usbd_edpt_busy
../../lib/tinyusb/src/device/usbd_control.c:78:12: note: each undeclared identifier is reported only once for each function it appears in
../../lib/tinyusb/src/device/usbd_control.c: In function 'usbd_control_xfer_cb':
../../lib/tinyusb/src/device/usbd_control.c:121:35: error: '_usbd_ctrl_buf' undeclared (first use in this function); did you mean 'usbd_edpt_busy'?
     memcpy(_control_state.buffer, _usbd_ctrl_buf, xferred_bytes);
                                   ^~~~~~~~~~~~~~
                                   usbd_edpt_busy
../../lib/tinyusb/src/device/usbd_control.c: In function 'start_control_data_xact':
../../lib/tinyusb/src/device/usbd_control.c:82:1: error: control reaches end of non-void function [-Werror=return-type]
 }
 ^
cc1: all warnings being treated as errors
make: *** [../../py/mkrules.mk:55: build-orbotron_9001/lib/tinyusb/src/device/usbd_control.o] Error 1




However, some observations after flailing a bit with 5.0.x and the disconnects:

1. The disconnects only happen when the RS232 serial device is connected, but they do not seem to be related to its activity (ie moving the controller or pushing buttons, which generates more packets, does not reliably correlate with the disconnects).
2. The disconnects do NOT happen when the device is in UF2 bootloader mode, with or without the RS232 serial device connected (went for >30 min without one).
3. The error message on reconnecting with the REPL was that it is running in safe mode because the microcontroller power dipped.

However, I cannot see any dip in the 3.3v power line when connecting the RS232 device (it stays at a solid 3.3V whether the device is connected or not); I did manage to be staring at the multimeter when a disconnect happened and if anything the power went UP to 3.4v for a moment but that could have been during the reboot. I tried to capture it on my cheapo $30 digital o-scope but the disconnects are elusive beasts and I haven't caught one yet. But I'm not convinced that's the issue; I haven't seen evidence of any spikes/variability in power and I stared at the o-scope pinning 3.3V for a long time while wiggling the device as much as I could.

Still willing to try 4.1.x if you can give me any insight as to the above. Thanks so much for your patience on this--CircuitPython looks awesome and will simplify the software for this beastie a lot when I can get this going!

vputz
 
Posts: 73
Joined: Mon May 03, 2010 5:43 am

Re: CircuitPython on custom board periodically disconnects/r

by tannewt on Thu Aug 01, 2019 2:18 pm

Oh cool! I listened to a podcast you were on about the Orbotron! I'm happy to help get this working.

I doubt it's a TinyUSB issue if the disconnect puts you in safe mode due to power. However, to switch to 4.1.x you likely need to `git submodule sync --recursive` and then `git submodule update --init --recursive`. That will make sure the TinyUSB version is correct.

What chip is on your custom board? Is it a SAMD21 or 51? How many decoupling caps does it have? Have you tried the same thing with other software?

tannewt
 
Posts: 1644
Joined: Thu Oct 06, 2016 8:48 pm

Re: CircuitPython on custom board periodically disconnects/r

by vputz on Thu Aug 01, 2019 2:57 pm

Aw, awesome! Yeah, it's the good old Orbotron, which is honestly a glorified serial adapter by now but it's evolved over the years--started off as a lowly Windows XP driver :D

I'll try the 4.1 branch anyhow just for fun; thanks for the reminder on submodules. I can't count the times submodules have tripped me up (we use them at my day job too).

So... this is a SAMD21E18 and I'll be quite honest that I don't know a ton about what I'm doing here, EE-wise, particularly when it comes to the decoupling caps. I based the schematic off one of the early SAMD21 Adafruit schematics but I honestly don't remember which one. I can post the Kicad schematic if it would help, but there's a 10uf cap to ground between +5 off USB and the SPX3819 voltage regulator to provide nice 5v, and then the 3.3v line coming off the voltage regulator has a 0.1uf to ground for supply. This 3.3v line goes to VDDANA (pin 9) and VDDIN (pin 30) and VDDCORE (pin 29) has a 1uf cap to ground.

Having said that, the +5v has another job to do: it's coming in to a TC1240A charge pump to provide 10V for pins 4 and 7 of the serial port (the SpaceOrb used a common abuse of the RS232 standard to provide power on DTR and RTS if I remember right). There's a 10uf decoupler for that 10v line as well.

What gets me is that I've never had trouble with random disconnects before using the existing Arduino sketch for the USB/hid stuff, and have used it for some pretty stout gaming sessions, and no users have complained about random disconnects, so while there may be some deficiencies (seriously, I guessed on a lot of this), it's interesting that only CircuitPython is disconnecting... maybe I really should try a different copy of the board.

The board is due a minor redesign (I never needed a reset button before, but moving to the UF2 bootloader I think that will be necessary, and I avoided LEDs because they're a pain to hand-build but if I can find a manufacturer I may add those too). But the power does look pretty solid with my SuperCheap o-scope (I need to invest in a good benchtop scope at some point...).

Puzzling stuff-but thanks a ton. I haven't looked at this in a while and there's a lot of context-switching going on here :)

vputz
 
Posts: 73
Joined: Mon May 03, 2010 5:43 am

Re: CircuitPython on custom board periodically disconnects/r

by tannewt on Fri Aug 02, 2019 12:56 pm

Ah, it's good to know it works with Arduino! Sounds like you do have enough decoupling.

One difference between Arduino and CircuitPython may be that we set the brownout detector to 2.7v: https://github.com/adafruit/circuitpyth ... port.c#L96 You could try removing that temporarily.

It still may point to a power issue that may or may not cause problems though. I'm not much of an EE myself to know.

tannewt
 
Posts: 1644
Joined: Thu Oct 06, 2016 8:48 pm

Re: CircuitPython on custom board periodically disconnects/r

by vputz on Mon Aug 05, 2019 5:59 pm

Maybe! Really interesting--thanks for the reference to the brownout detection.

...although it could be that I've taken a lot of your time over a freak combination of board and computer :(

I took this board to a different desktop and left it plugged in for 30 min or so without problem. But then I took another board and plugged it into my desktop, flashed it with UF2 and then CircuitPython... and it's been sitting here for 45 minutes without a disconnect, with the Orb (serial device) plugged in. I'm not sure what combination of things may be causing the original board to be failing, but a second board is working (so far) just fine.

So... well, it's inconclusive. But I may try a custom build for the "bad board" to disable brownout to see if that's it; at least with the current working board I can proceed with software development (I'm going to at least add a reset button to the hardware now that UF2 is awesome, and maybe at least a boot LED if not serial tx/rx).

Thanks a ton for your help, though--finding relevant parts of CircuitPython is going to be a huge benefit as I flail here (and hopefully the development process is going to be pretty fast actually; CircuitPython looks a lot easier to deal with than Arduino was in some ways--not that it wasn't hugely important! But this is really going to be a boon for this project.

vputz
 
Posts: 73
Joined: Mon May 03, 2010 5:43 am

Re: CircuitPython on custom board periodically disconnects/r

by tannewt on Mon Aug 05, 2019 6:31 pm

Great! Glad you made progress!

The one other thing I'd suggest checking is the current draw of each board. (I use a bench top power supply.) You may find there is current leaking somehow on the bad one that is causing the voltage to drop more easily. (But who knows, I'm not much of an EE.)

tannewt
 
Posts: 1644
Joined: Thu Oct 06, 2016 8:48 pm

Re: CircuitPython on custom board periodically disconnects/r

by vputz on Fri Aug 09, 2019 5:28 pm

New day, new problem, but this should be reeeelatively easy :) This is all working beautifully; I'm decoding packets, making a virtual model of the controller; all that works great and much more beautifully than the C++-based version. The last big step: outputting to a HID device.

I can't use the existing four-axis sixteen-button gamepad device, because I've got a six-axis beastie here. But I obviously (?) can't just edit the "Gamepad" in the library just iterates through the existing premade devices and returns the one representing a gamepad, with a premade HID descriptor.

I'm comfortable with HID descriptors (had to do it that way last round) but it's not clear how to add a device with an arbitrary HID descriptor. Sounds like I need to modify CircuitPython, but I confess it's not immediately obvious how; I see the shared-module usb_hid section, and a generated header in my build directory which creates a few devices including the gamepad (sort of). But I don't see where the descriptors are hiding, nor how to add a new one (or in a pinch just edit the gamepad descriptor).

So:

1) Is it possible at all to create an arbitrary HID device at runtime? Doesn't look like it, but I thought I'd ask.
2) If I need to modify CircuitPython, how do I either (preferred) add a new HID device, or (also OK) modify the existing Gamepad report descriptor?

Thanks--I'm almost getting an idea of how to do it, but I will waste a LOT of time without some pointers!

vputz
 
Posts: 73
Joined: Mon May 03, 2010 5:43 am

Re: CircuitPython on custom board periodically disconnects/r

by danhalbert on Sun Aug 11, 2019 10:17 pm

We don’t yet have USB descriptors crested at runtime. There’s an issue for that.

The descriptors are generated here; https://github.com/adafruit/circuitpyth ... criptor.py

More details after Monday when I don’t have to reply via phone. Hope this will help you get started.

danhalbert
 
Posts: 2024
Joined: Tue Aug 08, 2017 12:37 pm

Re: CircuitPython on custom board periodically disconnects/r

by vputz on Mon Aug 12, 2019 1:13 pm

Aha! I had run across that and knew it was involved, but I'm not clear how it fits into the build process yet. I can't directly work on this for a week (business trip) but if you could just give me a high-level description of what to modify or how gen_usb_descriptor fits in to the build plan, I can probably run with it when I get back.

Thanks a ton to both of you. Having started this process years ago with bit-banging and the V-USB library, I can only be continually grateful at how easy CircuitPython has been and how much cleaner the code is (finally I can just make this a nice subject/observer pipeline with tidy little processing nodes and probes to diagnose--it's SO MUCH NICER and the users will be able to reprogram their devices much more ably)

vputz
 
Posts: 73
Joined: Mon May 03, 2010 5:43 am

Re: CircuitPython on custom board periodically disconnects/r

by tannewt on Tue Aug 13, 2019 1:21 pm

The build rule for it is here: https://github.com/adafruit/circuitpyth ... sor.mk#L92

The current GamePad report is from here: https://github.com/adafruit/circuitpyth ... rs.py#L168

You could add a specific report there for you and plug it into the main script conditionally by USB PID/VID or by piping in the board name (my preference).

tannewt
 
Posts: 1644
Joined: Thu Oct 06, 2016 8:48 pm

Re: CircuitPython on custom board periodically disconnects/r

by vputz on Wed Aug 14, 2019 1:32 pm

THERE IT IS! Haha, you two are champions... I was looking for that guy and my grep-fu was weak. Can't wait to get back and implement this; should be clean and easy.

vputz
 
Posts: 73
Joined: Mon May 03, 2010 5:43 am

Please be positive and constructive with your questions and comments.