ItsyBitsy M4 Power Cycle Death

Please tell us which board you are using.
For CircuitPython issues, ask in the Adafruit CircuitPython forum.

Moderators: adafruit_support_bill, adafruit

Please be positive and constructive with your questions and comments.
Locked
User avatar
AQDINC
 
Posts: 27
Joined: Mon Jul 02, 2018 1:24 pm

ItsyBitsy M4 Power Cycle Death

Post by AQDINC »

Hello,

I have been using ItsyBitsy M4 boards as the basis for several varieties of flow controller boards that I have made. The boards work great, but I have had 5 boards quit working at various points. It appears that they are dying on power-up. On my circuit board, I am supplying 5V (4.96V on my multimeter) to the BAT pin of the ItsyBitsy M4 by using a 5V linear regulator. This linear regulator is supplied by a 24V bench supply, and it has a 0.3uF input cap and a a 0.1uF output cap.

I had 3 of the boards quit in the field, so I was not able to witness the circumstances surrounding their failure. Today, however, I was trying to see if I could make a board quit, and I was running rapid power cycles from the bench supply. After one such cycle, the device lights did not come on. I was afraid that the failure might be a result of my circuit board power setup, so I tried the same test using only the USB connector for power (isolating the ItsyBitsy from my circuit). Once again, the board quit after several quick cycles. This board, however, will sometimes work (about 20% chance).

Even on the dead boards I see 3V on the 3V pin, so I don't think I am blowing the ItsyBitsy's regulator. On some of the dead boards, the MCU will get very hot, but this does not always happen. I believe that the MCU is dying as a result of the quick on-off-on power cycles, but I'm not sure how to protect against this. I have cycled power on most of these boards hundreds of times just testing programs, and I have never had any issue. Is there a certain time period after the board is turned off that I must wait before applying power? Any help would be appreciated. I have 3 working boards, plus the one that sometimes works, and I would like to keep these working and send them to customers this week.

User avatar
sdsds
 
Posts: 4
Joined: Mon Oct 22, 2018 2:53 pm

Re: ItsyBitsy M4 Power Cycle Death

Post by sdsds »

Hi! Big caveats: (1) I'm totally new here. (2) I have an M0, not an M4. (3) I don't really know what I'm talking about! ;-)

That said, the power regulator shown on the M0 schematic is an AP2112 -- see https://www.diodes.com/assets/Datasheets/AP2112.pdf. The part has a separate "enable" pin; the ItsyBitsy circuit appears to show a 100K resistor between the input and enable pins. Presumably the intent is to somehow avoid startup transients causing trouble downstream....

Just idle speculation: it's probably possible to determine which component on the board is that resistor. Removing it (e.g. on a "dead" board) and replacing it with a jumper might let you manually delay the generation of 3.3V for ... seconds rather than milliseconds after the 2112 sees power.... Of course that would likely void your warranty! ;-)

Best regards,
--sdSds

User avatar
AQDINC
 
Posts: 27
Joined: Mon Jul 02, 2018 1:24 pm

Re: ItsyBitsy M4 Power Cycle Death

Post by AQDINC »

Hello,

The M4 schematic shows the same power configuration as the M0. Unfortunately, I need a solution that isn't manual (I assume replacing the 100K resistor with a jumper means creating a switch that I would have to flip). I am looking at ways to delay or soft-start the 5V regulation on my boards, presumably having the same effect of a longer startup delay. I am still wondering why the controllers are failing. What would cause the MCU to fail during a power cycle, and how can this be prevented?

User avatar
sdsds
 
Posts: 4
Joined: Mon Oct 22, 2018 2:53 pm

Re: ItsyBitsy M4 Power Cycle Death

Post by sdsds »

More caveats: (1) I'm not really a circuit designer; (2) reading schematics and visualizing the time-dependent signal characteristics isn't my strong suit either. ;-)

You're correct though that a jumper would be like a manual switch. Here's the thing: it would only be for an experiment. If that showed that a delay of e.g. 1 second before raising the enable pin voltage was enough to solve the problem you (and maybe others) are seeing, then a more involved modification of the circuit could make the delay automatic.

In particular, there's a 3M internal resistor between the EN and GND pins on the 2112. So if the external resistor were replaced with an appropriately sized capacitor it might automatically provide a fixed amount of delay. Unless I'm mis-visualizing the time-dependent characteristics of the circuit, that is! ;-)

Best regards,
--sdSds

User avatar
AQDINC
 
Posts: 27
Joined: Mon Jul 02, 2018 1:24 pm

Re: ItsyBitsy M4 Power Cycle Death

Post by AQDINC »

I did try it, and it does delay the startup by a second, but that doesn't bring back dead boards. I'm also not completely clear on how adding a capacitor would slow down the startup. It seems to me that once the capacitor was charged, no current would flow across the 3M resistor, and thus the EN pin would be at ground potential and the supply would remain off. I am trying to use an inductor to slow-start my 5V output (the supply voltage for the ItsyBitsy's BAT pin).

My main question is still remains -- why is the ItsyBitsy failing? I have hooked up all of the power pins to an oscilliscope, and I don't see anything that exceeds the ratings of any of the devices present, but I am still seeing MCU units dying.

I don't mind trying to slow down the power-on sequence as a safety, but I would like to understand why the device is failing so that I can be confident when I send the device to customers. What is causing the deaths?

User avatar
adafruit_support_mike
 
Posts: 67454
Joined: Thu Feb 11, 2010 2:51 pm

Re: ItsyBitsy M4 Power Cycle Death

Post by adafruit_support_mike »

How much time are you giving the microcontroller to shut down before turning on power again?

The registers in the microcontroller have some capacitance, and will retain charge after being powered off. If you flip the power on and off quickly enough for long enough, it's statistically inevitable that some part of the chip will be in an undefined state when the power is connected again. Once that happens, 'undefined' means 'undefined'.. it's impossible to predict what the circuits inside the chip will do. There are millions of places and ways the chip could fail.

If short-term power outages are a problem, it might be a good idea to add a small LiPo to keep the microcontroller running long enough to shut down gracefully if necessary. It's fairly easy to detect loss of external power, and you can use a GPIO pin to drain a capacitor connected to EN and force a minimum delay before the board can restart.

User avatar
AQDINC
 
Posts: 27
Joined: Mon Jul 02, 2018 1:24 pm

Re: ItsyBitsy M4 Power Cycle Death

Post by AQDINC »

Thank you,

I was giving it maybe just under a second from when I turned the switch off until when I turned it back on, but my bench supply also has some output capacitors that keep the power on a little longer. How long would I need to wait?

I don't completely understand how the capacitor on EN would prevent death on startup. My understanding is that I would detect a loss of power, then pull EN low using a GPIO. The capacitor would also be connected to EN, so it would take a small amount of time before EN went low and the power turned off. However, when the power is turned back on, EN would be high immediately. So, is there still a vulnerable period as the EN goes low where a power-on might cause a failure? I think I would rather have the critical power cycle time for failure be small and have capacitors smoothing the input.

As I said, I am not sure exactly what circumstances caused the failures in the field. I have only experienced one failures (maybe two, counting the one that sometimes works), and I am hesitant to do any more destructive testing, but my failures involved some fairly quick switching. I don't foresee this happening too often during normal operation, but I do want a robust solution. For this version, I am working with inductors and capacitors to try and ensure a smooth start-up.

For my next version, I will likely try to incorporate the capacitor discharge idea, but I would want to make sure that power could not be re-applied until the chip's registers were in a safe configuration. Maybe this is a series of capacitor discharge / FET combos that prevent power on and power off from happening too close together.

User avatar
AQDINC
 
Posts: 27
Joined: Mon Jul 02, 2018 1:24 pm

Re: ItsyBitsy M4 Power Cycle Death

Post by AQDINC »

I just had another board die. It had been minutes since the last power down, so the registers should have been completely drained of any charge. I also had the device hooked up to an oscilloscope, so I know that the startup was smooth.
20181024_121857.jpg
20181024_121857.jpg (26.81 KiB) Viewed 343 times
The yellow line is my 24V bench supply (different scale than the other lines). The light blue/turquoise line is the output from my 5V regulator. The pink line is the filtered ~4.5V power that is being applied to the ItsyBitsy's BAT pin. This is a very smooth application of power well within the range of the AP2112K supply (3.5-6V). The dark blue line is the 3V pin on the ItsyBitsy.

I have spent the last 2 days trying to make sure that the power into the BAT pin is smooth and slow, and I am happy with the result. However, the fact that this device still failed makes me suspect something in the ItsyBitsy's power circuit. With residual charge out of the question, what else would cause the failures I am seeing? Has anyone else experienced a similar issue, or can anyone duplicate this problem?

User avatar
BryonMiller
 
Posts: 222
Joined: Fri Mar 04, 2016 10:34 am

Re: ItsyBitsy M4 Power Cycle Death

Post by BryonMiller »

A couple of thoughts -
What is hooked to your Itsy Bitsy? And how is anything that is hooked powered? Any chance that anything is rising faster than the processor?

Also you say that the failure is happening at power up, but couldn't it be happening at power down and you would have no way of knowing until the next power-up? If so, same questions as above at power-down. For instance, could you have an external device that has a power holdup that turns off slower than the MCU. If so, can it be damaging the MCU by backdriving MCU?

User avatar
AQDINC
 
Posts: 27
Joined: Mon Jul 02, 2018 1:24 pm

Re: ItsyBitsy M4 Power Cycle Death

Post by AQDINC »

Thanks.

I have a serial transceiver for RS-232 and an op-amp that provides an analog signal. Both of these devices certainly would get power first in my design, so I went ahead and used inductors to delay their power startup. For my next version of the board I will add logic so that the MCU controls who gets power. I think this explanation makes a lot of sense. I will post again if any more boards fail

User avatar
BryonMiller
 
Posts: 222
Joined: Fri Mar 04, 2016 10:34 am

Re: ItsyBitsy M4 Power Cycle Death

Post by BryonMiller »

AQDINC wrote: I went ahead and used inductors to delay their power startup
As long as you're convinced the problem is occurring at power-on, inductors may help but you'd have to be careful you don't turn a power-on problem into a power-going-off problem. And I'm sure you're being careful to watch for any power-up/power-down inductor-induced ringing or spiking.
I will add logic so that the MCU controls who gets power
Does the MCU know power is about to go away? If not, it won't be able to turn off external circuits before the MCU itself is going down. Seems like you could still have a power-down race.

User avatar
AQDINC
 
Posts: 27
Joined: Mon Jul 02, 2018 1:24 pm

Re: ItsyBitsy M4 Power Cycle Death

Post by AQDINC »

I am not positive that the problem is happening on startup, but I previously had inductors slow-starting the MCU and I still had failure. To me this makes it seem like a power-up problem because the MCU would have had a few ms extra of power at shutdown. I don't believe that this was the perfect solution, but it was quick and easy with the parts on hand. I was also able to quell the ringing with some capacitors.

As far as logic goes, I am once again more concerned with power up. If everyone looses power at the same time, maybe all I need are some bigger bulk caps on the MCU to give it an extra split second on shutdown, or maybe I will still use inductors and make it smooth both ways. Either way, this will be a consideration for version 2.

User avatar
adafruit_support_mike
 
Posts: 67454
Joined: Thu Feb 11, 2010 2:51 pm

Re: ItsyBitsy M4 Power Cycle Death

Post by adafruit_support_mike »

AQDINC wrote:I don't completely understand how the capacitor on EN would prevent death on startup. My understanding is that I would detect a loss of power, then pull EN low using a GPIO. The capacitor would also be connected to EN, so it would take a small amount of time before EN went low and the power turned off. However, when the power is turned back on, EN would be high immediately.
You have the sequence backwards.

The voltage across a capacitor can only change in proportion to the amount of current flowing into or out of the cap. An M4's GPIO pins can handle about 10mA, which would drain a 1uF capacitor at a rate of 10kV/s. At a worst-case estimate, it would take about 300us for EN to drop to a voltage low enough to shut off the regulator.

When power comes back on, the only path current can take to the capacitor goes through the 100k pull-up resistor. A 1uF cap and a 100k resistor have a time constant of 100ms, with the voltage across the 100k resistor dropping by about 2/3 in that time. It would take at least 100ms for the voltage on EN to rise high enough for the voltage regulator to turn on and boot the microcontroller.

User avatar
sdsds
 
Posts: 4
Joined: Mon Oct 22, 2018 2:53 pm

Re: ItsyBitsy M4 Power Cycle Death

Post by sdsds »

Hi Mike. Did you have a chance to look at the scope output he has provided? The "dark blue line is the 3V pin on the ItsyBitsy" which is the output of the regulator, isn't it? Can you detect any delay in the rise of that voltage compared with the rise of the pink line which is, "the filtered ~4.5V power that is being applied to the ItsyBitsy's BAT pin?"

I can't tell what the time scale the scope is showing, but are you sure you're seeing, "at least 100ms for the voltage on EN to rise high enough for the voltage regulator to turn on?"

User avatar
adafruit_support_mike
 
Posts: 67454
Joined: Thu Feb 11, 2010 2:51 pm

Re: ItsyBitsy M4 Power Cycle Death

Post by adafruit_support_mike »

The traces look smooth, but the time base looks fairly long.

Try setting the trigger threshold at about 5.1V on the channel connected to the ItsyBitsy's input, cranking the timebase down into the microsecond range, and setting the scope to single capture. That will catch fast transients, which would be the most likely suspects for damage to a board.

Locked
Please be positive and constructive with your questions and comments.

Return to “Itsy Bitsy Boards”