How to pinpoint cause of servo jitter in PCA9685

[orochimaru]

Hello,

I have a project with 5 PCA9685 chained together, and almost all servo slots are filled (SG92/180deg). The PCA9685 chains is connected to an RPI via i2c.

I only need to move one servo at a time.

In general, it works. However, at random times, some servos get locked in a "jitter" state.
It seems as if they are trying to lock in a position, but jitter around it by very small angles.
At this state there is a constant draw of ~50-100mA per jittering servo.
An annoying cross-talk also appears, where moving a servo anywhere in the chain will make another servo jump to a random, obviously unintended position.

The power supply is perfectly adequate. Also using the screw terminals on the PCA9685 to provide external power to the servos. Common ground everywhere.

At the moment, the only solution that works is to power everything off via an external relay, wait for 15sec, and then continue. However, this is very time consuming.

Things I have tried:
- Resetting the PCA9685s via the corresponding adafruit reset()/deinit() library functions (Python). They seem to do absolutely nothing in general (like a blank function call), and other forum users have reported that as well.
- Using the OE pin. This stops the jitter, but it restarts immediately after restoring the OE to pass a new command.
- Using shorter power-off time delays. Anything below ~15 sec does not work. It seems that the PCA9685s have a memory register that restores their previous, jittering state.

I would appreciate any ideas on how to debug this further. From other things I have seen here in there in the forums I get that:

- The PCA9685 reset should work. However, users have seem to accomplish that only via external, command line tools for i2c handling, or some C libraries. Is there any way to perform this via the Python libraries?
Any other way to perform this via software?

- Users have also theorized about i2c noise, and that a capacitor could help as a LP filter. I have not seen an actual resolution using this approach though. Has anyone actually used this successfully? If so, what capacitor did you use and where did you connect it?

Thanks in advance!

[adafruit_support_bill]

Please post some more detail about the total number of servos, how you are powering them, and some photos showing the overall layout of the system.

[orochimaru]

Hello.

Here is a schematic.
(I am not showing the relay that cuts off the power to the PCAs, but it is irrelevant to the problem.
The purpose is to not need it in the first place :) )

Since I only need to move one servo at a time, I would be OK with a MOVE_SERVO/RESET_CHAIN/MOVE_SERVO... loop.
(This is what I am doing presently, but by cutting the power to the PCAs instead of a software reset).

Note that the problem can appear at any servo. (I have also tried with different boards and servos).

Any help is much appreciated.

circuit1.png (151.93 KiB) Viewed 85 times

[adafruit_support_bill]

Can you post some actual photos of the system so that we can see the wiring? With that many servos there is a lot of potential for interference and crosstalk between them. Servos are very electrically noisy devices and most servo cables are not shielded.

[orochimaru]

I will. But I won't have access for a photo for some days.
However, what I can note is that I am using standard dupont connectors and plain, unshielded cables.
(The most common kind, usually used with breadboards).
The cable length between each PCA9685 is 20cm, but the cable from the 1st PCA to the RPI is 50cm.
These are the shortest possible wire lengths that I can use, given the construct dimensions.
However, their total resistance (checked) is just a few Ohms (negligible).
The external power cables are 1.5mm^2, quite good quality too.

In the meantime, any help answering these questions would be much appreciated:

- How can I perform a software reset to the PCA9685s? They certainly get reset when I cut the power to them for 15 sec. Can I accomplish this with software? Would resetting the whole i2c bus from the RPI-side help?
This would solve my problem completely.

- If this is not possible, I would try anything instead of rewiring :). Would adding a capacitor and a resistance between SCL and GND help? I can hook up an oscilloscope and experiment a bit, if you think this can help. Would ferrites make a difference? Or changing the i2c bus frequency at the RPI?

Thanks!

[adafruit_support_bill]

The cable length between each PCA9685 is 20cm, but the cable from the 1st PCA to the RPI is 50cm.

If my math is correct, that gives a total i2c bus length of 130cm. That is long enough to potentially cause some issues - partly due to the capacitance of the wiring and partly from susceptibility to noise. Slowing down the i2c clock can help there.

Also, I'm interested in the complete wiring scheme - including the servo wiring. A basic law of EMI is that everything is an antenna - both for transmitting and receiving interference. Servo wiring will be broadcasting a combination of PWM switching and brush noise which can potentially interfere with the control signals for nearby servos as well as i2c communication to the PCA9685 boards. Because of this, routing of the servo wiring relative to the i2c bus and other servos can be a major factor.

- How can I perform a software reset to the PCA9685s?

The chip does not have a software reset. You would have to re-initialize all of the PWM registers explicitly in your code. But the problem may not even be in the PCA9685. EMI from one servo can directly affect a nearby servo by corrupting a correctly generated control signal. Once again: routing of the servo wiring relative to the i2c bus and other servos can be a major factor.

- If this is not possible, I would try anything instead of rewiring :).

Slowing down the i2c clock might help. Once we have a better idea of what your overall wiring scheme looks like, we might be able to suggest some other options.

[orochimaru]

Just an update of the things I have tried and their outcomes, in case someone faces the same problem:

• Reducing the i2c speed of RPI below 100K --> No effect

• Using shielded and shorter cables --> No effect

• Using capacitors at the DC pins --> No effect

• Using ferrites at the i2c pins --> No effect

• Using i2c extenders (LTC4311) --> Made things worse!!

What actually worked nicely in my case:
Messing with the PCA9685 registers before and after the servo movement :)

Code: Select all

output_hwnd=get_i2c_handle(pca9685_address) # in DEC, i.e., 64
pigs_write_2Register(output_hwnd, "0x01", "0x04")  #This enables the i2c channel for this PCA.
.... move your motor here.....
pigs_write_2Register(output_hwnd, "0x01", "0x00")      # This disables the i2c channel for this PCA, and returns its current draw to zero.
close_i2c_handle(output_hwnd)

Functions used. (Just wrapping the pigs CLI tool -> https://abyz.me.uk/rpi/pigpio/pigs.html):

Code: Select all

def get_i2c_handle(pca_int_address):
    command = ["pigs", "i2co 1 "+str(hex(pca_int_address))+" 0"]
    print(command[1])
    result = subprocess.run(command, capture_output=True, text=True, check=True)
    output_hwnd = result.stdout.strip()
    return output_hwnd

def close_i2c_handle(output_hwnd):
    os.system("pigs i2cc "+str(output_hwnd))

def pigs_write_2Register(i2c_dev_hwnd, reg, hexByte_str):
    command2 = ["pigs", "i2cwk "+i2c_dev_hwnd+" "+str(reg)+" "+hexByte_str]
    result2 = subprocess.run(command2, capture_output=True, text=True, check=True)
    output = result2.stdout.strip()

def pigs_write_stream_2Register(i2c_dev_hwnd, reg, hexByte_stream_str):
    command2 = ["pigs", "i2cwi "+i2c_dev_hwnd+" "+str(reg)+" "+hexByte_stream_str]
    result2 = subprocess.run(command2, capture_output=True, text=True, check=True)
    output = result2.stdout.strip()


def pigs_read_stream_From_Register(i2c_dev_hwnd, reg, n_bytes):
    command2 = ["pigs", "i2cri "+i2c_dev_hwnd+" "+str(reg)+" "+str(n_bytes)]
    result2 = subprocess.run(command2, capture_output=True, text=True, check=True)
    output = result2.stdout.strip()
    return output

[adafruit_support_bill]

Good to hear you found a solution. Reducing the number of active signals in the system reduces the potential for interference between them.

[orochimaru]

Hello again.

A final update, for anyone interested in the topic: buy a good regulated 5V power supply (e.g., Meanwell, ~20$).

This and the code above completely eliminated the problem for me.