5
\$\begingroup\$

We are puzzled by a surprise communication failure between our controller PCB and sensor PCB.

Circuit Diagram for Sensor Communication Failure: Circuit Diagram for Sensor Communication Failure

The communication fails unless we do one or more of the following. Then it works fine:

  • Disconnect the shield from ground (1), leaving shield floating
  • Put a resistor between the shield and ground (1)
  • Remove the capacitor (2) going into the 5V to 3V voltage regulator
  • Touch ground anywhere on the sensor (3)

We can't explain why these fixes work. Leaving the shield (1) floating (disconnected) means its not providing a path to ground for noise. The 1µF capacitor (2) should provide a path to ground for noise, and make the voltage more stable. Touching the sensor can should only change its capacitance.

We tried removing the ferrite bead (FB) in case there was some LC oscillation happening but that didn't affect the behavior. The capacitors to on the sensor board are all connected to a ground plane by vias.

Could disposing of noise by grounding the shield or having a capacitor actually make communication worse?

Why would touching the sensor ground fix the problem? Does adding capacitance change the path of noise?

Edit: here is an o-scope image of the SDA data signal on the sensor side. It's a little slow on the rise time. However I can't see any difference in the waveform when I do the 4 fixes above :/

SDA data oscilloscope reading on sensor side

close up sda o scope reading

Edit 2: here are part numbers and datasheets.

Controller:

Sensor:

Edit 3:

I had thought that the grounding issue was leading to communication failure. However, stepping through the code in the controller, I found that the sensor is still able to "acknowledge" the address/write byte sent by the controller. It just doesn't "acknowledge" the address/read byte, and then sends all 1s back instead of sensor data. I think this means that the communication part is working fine, but somehow the grounding issue is disturbing the sensor's internal processing to where it can't get a stable reading.

\$\endgroup\$
16
  • 1
    \$\begingroup\$ How long is the cable between Controller and Sensor Board and what kind of communication are you using? (It looks like I2C maybe?). If it is I2C (or any other single-ended type) you should probably not twist the signals together. It is much better to twist each signal with GND or not using twisted cable at all. \$\endgroup\$
    – Pan Vi
    Jul 2 '21 at 5:24
  • 2
    \$\begingroup\$ Can you look at the signals with oscilloscope, what is happening? What pull-ups are you using? Are they on both sides (Controller and Sensor Board)? You can try to lower the value. The problem is that fixes like "removing capacitor" offten fail when you move the project from bench to the noisy enviroment with all kinds of motors etc. (at least in my experience). I2C is not meant to be used to on long wires, maybe try looking into I2C extenders, something like LTC4331... \$\endgroup\$
    – Pan Vi
    Jul 2 '21 at 6:08
  • 1
    \$\begingroup\$ Doing (2) should make no difference. \$\endgroup\$
    – Andy aka
    Jul 2 '21 at 7:38
  • 1
    \$\begingroup\$ The green lines around your controller & sensor - are they meant to represent a metal enclosure? Also 15 feet, or even 5 feet is too long a run to expect a single ended interface to work reliably. \$\endgroup\$
    – SteveSh
    Jul 2 '21 at 19:43
  • 1
    \$\begingroup\$ Vehicle ground, right. Does the sensor case normally also contact something that is grounded (eg. the engine), or is it insulated? \$\endgroup\$ Jul 4 '21 at 21:48
2
\$\begingroup\$

A very well-presented question.

You are looking in the wrong place; noise is not your problem. This is a grounding issue. When your sensor circuit draws high frequency current, as when it transmits, the power line's impedance means that the circuit's voltage on the sensor end is appreciably different than that on the microcontroller end. This means that your grounds are at different voltage levels. You can lessen this problem by removing or reducing loads at the transmission line ends or improving the ground connection, as you have demonstrated.

Since the ground on the microcontroller end is not at the same instantaneous voltage as the ground on your sensor end, using a voltage signal for communication with power ground as your signal reference is not ideal. This is why many sensor designs with cables use current rather than voltage, or add pairs with opto-isolators to the communication signal circuit to avoid this problem. Either approach would make your design more bulletproof.

Good luck!

\$\endgroup\$
5
  • \$\begingroup\$ Thank you so much for your point about ground disagreement. That would explain why the capacitor was detrimental, it was "Decoupling" the grounds just like it is supposed to and making them disagree. I bet touching the sensor allowed some of its high frequency current to be absorbed, improving the agreement between grounds. And disconnecting the shielding would improve disagreement between grounds too because the shield isn't dumping noise on the controller end (and it's also not effective as a shield) \$\endgroup\$
    – Luminaire
    Jul 3 '21 at 21:44
  • \$\begingroup\$ Your point led us to an interesting fix. To suck off some of the high frequency current, we soldered the shielding to the ground wire within the harness, near the sensor (shown as an X on the diagram). This solution has proven very robust in our field test, even with our 15foot harness wrapped around the noisy components in the engine. It's still feeling a bit like voodoo, but we have something to run with for the several thousand units in production while we work on a long term fix \$\endgroup\$
    – Luminaire
    Jul 3 '21 at 21:47
  • \$\begingroup\$ One question about the grounds disagreeing. If that is really the case, should I be able to pick it up on the oscilloscope? I tried probing the grounds on both ends, and set the oscope to trigger if they were different, but the biggest difference I could detect was 150mv, even with a 20ns resolution. And that 150mv is basically the noise floor. I get the same difference just hooking the two probes together. \$\endgroup\$
    – Luminaire
    Jul 3 '21 at 21:50
  • 1
    \$\begingroup\$ Happy to hear that you have a fix! What I think you will find is that a scope will show it depending on where is it grounded. Often a scope is grounded through your power line ground, so attaching the ground wire from the scope probe will change or improve your grounding scheme. \$\endgroup\$ Jul 6 '21 at 2:00
  • 1
    \$\begingroup\$ You might be able to see the effects of scope grounding by touching the scope ground to the circuit ground near to and far away from the signal being measured. I think you might see a difference in the "spikes," which actually are likely caused by the ground difference. \$\endgroup\$ Jul 6 '21 at 13:10
2
\$\begingroup\$

That negative spike looks quite strong, maybe 1.5V or something. That's outside of the normal allowed voltage for microcontroller pins.

enter image description here

Hard to tell where it comes from, but IMO you could try adding a small resistor like 100R in series with the signal to slow down the falling edge a bit. If the pullup is 10k, this low value won't prevent it from pulling the voltage down when it needs to.

\$\endgroup\$
1
  • \$\begingroup\$ I had seen that dip and wondered if it was significant. Your point about the microcontroller allowed pin voltage is absolutely right. I will try to figure out where that negative spike is coming from. There is an ESD protection steering/tvs diodes on the i2c lines in the sensor, I wonder if that could be the culprit. Adding series resistors seems like a good solution, I will try it and let you know \$\endgroup\$
    – Luminaire
    Jul 4 '21 at 21:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.