# Need a method for synchronization of galvanically separated microcontrollers

I would like to get some ideas for my problem. I have a HW which has a main processing unit (Beaglebone, host controller) which is connected to four galvanically seperated STM32 microcontrollers. All of the STM32 microcontrollers are doing analog measurement with the same sampling frequency (100ksps). The STM32 mcus are doing the acquisition and the Beaglebone collects the data over SPI and evaluates it. The acqusition is started on all of the microcontrollers almost the same time. I would like to run the measurements over multiple hours. All STM32 has a 16MHz external crystal for clock source, with PLL the core is running at 100MHz. But because the crystals have tolerance the sample points will be shifted between the stm32 microcontrollers. After some minutes the differences are significant. It would be nice if the delay between the microcontrollers would be between +-10us - 100us.

The acquisition is continuous. I do not want to miss any samples.

Question 1: is there a method to do sampling correction/syncronization between the microcontrollers? Can you suggest a literature in this topic?

Question 2: if hardware redesign is possible, currently I am thinking in software solution, how can I put syncronization between the seperated microcontrollers?

Remark: what if the main processing unit measures somehow the delay between the microcontrolers. Then it sends a number to each of the microcontroller about how much is the difference referenced to an ideal frequency (sampling frequency or the operation frequency of the host). And then each microcontroller would change the sampling frequency a bit (100.1ksps or 99.9ksps). Something like this is in my mind. Can this work? Did anyone do something before?

Every comment are welcomed!

• How many STM32 slaves are there? How far apart are they? – jonathanjo Jan 17 at 11:28
• Use a sync pulse from the master. – Andy aka Jan 17 at 11:34
• @Andy, with hw modification the sync pulse could help – D_Dog Jan 17 at 12:01
• @jonathanjo all of them are placed on one PCBA (150x200mm). The maximal length of the SPI bus is 100mm approx. But it is galvanically isolated. – D_Dog Jan 17 at 12:01
• Do your STM32 micros have a clock calibration feature which allows you to tweak the master oscillator frequency up or down by a few ppm? You could calculate a correction factor in your Beaglebone for each STM32 every so often and apply that to keep them running at (almost) the exact same frequency. A hardware sync/trigger for them (choose one as the master) would probably be easier though... – brhans Jan 17 at 16:21

Both of these work on the idea that there is at least some delay in the sample loop of the slaves, which could be adjusted to make the slowest slave catch up with the fastest. That is, that there's a software timer which triggers an individual capture of the ADC.

Pure software You could read about the Network Time Protocol, and borrow/adapt as much of it as you thought necessary. At base, a master sends out time signals and the slaves calibrate themselves by a (software) phase-locked loop. With dedicated microprocessors and SPI communications, those mechanisms should get your your accuracy.

Explanation of NTP mechanisms Documentation, Presentation. NTP has quite a number of modes. The ones which might have mechanisms you can borrow are making the control host a fake reference clock (deemed to be correct) and having the slaves work like broadcast clients (even if they are receiving the messages one at a time). The main thing to read about is the clock discipline algorithm, which explains how the client gets its clock in sync with the master.

I'd consider using one of the STM32s as a clock master, and the host controller gets a nominal time from that, rather than using the host controller's clock. This is because we're not trying to sync the slaves to the real time, we're trying to remove time skew from the multiple samples. (As far as I've understood your goal.)

If you find you want to do it according to standards, NTP is defined as an internet RFC 5905, though you might want to also read the earlier versions which are a bit simpler. You might also want to look at PTP (precision time protocol), which is an IEEE standard IEEE Std 1588-2002. It's much more accurate, also much more complex, and much more expensive!

At the higher end of your resolution (0.1 ms), it should be entirely possible to simply get the host to tell each slave periodically "the time is X ns", and leave them to sample on X % 10000 = 0. All the jitter will be in the host controller. Assuming Linux kernel, you'll need to read clock_gettime(3) and pay attention to CLOCK_MONOTONIC or perhaps CLOCK_MONOTONIC_RAW settings.) The only thing to fret about is how to deal with a slave smooths the changing of its clock so it doesn't miss any sampling: I'd guess you can just sample "right now" if you have to jump. It depends on how much spare time the slaves have between samples.

Hardware If you can afford a wire and an input on the slaves (and the additional isolation), run an extra wire to tell all the slaves to sample. If that generates interrupts on the slaves, they will be about as synced as it's possible to get in software, and could be much better than your target. The optimal syncrhonisation (given in Jeroen3's answer), is to trigger the ADC capture directly from an IO pin.

• I would skip NTP and go PPTP, but yes, A software only solution can get you to such tolerances. – Lior Bilia Jan 17 at 11:45
• @LiorBilia thanks, added to answer. My suggestion was really to use NTP as a model to understand the clock discipline algorithm. – jonathanjo Jan 17 at 11:58
• NTP (the protocol, not the algorithm) is definitely unnecessarily complexity for a microcontroller. – user253751 Jan 17 at 13:25

Question 1: is there a method to do sampling correction/syncronization between the microcontrollers? Can you suggest a literature in this topic?

All STM32 ADC are capable of using an external trigger to initiate sampling.
You can distribute a 100 KHz clock and use that to trigger the conversions synchronously.
This is the easiest and most reliable method, often found in many data acquisition systems.
How to do this is described in the reference manual of your chip.

You may need to choose between starting a scan or a single conversion per edge.
See AN3116 STM32™’s ADC modes and their applications

Question 2: if hardware redesign is possible, currently I am thinking in software solution, how can I put syncronization between the seperated microcontrollers?

SPI is also a synchronous protocol, so that problem you have already tackled. You can get 100 Mbps isolators for cheap. You just have to keep an eye on the jitter they have.

You could also try to make some synchronization happen by calibrating against the SPI clock or intervals. By doing some software PLL like functionality.
But I'm afraid you'll be going down a rabbithole that way.

• thanks for the comment. Under galvanic isolation I meant the SPI is connected over isolator chips. – D_Dog Jan 17 at 12:41
• @D_Dog Use more isolator chips to distribute sampleclock. – Jeroen3 Jan 17 at 12:48
• for next layout, I definitly think about that. – D_Dog Jan 17 at 13:58

is there a method to do sampling correction/syncronization between the microcontrollers? Can you suggest a literature in this topic?

So, you'll need to synchronize their sampling clocks. That requires

1. keeping the core frequencies of the MCUs identical and
2. marking a common start point in time.

The first one can be managed by distributing an oscillator to the MCUs, also in the same galvanically isolated manner. Your 16 MHz oscillators need to go! Instead, your central board gets one, and buffers it and distributes it to your boards.

The second is way harder: Maybe the ADCs can be triggered to an external pulse (I don't think so, but really, read their respective documentation). Then, well, just distribute the same pulse to all of them.

If that's not the case, you'd still need to distribute a common point in time when to start, and the only way I could see you doing that reliably would be through distributing a pulse to a pin on all these MCUs, and having an interrupt be executed on a positive edge of that pulse, which in turn starts the first ADC conversion.

Problem is that on Cortex-M, interrupt latencies aren't as strictly defined; but, honestly, 10 µs should be pretty doable – that's a lot of clock cycles.

If you need better coordination, external ADCs that take a separate conversion clock source would be better suited.

if hardware redesign is possible, currently I am thinking in software solution, how can I put syncronization between the seperated microcontrollers?

Hm, so, without addition of a distributed clock and trigger signal, it's going to be very hard. I don't see a viable method, as there's no clock recovery hardware in the STM32 that you could use to "discipline" the internal clock to run locked to eg. the clock of the communication bus with your main board.

And then each microcontroller would change the sampling frequency a bit (100.0001ksps or 99.99999ksps).

Can't do that with that level of precision, since the sampling times are essentially set by internal dividers of the core frequency.

• Thanks for your answer. Your suggested idea is related to hardware modification which is really fulfill the problem. But for the first try I would like to compensate it as jonathanjo suggested from software. And you are right the 100.0001ksps or 99.9999ksps is not producable but I can set 100.1ksps and 99.9ksps sampling clock. And I belive the clock skew can be compensated. Probably it needs a lot of trick, but it could work. I have a room of couple 10us accuracy. – D_Dog Jan 17 at 12:11

Why not do it over SPI?

The time/delay of an SPI transmission should be almost constant. There is only some jitter for the clock domain crossings and for how long it takes to enter the SPI interrupt subroutine in the slaves¹. At most this jitter will be a few clock cycles. If your microcontroller and SPI clock frequency is several MHz it will be much lower than the ±10μs accuracy requirement.

Make the master send a start time to the slaves after boot. Then send the current time periodically to the slaves and have them slow down or speed up their clocks to match it.

Something like this on the master:

int main() {
// boot
for(int i = 0; i < NUM_SLAVES; i++) { // delay/jitter between iterations not important
disable_irq(); // IRQs would introduce additional jitter
send_start_time_to_slave(i, get_system_time());
enable_irq();
}