Cycle counting with modern CPUs (e.g. ARM)

Question

In many applications, a CPU whose instruction execution has a known timing relation with expected input stimuli can handle tasks that would require a much faster CPU if the relationship were unknown. For example, in a project I did using a PSOC to generate video, I used code to output one byte of video data every 16 CPU clocks. Since testing whether the SPI device is ready and branching if not would IIRC take 13 clocks, and a load and store to output data would take 11, there was no way to test the device for readiness between bytes; instead, I simply arranged to have the processor execute precisely 16 cycles' worth of code for each byte after the first (I believe I used a real indexed load, a dummy indexed load, and a store). The first SPI write of each line happened before the start of video, and for every subsequent write there was a 16-cycle window where the write could occur without buffer overrun or underrun. The branching loop generated a 13 cycle window of uncertainty, but the predictable 16-cycle execution meant that the uncertainty for all subsequent bytes would fit that same 13 cycle window (which in turn fit within the 16-cycle window of when the write could acceptably occur).

For older CPU's, the instruction timing information was clear, available, and unambiguous. For newer ARMs, timing information seems much more vague. I understand that when code is executing from flash, caching behavior can make things much harder to predict, so I would expect that any cycle-counted code should be executed from RAM. Even when executing code from RAM, though, the specs seem a little vague. Is the use of cycle-counted code still a good idea? If so, what are the best techniques to make it work reliably? To what extent can one safely assume that a chip vendor isn't going to silently slip in a "new improved" chip which shaves a cycle off the execution of certain instructions in certain cases?

Assuming the following loop starts on a word boundary, how would one determine based on specifications precisely how long it would take (assume Cortex-M3 with zero-wait-state memory; nothing else about the system should matter for this example).

myloop:
  mov r0,r0  ; Short simple instructions to allow more instructions to be prefetched
  mov r0,r0  ; Short simple instructions to allow more instructions to be prefetched
  mov r0,r0  ; Short simple instructions to allow more instructions to be prefetched
  mov r0,r0  ; Short simple instructions to allow more instructions to be prefetched
  mov r0,r0  ; Short simple instructions to allow more instructions to be prefetched
  mov r0,r0  ; Short simple instructions to allow more instructions to be prefetched
  adds r2,r1,#0x12000000 ; 2-word instruction
  ; Repeat the following, possibly with different operands
  ; Will keep adding values until a carry occurs
  itcc
  addscc r2,r2,#0x12000000 ; 2-word instruction, plus extra "word" for itcc
  itcc
  addscc r2,r2,#0x12000000 ; 2-word instruction, plus extra "word" for itcc
  itcc
  addscc r2,r2,#0x12000000 ; 2-word instruction, plus extra "word" for itcc
  itcc
  addscc r2,r2,#0x12000000 ; 2-word instruction, plus extra "word" for itcc
;...etc, with more conditional two-word instructions
  sub r8,r8,#1
  bpl myloop

During execution of the first six instructions, the core would have time to fetch six words, of which three would be executed, so there could be up to three pre-fetched. The next instructions are all three words each, so it wouldn't be possible for the core to fetch instructions as fast as they are being executed. I would expect that some of the "it" instructions would take a cycle, but I don't know how to predict which ones.

It would be nice if ARM could specify certain conditions under which the "it" instruction timing would be deterministic (e.g. if there are no wait states or code-bus contention, and the preceding two instructions are 16-bit register instructions, etc.) but I haven't seen any such spec.

Sample application

Suppose one is trying to design a daughterboard for an Atari 2600 to generate component video output at 480P. The 2600 has a 3.579MHz pixel clock, and a 1.19MHz CPU clock (dot clock/3). For 480P component video, each line must be output twice, implying a 7.158MHz dot clock output. Because the Atari's video chip (TIA) outputs one of 128 colors using as 3-bit luma signal plus a phase signal with roughly 18ns resolution, it would be difficult to accurately determine the color just by looking at the outputs. A better approach would be to intercept writes to the color registers, observe the values written, and feed each register in the TIA luminance values corresponding to the register number.

All this could be done with an FPGA, but some pretty fast ARM devices can be had far cheaper than an FPGA with enough RAM to handle the necessary buffering (yes, I know that for the volumes such a thing might be produced the cost isn't a real factor). Requiring the ARM to watch the incoming clock signal, however, would significantly increase the required CPU speed. Predictable cycle counts could make things cleaner.

A relatively simple design approach would be to have a CPLD watch the CPU and TIA and generate a 13-bit RGB+sync signal, and then have ARM DMA grab 16-bit values from one port and write them to another with proper timing. It would be an interesting design challenge, though, to see if a cheap ARM could do everything. DMA could be a useful aspect of an all-in-one approach if its effects on CPU cycle counts could be predicted (especially if the DMA cycles could happen in cycles when the memory bus was otherwise idle), but at some point in the process the ARM would have to perform its table lookup and bus-watching functions. Note that unlike many video architectures where color registers are written during blanking intervals, the Atari 2600 frequently writes to color registers during a the displayed portion of a frame, and many games rely upon pixel-accurate timing.

Perhaps the best approach would be to use a couple discrete-logic chips to identify color writes and force the lower-bits of color registers to the proper values, and then use two DMA channels to sample the incoming CPU bus and TIA output data, and a third DMA channel to generate the output data. The CPU would then be free to process all of the data from both sources for each scan line, perform the necessary translation, and buffer it for output. The only aspect of the adapter's duties which would have to happen in "real time" would be the override of data written to COLUxx, and that could be taken care of using two common logic chips.

Answer 1

I vote for DMA. It's really flexible in Cortex-M3 and up - and you can do all kind of crazy things like automatically getting data from one place and outputing into another with specified rate or at some events without spending ANY CPU cycles. DMA is much more reliable.

But it might be quite hard to understand in details.

Another option is soft-cores on FPGA with hardware implementation of these tight things.

Answer 2

Timing information is available, but, as you pointed out, can occasionally be vague. There's a lot of timing information in Section 18.2 and Table 18.1 of the Technical Reference Manual for the Cortex-M3, for example, (pdf here), and an excerpt here:

which give a list of conditions for maximum timing. The timing for many instructions is dependent on external factors, some of which do leave ambiguities. I've highlighted each of the ambiguities I found in the following excerpt from that section:

[1] Branches take one cycle for instruction and then pipeline reload for target instruction. Non-taken branches are 1 cycle total. Taken branches with an immediate are normally 1 cycle of pipeline reload (2 cycles total). Taken branches with register operand are normally 2 cycles of pipeline reload (3 cycles total). Pipeline reload is longer [How much longer?] when branching to unaligned 32-bit instructions in addition to accesses to slower memory. A branch hint is emitted to the code bus that permits a slower system [How much slower?] to pre-load. This can [Is this optional?] reduce [By how much?] the branch target penalty for slower memory, but never less than shown here.

[2] Generally, load-store instructions take two cycles for the first access and one cycle for each additional access. Stores with immediate offsets take one cycle.

[3] UMULL/SMULL/UMLAL/SMLAL use early termination depending on the size of source values [What sizes?]. These are interruptible (abandoned/restarted), with worst case latency of one cycle. MLAL versions take four to seven cycles and MULL versions take three to five cycles. For MLAL, the signed version is one cycle longer than the unsigned.

[4] IT instructions can be folded. [When? See comments.]

[5] DIV timings depend on dividend and divisor. [Same problem as MUL] DIV is interruptible (abandoned/restarted), with worst case latency of one cycle. When dividend and divisor are similar [How similar?] in size, divide terminates quickly. Minimum time is for cases of divisor larger than dividend and divisor of zero. A divisor of zero returns zero (not a fault), although a debug trap is available to catch this case. [What are the ranges, which were given for MUL?]

[6] Sleep is one cycle for the instruction plus as many sleep cycles as appropriate. WFE only uses one cycle when event has passed. WFI is normally more than one cycle unless an interrupt happens to pend exactly when entering WFI.

[7] ISB takes one cycle (acts as branch). DMB and DSB take one cycle unless data is pending in the write buffer or LSU. If an interrupt comes in during a barrier, it is abandoned/restarted.

For all use cases, it will be more complex than the "This instruction is one cycle, this instruction is two cycles, this is one cycle..." counting possible in simpler, slower, older processors. For some use cases, you will not encounter any ambiguities. If you do encounter ambiguities, I suggest:

1. Contact your vendor and ask them what the instruction timing is for your use case.
2. Test to specify the ambiguous behavior
3. Re-test for any processor revisions and especially when going through vendor changes.

These requirements probably make the answer to your question, "No, it's not a good idea, unless the difficulties encountered are worth the cost" - but you already knew that.

Answer 3

One way to get round this problem is to use devices with deterministic or predictable timings, such as the Parallax Propeller and XMOS chips:

http://www.parallaxsemiconductor.com/multicoreconcept

http://www.xmos.com/

Cycle-counting works very well with the Propeller (assembly language has to be used), whilst the XMOS devices have a very powerful software utility, the XMOS Timing Analyzer, which works with applications written in the XC programming language:

https://www.xmos.com/download/public/XMOS-Timing-Analyzer-Whitepaper%281%29.pdf

Answer 4

Cycle counting gets more problematic as you get away from low level microcontrollers and into more general purpose computing processors. The first usually do have well specified instruction timing, partly for the reasons you site. It's also because their architecture is fairly simple, so instruction times are fixed and knowable.

A good example of this are most Microchip PICs. The 10, 12, 16, and 18 series have very well documented and predictable instruction timing. This can be a useful feature in the kind of small control applications these chips are intended for.

As you get away from ultra low cost, and the designer can therefore spend some more chip area to get higher speed from a more exotic architecture, you also get away from predictability. Take a look at modern x86 variants as extreme examples of this. There are several levels of caches, vitualization of memory, lookahead fetch, pipelining, and more, that makes counting instruction cycles nearly impossible. In this application it doesn't matter though since the customer is interested in high speed, not instruction timing predictability.

You can even see this effect at work in higher Microchip models. The 24 bit core (24, 30, and 33 series) has largely predictable instruction timing, except for a few exceptions when there are register bus contentions. For example, in some cases the machine inserts a stall when the next instruction uses a register with some indirect addressing modes whose value was changed in the previous instruction. This kind of stall is unusual on a dsPIC, and most of the time you can ignore it, but it shows how these things creep in due to the designers trying to give you a faster and more capable processor.

So the basic answer is, that's part of the tradeoff when you chose a processor. For small control applications you can chose something small, cheap, low power, and with predictable instruction timing. As you demand more processing power, the architecture changes so that you have to give up predictable instruction timing. Fortunately, that's less of a issue as you get to more compute-intensive and general purpose applications, so I think the tradeoffs work out reasonably well.

Answer 5

Yes, you can still do it, even on an ARM. The biggest problem with that on an ARM is that ARM sells cores not chips, and the core timing is known, but what the chip vendor wraps around it varies from vendor to vendor and sometimes from chip family to another within the vendor. So a particular chip from a particular vendor can be quite deterministic (if you dont use caches for example), but becomes harder to port. When dealing with 5 clocks here and 11 clocks there using timers is problematic as the number of instructions it takes to sample the timer and figure out if your timeout has expired. From the sounds of your past programming experience, I am willing to bet you probably debug with an oscilloscope as I do, so you can try a tight loop on the chip at the clock rate, look at the spi or i2c or whatever waveform, add or remove nops, change the number of times through the loop and basically tune. As with any platform, not using interrupts greatly aids the deterministic nature of instruction execution.

No, it is not as simple as a PIC, but still quite doable, esp if the delay/timing approaches the clock rate of the processor. A number of the ARM based vendors allow you to multiply the clock rate and get say 60MHz off of an 8 mhz reference, so if you need some 2mhz interface instead of doing something every 4 instructions, you can boost the clock (if you have the power budget) and then use a timer and give yourself lots of clocks to do other things as well.