Electrical Engineering Stack Exchange is a question and answer site for electronics and electrical engineering professionals, students, and enthusiasts. It only takes a minute to sign up.
Sign up to join this community
How does CPU know that a write to RAM was successful, like how does a faster CPU know that data was successfully written to a slower RAM?
You're free to mention any system architecture because I'm not familiar with a specific mechanism by which the CPU does so.
The communication channels between a modern CPU and its RAM are not just unidirectional.
If you trace the path between a CPU core and the system's main memory, you will first find so-called load and store queues, followed by a few layers of caches, then the memory controllers, and then finally the DRAM itself.
When the CPU issues a write request to an address in memory, this request gets placed in the core's store queue. This queue is used to track outstanding writes to memory - whether they have been started yet, are currently still in-flight, and which data they write to which address. The write request remains in this queue until the other parts of the CPU (for example the caches or one of the memory controllers) signals back to the CPU core that the write request has completed. At that point, the CPU core can erase the write request from the store queue again and use the freed-up entry to track another write.
When the store queue fills up, the CPU core has to stall until at least one of the queued-up write requests completes and frees up another entry. As a result, if the memory subsystem can't complete the CPU core's writes fast enough, the core will get throttled due to buffer backpressure.
There is of course also a load queue that functions in a similar way.
The communication between the CPU core and the memory subsystem (caches, memory controllers) usually happens via some kind of standardized protocol, which in the simplest case accepts requests from the CPU core and returns responses some time later. That way, the core can send a request out to the memory controller (or cache), and after a while, it gets a response back that tells it that the request has completed. In the case of memory read accesses, the data that was read from memory is also returned in these responses.
An example of such a protocol is AXI4. Its documentation is freely available.
In a real (multi-core) system, this protocol will of course not just handle read and write requests, but also other messages used for cache coherency and other things (i.e. cache line invalidation messages, read for ownership requests, possibly even atomic memory transactions, etc).
Here is how it works for modern systems using DDR3/DDR4 memory:
The DDR memory sticks contain a SPD chip which reports the RAM speed information over a configuration bus. This is read out at boot by BIOS/UEFI.
BIOS/UEFI configures the memory controller interface that is integrated on the CPU chip. This sets the memory timing, including clock speed and number of clock cycles to wait for an access to complete. Usually these come from the SPD chip, but can be overridden for overclocking or performance tuning.
CPU memory controller obeys the set timings. If the timing is set too fast, memory access errors will occur and will cause data corruption. Every access to the memory is expected to take a predictable number of clock cycles.
In a simple chip, such as a microcontroller, the RAM, like any other clock-cycle dependent logic, is designed to be fast enough to accept a write or serve up a read at full CPU bus speed, and/or the bus speed is limited to what the RAM can handle. Ensuring this RAM - CPU timing is met falls to chip designer, in a process called AC Sign-Off, also known as Timing Closure. The same applies for a board-level design that uses standalone RAM and CPU chips: timing must be met by system design, through analysis of worst- and best-case delays of both the CPU and the RAM.
If for some reason the RAM is slower than the CPU, as is often the case with off-chip memory, there is additional system logic with awareness of the RAM's speed, or latency. This logic inserts CPU 'wait states' to allow time for the RAM access to complete, the number of wait states being pre-determined depending on the RAM’s known speed characteristics.
How does the logic 'know'? Simple systems often arrange off-chip memory types, such as RAM, ROM or DRAM, by address range. An example is how system logic uses the 8086 READY signal: when a DRAM address appears, it negates READY during the DRAM row/column latency period, then asserts it again when the data transfer completes.
A little sidebar. A sophisticated DRAM controller has knowledge of the DRAM page state, and if more than one DRAM is in use, knowledge of each DRAM's page state. This is a key area of system tuning: arranging DRAM address bits to avoid row miss penalties and thus keep the DRAM access latency low. Based on this, the DRAM controller will inform the host and assert READY (or its internal equivalent) sooner if the asked-for page is already available.
More complex CPUs, including complex ARM SoCs and desktop CPUs, interpose an internal super-fast RAM system, called a cache, in between the CPU and the system memory. The cache holds local copies of data recently fetched from system memory, and supplies them to the CPU without delay. This mostly avoids the RAM latency issue, allowing the CPU to run at full speed without wait states.
A cache controller monitors memory access activity and takes one of several actions based on the RAM read or write request, three of which will cause the CPU to wait:
The last case, write-back allows the CPU to write without delay. The cache controller does its best to post the write back to system memory. This write-back strategy is ok for a single CPU running a single thread, but requires extra care when multiple CPUs share the same system memory. There needs to be a means to ensure cache coherence between CPUs.
While we're talking about caches, the programmer has some control over cache behavior to avoid misses or write latency. For reads, software can initiate a cache pre-fill to avoid a cache miss. Similarly, software can force a latent write-back by initiating a cache flush. In most cases however the cache controller manages this on its own without user intervention.
"Time" is how it knows, if it knows at all. Back in the old days I had RAM that needed 200 ns to do a read or write. So one option is to set the CPU clock speed so that every 200 ns it would do a memory cycle.
I had fast and slow memory in one of my computers. The memory boards had a timer that you set for the speed of the memory. So the fast board held the "wait" line low for 100 ns while the slow board held back the CPU for 300 ns.
Some of the modern CPUs have the option of many different types of memory. There is a memory controller where you describe the speed and type of memory at each address range. The fastest memory is on the same silicon as the CPU and can be read or written on one clock of time. Off board memory might need two or three clock cycles to respond. The CPU is placed in a wait mode until the right amount of time goes by. In the case of EEPROM, the read is a little slow but the write takes a long long time (depends on temperature, supply voltage, age of the memory etc). In this case there are several different ways to know when a write is done. You can ask the memory 'is it OK to write again?' and it will respond with "Yes" or "No".
How does CPU know that a write to RAM was successful, like how does a faster CPU know that data was successfully written to a slower RAM?
It roughly depends on the decade in which a given system would have been state-of-the art.
Early computers had CPUs subjugated to the speed of volatile memory: the RAM speed determined how fast the whole thing could run, since it was usually the limiting factor.
As semiconductor memory became mainstream, the bipolar static memory was much faster than other kinds, so a system could use bipolar memory for fast access and cheaper "regular RAM" (e.g. dynamic RAM or NMOS static RAM) for slower accesses. The RAM accesses would slow the CPU down as required, i.e. the CPU would either be designed to slow down the bus for RAM access or there'd be an external WAIT signal to insert wait states into the memory cycle.
Eventually, the "gap" between the CPU and the memory grew, and memory access wasn't something that the CPU was doing directly. The CPU would send read/write requests to a memory hierarchy controller that would then use various tiers of memory (in terms of speed and capacity) to fulfill the requests. Once a piece of data was retrieved, the CPU would be informed, and could proceed with the task that was waiting for that data. Modern high-performance CPUs can execute instructions out-of-order, so that if one instruction is stuck waiting for data, other instructions ahead of it can be speculatively executed.
Even in a fairly "low-end" by modern standards microcontroller system it's possible to have a memory hierarchy. Raspberry Pico's RP2040 processor has on-board RAM that can be used for data and code. Part of that RAM can be allocated to be a cache for external memory that uses single, double or quad SPI (serial) interface and is quite a bit slower than the internal RAM.
At a high level writes are or can be fire or forget since all the information is there. You mail a letter, once in the mailbox you are done it is delivered when it gets delivered. Some designs have feedback but usually the feedback is at the nearest level, it has left the processor into the memory controller, just like I put it in the mailbox, I saw myself do it, done, I can go do something else. Reads you have to wait, like the dark ages of mail. You mail a letter with your order and your check then you wait until the product arrives. A full round trip. The task is not done until the response arrives.
With writes depending on the architecture, the overall system architecture, there are and may be a number of write buffers along the way to the ultimate target memory. dram for example. the processor itself the bus controller on the chip side of the processor bus, each layer of cache, the dram controller, can all have buffers that can store some number of writes. Each of these buffers is fixed in size and when full will back pressure the bus that feeds it so it is like my mailbox is packed with outgoing mail I cant squeeze another letter in I will have to just hold on to the letter until the mailbox has been emptied by the postal person. For each bus and buffer this can happen and the back pressure can go back all the way to the processor or can be herky/jerky. In general that is the only way you know there is something going on with the write system, when you get stalled, but usually your code does not know that it is stalled on the execution of a write instruction.
There are certain times where you need to know the write happened, particularly with bus controllers or peripherals, if you want to change say some memory mapping of a bus controller there is a write path from the processor to the bus controller itself then there are read/write paths along that bus which may not be routed the same way through the system. So you may have to do a write to the bus controller, then read back the register, THEN you can use the bus that is being controlled. The read requires a round trip, you can usually discard the contents of the read the purpose of the read is that for most systems or lets say this system in particular, there is also a command buffer, so reads and writes and any other bus commands that can be on this bus will go linearly into a buffer and get executed in the order they arrive, so if you do a write then a read, the write happens first, causing the bus controller to change how the bus works, the read then will happen after the write when it gets there, but your processor/process is stalled until the read completes, by the time the read completion gets back to the processor core the write is complete the bus is ready to use. there are system architectures where reads can pass writes and that is a whole other deal. But it is not uncommon to do a read operation after a write operation to have an indication to know that a write has completed. Otherwise you actually do not and will not know, the whole architecture around the processor and the layers of busses are designed such that you can fire and forget writes until/unless they back pressure to the point that the processor stalls on a write.
the busses themselves, today you will see things like write address bus, write data bus, read address bus, read data bus, both have handshakes (read and write) you start on the address bus i have this thing I want to write, here is the address. Then the controller on the other side, WHEN IT HAS BUFFERS SPACE AND CAN PROCESS YOUR COMMAND, comes back on an ack bus and says I see your write I am ready for your data. then you write the data on the data bus using an ID that was either created by you on the write address or an ID that came from the controller. then you get an ack back on the write data bus. this also allows for many transactions to be in flight at the same time (and for some architectures things can pass each other and get out of order depending on how far each item goes). The read would work the same way from a handshake perspective the real difference is the write data ack only has to go maybe to the controller or a first level cache and then get an ack. Where a read data ack has to go all the way to the target and come back with the data. so the data bus handshake TIME is longer for a read.
There are other architectures, some folks may say it has to do with the generation/evolution of when your thing was created, but that is not true there are many open and purchased/closed designs that use the good old read/write address/data chip select type bus and it is all based on timing of clocks and maybe a wait control signal to hold off the master to allow reads or writes to happen, you will see these on even the most cutting edge technology peripherals as well as a number of other designs. Once you get into a memory target/subsystem though you still will likely have writes buffered and reads have to take a round trip (although into a cache hopefully making that particular trip take less time than a miss).
reality is it is what the designers decided to do for that system, you will see the good old fashioned put the address and read/write line request on the bus and count X clocks and expect an answer on a specific clock. All the way to multiple busses with handshakes and ids and other parameters working multiple transactions at a time, even out of order. And everything in between.
Why should the CPU know if the write to RAM has completed? The CPU's job is to maintain the programmer visible register state. The programmer has some control and visibility over the programmer visible register state only.
how does a faster CPU know that data was successfully written to a slower RAM?
The only scenario in which a programmer should worry about writes to RAM taking longer than expected is if you have a read that immediately follows it to the same address. In other words, a store instruction followed by a load instruction to the same address. In this case, the read cannot get the updated value if the write to RAM has not completed, and there would be no way of knowing the value was incorrect. Memory barriers exist to solve this problem. Memory barriers enforce the program order of memory operations before and after the barrier.
Let's look at the implementation in the VeeR EL2 Core, a 4 stage pipelined RISC-V processor to get an understanding how memory barriers work.
Quoting from the Programmer Reference Manual of this processor:
The fence instruction ensures the store buffer is empty (i.e. data to be stored has left the store buffer), before going to the next instruction. This means the data to be stored and the write instruction to the RAM reaches the RAM controller before the read instruction. The RAM controller ensures the write to RAM is completed before the read from the RAM. Note that the RISC-V specification does not specify any deadline for the RAM write to complete. The Progress Axiom in the memory model of the RISC-V specification ensures that the write to RAM completes in a finite amount of time.
If you want to examine CPU to memory interfaces in detail, perhaps use a FGPA with a soft processor.
As an example Xilinx provide the MicroBlaze Soft Processor Core which allows you to configure options such as:
To get an overview you can start by using the Vivado Block Diagram editor to configure the MicroBlaze, memory and peripherals without having to write VHDL or Verilog.
Buses can be probed either in a simulator or the Integrated Logic Analyzer (ILA) in an actual FPGA.
