1
\$\begingroup\$

In a general purpose computer(like normal pc), how does the CPU read the RAM, assuming that it first reads from the Cache.

Assuming the cache is an n-way set associative cache. Again, we would have L1 Cache and L2 Cache.

  1. Usually, CPU reads a block of data from the RAM. So these blocks can be several words. Doesn't this cost a lot of time/clock cycles? How is/can this be made more efficient? (Not talking about using cache here, but about the data transfer)

  2. RAM is quite slower than the CPU, how does the CPU still manage to be efficient?

\$\endgroup\$
  • \$\begingroup\$ often the data is delivered to the CPU is large chunks and it's dealt with very quickly and then the large chunk is moved on somewhere else/dumped after processing. Actual streaming of data is not as fast as the CPU in most cases \$\endgroup\$ – KyranF Mar 14 '15 at 0:07
  • \$\begingroup\$ is this homework? \$\endgroup\$ – tcrosley Mar 14 '15 at 1:10
2
\$\begingroup\$

There is throughput and latency.

On very simple, slow cores, the cache runs at the same speed as the CPU and can provide data in 1 cycle, so data is available immediately without stalling. When there is a cache miss, data is taken from main memory, and initial latency can be over 10 cycles. The good thing is that once the first data is available, the following data can be obtained quickly, hence the idea of burst transfers and cache fills. The CPU only needs a byte, or a 32bits word, but 32 or 64 bytes are transferred at once from memory to the cache.

On more advanced CPUs, the ones with L1, L2, DRAM and gigahertz clock, even the L1 cache contents cannot be obtained immediately. For instruction, there are mechanisms for predicting the instruction flow and fetching instructions in advance : Continuously fetch consecutive addresses, unless the instruction is a branch, a call,... For data, it is more complex. Using pipelining, some CPUs are able to have several outstanding data transfers before stalling. The real current solution for mitigating long latencies is out of order execution, the CPU does as much work as possible, even executing instructions not in program order, in order to hide the long latency of instructions like data reads and writes.

\$\endgroup\$
  • \$\begingroup\$ It would probably be worthwhile to mention critical chunk first cache block fetching (e.g., fetching the chunk with the requested data first — technically criticality ordering need not be the same as request order — and then the rest of the block) and early restart (i.e., only stall until the requested data is available). Also the additional latency of the burst is relatively low (e.g., 1GHz DDR3 takes 4ns to transmit the 8 transfers in a burst compared to typically over 50ns to start the access). Assuming locality, one of the other chunks supply data for what would be another miss. \$\endgroup\$ – Paul A. Clayton Mar 14 '15 at 12:52
1
\$\begingroup\$
  1. Yes. By using more efficient RAM, such as memory that supports burst mode.

  2. By using cache, which is faster RAM, along with a memory controller that continues to read from RAM while the CPU is busy executing what's in the cache.

\$\endgroup\$
  • \$\begingroup\$ It is also worth noting that modern "normal PCs" use multiple levels of caching, and the logic used to determine what is cached (especially in the case of modern read-ahead algorithms) varies from product to product and is not trivial. Also part of writing software often includes optimizations that help the CPU along by accessing memory in predictable / efficient ways. This is certainly a very broad question. \$\endgroup\$ – Jason C Mar 14 '15 at 0:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.