I was studying pipeline concept in microarchitecture. My professor told me that memory read and write operations take longer time to execute since DRAM has a maximum frequency of 1333Mhz. Hence, when a memory i/o operation is executed, the execution takes longer time. This increases the latency of the processor. Other instructions(fetched after memory i/o instructions) get 'delayed'. I was thinking of a vague idea that why cant we have a different execution block for performing all the memory operations. In the fetch hardware itself, once we know that the instruction is a mem instruction, instead of being executed by the main execution unit itself, we could have a different execution unit for such mem instructions. Could someone comment on the thought? Should i propose this idea to my professor? He is a bit grumpy!
-
\$\begingroup\$ Many modern processors are both superscalar and use out-of-order execution. You may like to read up on these concepts, specifically the idea of multiple pipelines and a reorder buffer. These allow some memory operations to be executed separately but note it always needs to appear like in-order execution outside the processor. \$\endgroup\$– DavidCommented Nov 22, 2015 at 22:49
2 Answers
There's no reason you can't do that, but there's also no obvious reason you'd want to. Note that such a mechanism would be useless when dealing with memory reads, since the memory is read in order that the execution unit can do something with the retrieved data. Whether the data is read via the execution unit or some dedicated mechanism, the CPU will have to wait for the data before it can do anything else.
It's true that you might save a bit of time by handing off write operations to your proposed mechanism, but that wouldn't help a lot and would require considerable added circuit complexity.
Note that the standard approach to dealing with limited external memory bandwidth is cache, and the cache manager is arguably (sort of) an example of what you are proposing.
-
\$\begingroup\$ If the memory execution unit is decoupled, you can sometimes hoist "read" instructions earlier in the instruction stream, so that the "read" is complete by the time the read result is required. (Not always : sometimes the read address cannot be computed any earlier) \$\endgroup\$– user16324Commented Nov 23, 2015 at 0:19
Yes, this is done in many designs (pre-dating the question by many years), although typically the front of the pipeline will be common. Once the instruction is decoded up to the point of having mainly memory transfer information available, it can be offloaded to a dedicated unit which may keep track of a small number of outstanding transactions.
The critical thing to remember is that these load or store instructions (it's much more trivial in a RISC instruction set, the compiler can optimise the scheduling) will need to interact with other instructions - you can't just fire-and-forget a load. This means that your simple pipeline view doesn't really hold true any more - there are hazard interlock functions which need to enforce correct ordering, and most likely you will want forwarding paths - so a store which depends only on a load (pointer address, for example) can get it's data direct from the load-unit, rather than the store remaining in the main pipeline until the load has returned data.
Instruction ordering, and a clearly defined definition of data transfer ordering are critical to software working properly. Even if the programmer realises that the processor is permitted to perform instructions out-of-order, they need a mechanism to control when the ordering is important. In a multi-thread or multi-processor system, these issues introduce even more complexities.
