While we have dual cores and quad cores (and now octo cores). I often wonder if they sit side by side or are they completely stacked on top of eachother. If either or, what kind of benefits or draw backs will this bring to the motherboard, other hardware and performance overall?
They are side by side on chip, and tightly integrated with the memory and control interface/datapaths.
You certainly cannot stack them on top of one another (unless they are specially built for this).
The benefits of multicore are that the individual are so close together (on the same silicon die) and do not have the huge cap of normal input/output pads for internal signals. This lets the cores communicate very quickly with one another.
Notice how the memory controller has access to each core. And the cache (high speed memory, much faster than RAM) is shared among two sets of 3-cores.
Once the cores are physically separated per chip, all of those very close connections get incredibly long. That requires a lot more drive power. This in turn slows everything down.
On top of that they are not built for this. You might be able to run two cores in parallel (with special MOBOs) but it would not be the same as 'dualcore' vs 'quadcore'. (and not as good as if they were integrated on a single chip). It could handle independent threads well, but if they depended on each other the communication between the cores off-chip would be very slow. And a lot of internal nodes / caches are not exposed to the CPU's I/O.
Simply put, an end-user cannot do so directly. You'd need ways to avoid contention for resources such as bus lines, memory, etc.
That was a problem with the old home computers and why DMA and interrupts exist. For instance, while the old computers used various coprocessors, they could not be used at the same time as the main CPU. So the CPU would need to be halted and taken off the bus while another device is placed on the bus and allowed to use the memory. Then everything would switch back to the way it was, and the CPU would continue from there.
So if you wanted to take old CPUs and cobble together a multi-core system, you'd need to create an elaborate memory controller with caches. Then you'd likely need to figure out how to create a way to select the context to determine which CPU runs what.
As far as what CPU manufacturers do, yes, they do place the cores beside each other. They also have internal wiring to negotiate "who" can do what when. These are signals that make no sense to expose to the motherboard. While they could stack the individual dies, that seems like it would complicate heat dissipation. They are placed beside each other to provide more surface area to transfer more heat to the heat spreader (and heat sink).