It doesn't, here is a comparison of cache sizes:
A GPU pipeline looks like this:
A CPU pipeline looks like this:
So why don't you see memory modules on a GPU PCB if they are the same? The first one I can think of would be that GPU's memory is much faster (and uses new memory technologies like DDR3 for a CPU vs DDR5 for a GPU) (look at the table). Memory module pricing is also likely an issue. It is much easier to put chips on a board than a module with connectors and size is also likely a constraint.
The actual size of memory is only limited by how much addressable space, so GPU's would probably win there, since CPU's generally have 64 bits and GPU's have more. What people use in practice is usually due to cost.
They are similar in that each one has an L1, L2 and L3 cache, then global memory. If the data isn't in the L1 cache, it looks in L2 which is bigger and slower, then in L3 which is bigger and slower than L2. If the data is not there it then searches in the global memory, and the processor has to wait a few clock cycles for this to happen.