Many often confuse the meaning of Moore's Law... it refers to the number of transistors on a chip, not performance.
A while back, it became apparent that the gains from increasing clock frequency on chips was not worth the expense and chip makers started adding extra cores to CPUs.
However, the increase in the number of cores on consumer chips has not matched the increase in transistors on each chip.
I surmise that a lot of these transistors have gone into features such as prediction logic ect, because it is difficult for some workloads to be parallelized, or many programmers find parallelizing their programs too time intensive, or CPUs are optimized for existing programs.
However, from my perspective, I would like to see transistors go into increasing core count and on-chip-cache as this would benefit my programs more than marginal increases in single threaded performance given that I have no trouble writing multi-threaded code for most of my particular goals.
If I use the extra transistors for a really large cache, I will not have to make as many trips to memory, which can also be a big performance booster.
Am I incorrect as to the reason core counts do not seem to be increasing at the same rate as the number of transistors? Or is there also some diminished return for increasing core count even for easily parallelized work loads such as memory bandwidth?
Why have core counts not increased at anywhere near the rate as the number of transistors on a chip?
Edit: Just because a workload can be run in parallel does not mean it is an appropriate task for a GPU ect which tend to deal with doing a lot of floating point calculations. CPUs have diverse general purpose capabilities which more specialized chips lack.
An example of this could be, let's say I have a set of 50 heuristic functions I need to run against a large set of data that is already in memory.
This is easy to multi thread, give each function its own thread, and you can multi thread it further by diving up subsets of the data for each function (if the data is not highly interdependent). You could easily satiturate all the cores of even a top end Xeon processor, but you won't be able to make much use of a GPU or SIMD.
Or, just a common web application serving many different requests that do not need to be coordinated.
Or, just several different applications running on the same server for political or administrative reasons.