Typical CPU instruction set: the CPU has several functional units, and when each instruction is read, some bits specify which functional unit is to be activated, while others specify the details of the operation.
Today's CPUs tend to spend some of their large transistor budgets on out of order execution, where they have a bunch of instructions in flight at once, trying to keep multiple functional units busy all while creating the illusion that it's still a serial instruction stream for backward compatibility. For the purposes of this question, to keep it simple, say we are talking about 90s technology with no more than a few million transistors.
VLIW: let's save time and transistors on decoding, and keep all the functional units busy, by having an instruction word that has a group of bits for each functional unit, specifying work for them all at the same time.
One problem with VLIW is it exposes more hardware details, making it difficult to keep backward compatibility with subsequent CPU generations without losing the advantages. Let's say we get around that problem with byte code and a JIT compiler.
Another problem is that not all workloads want to use all the functional units, and that's what I'm looking at now.
Let's say we're designing a CPU for an early 90s workstation, in the era of the 486 and i860. Roughly speaking, there will be two kinds of workloads, number crunching (e.g. CAD, simulations and of course 3D games) that uses the floating point units, and other stuff (e.g. word processing, compiling) that doesn't; this roughly corresponds to SPECfp vs SPECint.
For number crunching workloads, we could design a straight VLIW, maybe with a 64-bit instruction word that specifies integer and floating point operations, control flow etc all at once, and that will give great performance. But for integer workloads it will have poor code density because the bits specifying floating point operations will always be saying 'no FP ops this clock cycle'.
Would it make sense for the CPU to have a mode bit to switch between the two kinds of workloads? In FP mode, it works like the above, but in integer mode, maybe each 64-bit word just supplies a pair of 32-bit integer instructions? Would that give most of the simplicity and performance advantages of VLIW without the potential code density disadvantage?