Yes, it's possible to make ICs more efficient, at the expense of speed. The problem is that a one is created by charging the gate capacitance of a mosfet to a certain value. If you want to change that bit to a zero, you must discharge that capacitor.
The determining equation for power in a CMOS process for active power is:
C = capacitance, V = Voltage, f = frequency(clock speed) and A = activity factor.
You add up billions of gates and the heat becomes significant. A theoretically 100% efficient machine would have infinite propagation delay, at least until someone discovers a better way than using voltage to signify a bit. Until then, the only ways to reduce heat are:
- Reduce clock speed.
- Reduce gate capacitance, usually by making smaller transistors.
- Reduce voltage.
- Turn off parts of the chip you don't need.
This is the approach taken in battery-powered applications. I have a 1 GHz ARM board that runs completely cool to the touch without fans or heat sinks. That's a remarkable improvement over a 1 GHz desktop IC from 10 years ago or so, but the current bleeding edge computers take those same improvements and crank up the speed, so it stays just as hot.