USB3 interface contains two nearly-independent busses and has to have two controllers in one chip. One operates as USB3, and another operates as USB2. The signals run over separate set of wires. For a flash drive either USB3 or USB2 part of the controller IC is activated, depending on the host port capability. As "user1850479" commented above, USB3 controller has to operate at much higher clockrate (~10x) internally, and must support much more complex USB3 (full-duplex) interface driving both transmitter and receiver simultaneously.

When connected to USB2-only port, the flashdrive IC operates only in USB2 mode, and the entire logic of USB3 (and PHY) is in shutdown state. That's why there is substantial difference in power dissipation between two modes.

Now, the question "What does such a controller do when "sitting idle"?" is an interesting topic. The USB3 interface is designed to support four levels of activity, U0 state (continuous active operation), and power saving states U1/U2 and U3 (suspend). 

In U0 state the transmitter an receiver are continuously transmitting/receiving/decoding link management packets, and even the "idle" state of the USB3 bus is a continuous transmission of "logical idle" state, which is encoded and scrambled, and does not differ from any data or control packets, and has the same signal amplitude. The signal looks like a plain noise. Maintaining this state requires a lot of power (~0.5-1W).

In U1 and U2 the interface is going up and down into active/inactive state with different enter and exit latencies. Inactive state is "electrical idle", and signals are at zero. However, to operate in this mode both host and device must support it. The entering and especially exiting the U1/U2 states needs to be coordinated. And every exit from low-power state involves link re-training, with includes LFPS wake-up handshake and exchanging thousands of clock synchronizing and symbol sync patterns. The process occurs to be challenging for the entire industry, and for more than a decade the support for U1/U2 in mass storage devices can be found only in 20-30% of devices on the market. In not so distant past not every PC would support the USB3 link power management, it was simply disabled in OS to avoid incompatibility with 80% of devices.

Bottom line, if a device or host do not support USB3 link power management, the USB3 device will run relatively hot.