It basically comes down to the standards that define speeds, and devices that need to use these speeds. ICs then need to be designed to work at these speeds, and the transmission lines (cables) need to be able to pass that signal adequately. As there is need for faster speeds, also the technology used to make chips work at these speeds have advanced, such as automatic receiver equalization or pre-emphasis at the transmitter.
At some point HDMI supported up to 6.7 Gbit/s rate to support 1080p with 36-bit colors, so nothing faster existed. Then that increased to 9 Gbit/s when first 4K resolutions were defined in HDMI and devices started using those resolutions so better chips were needed. Next version of standard supported up to 18 Gbit/s to allow more bit depth and refresh rates, and also defines a standard for a cable to pass this, so again new chips to support this were needed. Same thing with next version again.
Communication protocol has so far remained basically the same, it is just faster. Cables need to be good enough to work with this, so longer cables need to be better quality than shorter cables, as long as they don't attenuate signal more than what standard requires, regardless of their length.