This is done various ways. Think of it as a bootloader perhaps. Either purely in hardware or combination of hardware and software, the device would need just enough functionality to show up on whatever bus (USB, PCIe, etc)(get enumerated), but also be identifiable as some product. Then the driver will download the latest application firmware so that it does not have to be burned to flash every time an update is available and also flash is expensive.
It can be a case with usb for example that the logic in the chip can enumerate as a simple device, the firmware is downloaded into sram, then the logic or that software then re-enumerates by electrically disconnecting then reconnecting but reconnecting as per the software/firmware has chosen to use the device.
PCIe does technically have a re-enumeration thing but not the kind of thing you would use for this.
So it is not a chicken and egg problem, there is logic or firmware or a combination of the two that gets the item on the bus initially but the full featured product firmware is then downloaded.
If there is firmware on the device in order to facilitate this it may be a mask rom or a rom (or flash) on the part that the chip vendor has provided for the chip customers. Or it may be the case that the chip level product does not support anything like this but the system design for the product is to have just enough firmware to get started and then the rest downloaded live.
So if there is software/firmware involved (in getting on the bus in the first place) there is some form of non volatile storage containing code on the product, so it is not a case of there is zero rom/flash but maybe less rom/flash than if the whole of the firmware lived on the product.
