Connect the data lines from the USB port to the device, connect the ground of the external power supply to the USB ground and connect the external supply positive line to the device side USB supply only
Here is one example that shown the described connection scheme using a linear regulator as power source (limited to 1A that may not be enough) but you can use any 5v source.
In order to switch the power of the USB client when the plug is removed you can use a mosfet that turns on when the plug is inserted to the PC.
When there is no Vcc voltage in the base of the transistor the resistor R1 works as a pullup and keeps the mosfet off by keeping the gate high.
When the plug is inserted the PC side Vcc is applied to the base of the transistor so the current that flows through it creates a voltage drop across R1 and turns on the mosfet.
Note that the mosfet should be a logic level type that turns fully on with -5v Vgs.
Also the input should not exceed the max allowable Vgs (usually 20v but this is absolute max)
Another alternative is when a switching regulator is used that has a control pin, in that case the PC side Vcc supply can be used as a signal to turn the regulator on when the USB plug is connected.
The following schematic (I've modified the USB plugs slightly) shows such an example using a 1A switching regulator but the same principle applies to switching regulators of higher current.