Your original schematic is somewhat confusing, as it does make it look like you are trying to short-circuit the USB power rail. I have redrawn it below with a circuit that should do what you want. It uses the switch to pull down the gate of a P-channel enhancement mode MOSFET.
simulate this circuit – Schematic created using CircuitLab
The main point to note is that the current to drive the LED no longer comes from the switch IC, but instead directly from the USB power rail. The switch input is not intended to provide significant power, and to get the current amplification benefit from the transistor it needs to get its current from a higher amperage source.
You could achieve this using a PNP BJT, but that will have a fixed saturation voltage across the collector and emitter of the PNP transistor (around 0.15V) requiring a recalculation of R1. It also requires a series base resistor whose value must be calculated with the Hfe of the BJT in mind. It does have the advantage of not placing a low gate threshold voltage requirement on the transistor, and more suitable PNP BJTs are available in through-hole packages than P-channel MOSFETs.
simulate this circuit
This circuit uses the BC557 purely because it is the complementary part to the BC547 you used in your original attempt. The guaranteed Hfe of the lowest grade (A) parts is 110, so for -20mA collector-emitter current we need at least 180uA of base current. Using a 4k7 base resistor we have 5-0.7 = 4.3V across it, giving a base current of 4.3/4.7k = 915uA. This is enough to drive the BJT hard into saturation without excessive base current flowing. Note that with the MOSFET version of this circuit there would be practically no gate current flowing, so you would gain greater efficiency that way. I haven't changed R1 to allow for the collector emitter saturation voltage - the required reduction would likely be around 10 ohms, so you may choose to neglect this.