I think this question may have already been answered, in a way... But I am not satisfied. I'm reading this book about creating a NOT gate and it mentions that electricity flow from the collector to LED2 connected directly to ground stops when the transistor begins conducting.
Question is, why would LED2 stop if it is connected in parallel? 
Current only flows into the base of the transistor, not out. When the switch is closed, the circuit acts like this:
simulate this circuit – Schematic created using CircuitLab
LED2 is shorted out. With no voltage across it, it doesn't conduct. All of R2's current goes straight to ground.
With the switch open, the circuit acts like this:
The transistor is now off (no collector -> emitter current), so 6 volts is put across R2 and LED2 in series. This allows LED2 to turn on and conduct, with R2 limiting the current.
The "gate" you've shown is confusing I think because while its purpose may be to have LED2 and LED1 give opposite indications, the input to the "gate" is a current and the output is a voltage. In most cases, it makes more sense to define logic gate behavior for both inputs and outputs in terms of voltage. For example:
simulate this circuit – Schematic created using CircuitLab
In this example, a logic high is a signal which will sit at least 1.5 volts above ground if no more than 5mA is drawn from it; a logic low is a signal below 0.6 volts. The left circuit will output a voltage of at least 1.5 volts until the current drawn from it exceeds 5.1mA. The gate in the middle (starting before the 1k resistor) draws slightly under 1mA if the input is at 1.5 volts); one could attach four more similar gates without exceeding the 5mA limit of the source (though any current used to feed gates will reduce the LED brightness). If the input is over 1.5 volts, the base-emitter current will be at least 0.8mA, and the transistor should have no trouble sinking the 6.4mA necessary to pull its collector down to 0.6 volts. If the input is below 0.6 volts, then R3 should have no trouble pulling the output up above 1.5 volts if nothing downstream demands more than 5mA.
Note that all LEDs are referenced to ground. If one is willing to have a buttons and LEDs which are floating, one can build an "inverter" without needing any transistors, but incorporating such a device into a larger circuit will be difficult. Assuming LEDs want roughly 10mA at about 1.8 volts:
When the switch is open, D2 will drop 1.8V; the 3.2 volts remaining will be divided between R1 and R2, with each dropping 1.6 volts at 10.6mA, so D2 will get 10.6mA. When the switch is closed, D1 will drop 1.8V. R2 will drop 3.2 volts at 21.3mA, and R1 will drop 1.8 volts at 1.2mA, leaving about 9.3mA for the D1.
The purpose of a NOT gate is to have the OUTPUT (LED2) be the opposite of the input (LED1). If LED1 is lit, LED2 should be off. That being said,
When SW1 is pressed, current flows through LED1 turning it on, and also providing current to turn on Q1. When Q1 is "ON" is provides a low resistance path to ground--a short--bypassing LED2. LED2 was originally on because it was begging power through R2, but now that Q1 is "ON" that power is being shunted away from LED2 directly to ground through Q1. So LED2 is off.
When you release the switch, SW1, Q1 no longer gets current and shuts "OFF". LED1 is not getting any current and is also off; however, LED2 is now on because the current is flowing through it instead of Q1.
Make sense?
