I'm trying to learn the device physics of these transistors. If I were out of diodes and really needed one to complete my circuit, could I just replace the diode with the emitter and base of a BJT? Or is the idea that BJTs are made of diode junctions just an analogy.
What you can do is replace a diode with a diode-connected BJT: a two-terminal device made by tying the base and collector together. Sometimes diodes are made this way on integrated circuits. One good reason for actually using a BJT as a diode is that you can closely match another BJT of the same type. This is useful, for instance, if you're trying to build a current mirror with discrete components. If both devices have identical voltage-current curves, and are close together (thermally coupled) then the mirroring is accurate and, free of thermal runaway. Note how current-mirror circuit schematics show a diode-connected BJT, rather than a diode.
By the way, with some resistors, you can turn a transistor into another two-terminal device, the VBE multiplier (a.k.a "rubber diode"). By varying the resistor values you can make the rubber diode exhibit different voltage drops. The diode-connected BJT can be regarded as a special case of the rubber diode, where one resistor is infinite (because there is no connection from base to emitter) and the other is zero ohms (base is tied to collector).
But the question asks, why don't we just use the base-emitter diode in isolation (leaving the collector disconnected). While that will work as a diode, the diode-connected BJT has the advantage that most of the current flows through the collector. In the diode-connected arrangement, the BJT is prevented from saturating, so the current splits between base and collector according to the transistor's beta. If you use only the base and emitter terminals, then you're only pushing base current. Transistors are usually constructed with the assumption that the base current (and hence handling ability) is small compared to collector current.
(The diode-connected transistor doesn't saturate because the collector and base diode cannot forward bias. Transistor saturation occurs when both diodes are forward biased.)
Yes, you can within some limitations. The B-E and B-C junctions are diodes, but a bipolar transistor is more than two diodes back to back. The diodes are made in a sandwich so that the depletion region of each junction extends to the other.
You can use the B-E junction as a diode, or the B-C junction, but as Kaz said it is more effective to short C and B if you want a overall diode. This gives you a higher forward current rating. The base isn't intended to handle large currents.
The downside is that the reverse voltage of the diode will be rather poor. Take a look at a BJT datasheet and you will see that the maximum reverse B-E voltage is low, usually quite a bit lower than the maximum C-E voltage. If you can live with the low reverse voltage, then a BJT with colletor and base shorted can be used as a ordinary silicon diode.