I know that a capacitor would resist a change in the voltage across its two ends.
You've got the right idea. I would say that it tends to maintain the voltage across it in the short term.
Now if I connect this capacitor to a DC source, and if it has to maintain the same voltage as before, shouldn't the capacitor act like a short circuit throughout (so that the voltage = 0 V)?
There's a little problem in the model. It assumes an ideal voltage source. A real one will have some internal resistance. If it didn't an infinite current would flow into the capacitor to bring it up to the voltage of the battery. The next problem is that the capacitor itself will have some internal resistance - the equivalent series resistance (ESR) - which is also missing from the model.
Why should it build up its voltage to be equal to the source/battery voltage?
Because charge will flow in due to the difference in potentials.
Similarly, why should a capacitor discharge when disconnected from the power supply?
Because charge will flow out due to the potential difference across the resistor.
If it has to maintain the same voltage (say V) across its ends, it shouldn't discharge right?
It will maintain the same voltage across its ends while disconnected. Once you connect it to something else charge can and will flow in and out if there is a voltage difference.
Shouldn't it just hold the potential within it so as to avoid a voltage change across its terminals?
Nope. If that were the case they would be useless as it would be impossible to ever charge them.
After the switch is thrown you have a simple C-R discharge. The time constant is given by \$ \tau = RC \$ and you should memorise the following for a charge or discharge curve:
- At t = 1τ the capacitor voltage will have reduced by 63%.
- At t = 3τ the capacitor voltage will have reduced by 95%.
- At t = 5τ the capacitor voltage will have reduced by 99%.

Figure 1. RC discharge from 9 V. Source:[http://electronicsclub.info].
From the comments:
Why would the capacitor try to equal the battery voltage? Wouldn't it like to maintain the same voltage across it as earlier? (i.e. 0 V.)

simulate this circuit – Schematic created using CircuitLab
Figure 2. The charging cycle. R1 is the battery source resistance. R2 is the capacitor ESR.
- Look at Figure 2. At the instant the switch is closed there is 9 V on the battery and 0 V on C1. That means that there is 9 V across R1 and R2 so current will flow. This is basic Ohm's Law \$ V = IR \$.
- Current is the rate of charge flow. If there is current then there is a movement of charge from the battery to the capacitor.
- The relationship between charge, capacitance and voltage is given by \$ Q = CV \$. For a given capacitor value the charge and voltage are proportional.
So why do people say that a capacitor tries to maintain the same voltage across its ends in a circuit?
See my answer to RC differentiator giving a higher output amplitude than input amplitude for an example of where this is a useful concept.
it has to maintain the same voltage as before
is incorrect ... think of the capacitor as a bucket with a 1cm hole in the bottom ... if you set the bucket in a lake, without submerging the bucket fully, the water will flow into the bucket through the hole until the water in the bucket and the water outside of the bucket are at same level .... when you raise the bucket, the water flows out ... the charge in the capacitor behaves in a similar manner as the water in the bucket \$\endgroup\$