Yes, your analysis is correct for a infinite capacitor.
However, anything less than that can be detected in arbitrarily short time. The problem is that the size of the signal to notice the difference gets smaller as the time to run the experiment gets smaller. Larger current makes the effect larger in the same amount of time.
Let's say your current is limited to 1 A and you have a 12 bit A/D in a 3.3 V microcontroller. Let's see how large a capacitor this could detect. The voltage change of a cap as a result of some Amps for some seconds is:
V = A s / F
Where A is the current in Amps, s is the time the current is applied in seconds, and F is the capacitance in Farads. Flippping this around to solve for the capacitance yields:
F = A s / V
The minimum voltage change we can detect is (3.3 V)/4095 = 806 µV. Plugging in our particulars, we get:
F = A s / V = (1 A)(1 s)/(806 µV) = 1.2 kF
That's a very large capacitor. If you can supply 5 A and wait 2 seconds, then you can detect a 10x larger capacitor. Or conversely, be able to measure 1.2 kF to 1 part in 10.
Yet another way to look at this is to apply a constant voltage for a fixed time, then see how much the open-circuit voltage went up afterwards. The voltage on the capacitor will rise exponentially, asymptotically approaching the fixed voltage being applied. Again let's say we can measure down to 1 part in 4095 of the applied voltage. That comes out to 0.000244 time constants. If that's how long 1 second is, then the time constant must be 4096 seconds. With a 1 Ω resistor, that means the cap is 4.1 kF.
Note that cheap $20 voltmeters can measure much smaller voltages than a 12 bit A/D running from 3.3 V.
Basically, it takes a unrealistically large capacitor to not be detectable via rather simple means.