A capacitor like that is typically used to provide power for short-term current spikes in the circuit is is directly connected to.
A typical application that needs these type of capacitors (called bypass or filter capacitors) are digital integrated circuits that need a extremely short spike of power every time the state change, but are very low power as long as the state is steady. This spike is caused by internal parasitic capacitances that need to get quickly charged to a different voltage. The inductance of the connection to the power supply might be too high for the power supply to deliver the required current spike, so a capacitor is connected close to the circuit causing the spikes, so that the connection to it has less inductance.
Another use of capacitor for DC is a filter capacitor after an rectifier, to have voltage available even when the AC input is just crossing zero. In that case, I would consider the capacitor part of the power supply, not of the supplied circuit.
In your application, the load is dominated by the coil of the solenoid, which can be modelled as a series circuit of an inductor and a resistor. At turn-on time of the transistor, the inductive behaviour of the coil is dominant, causing a slow rise of the current, and afterwards the pure DC resistance of the coil determines the current, which needs to be constantly supplied while the solenoid is turned on. Neither during turn-on nor in the steady state, a short current spike is consumed, so the capacitor is pointless. Also your LiPo battery does not need a capacitor to deal with zero crossings, as it is a DC source.
Side note: A different situation occurs on AC solenoids with a significant movement of the core: The inductance of an AC solenoid with the core pulled in is significantly higher than at turn-on time. As AC current is determined by the impedance, which might be dominated by the inductance of the solenoid coil, the AC current during pulling can be many times higher than the current in steady active state, up to the destruction of an AC solenoid (due to continuous over-current) if the mechanical movement does not occur, because it is blocked. Of course, an electrolytic cap can not be used to catch the current spike of an AC circuit.