At around 20 °C, water has a relative electrical permittivity of about 80. Air is 1 so, any water content could be a significant factor in altering the capacitance measured. Given that capacitance is proportional to permittivity there should be a fairly direct link.
I guess all you have to do is calculate the percentage presence of water in a given "air" volume for a certain level of humidity. For instance, wiki states this: -
A parcel of air near saturation may contain 28 g (0.99
oz) of water per cubic metre of air at 30 °C (86 °F)
And, this document pretty much shows the same thing (ignore the red lines). At 30 °C and 100% RH, the water content is around 25 g per cubic metre: -
So, using the above example, 1 cubic metre contains 28 g of water or, as a volume that's 28 ml. And, there's 1,000,000 ml in 1 cubic metre so, as a volume percentage, 28 mg of water per cubic metre is 0.0028%.
Given that the permittivity of water is 80, then the capacitance could increase by 80 x 0.0028% = 0.224%.
But, you may also attract condensation onto your sensor and that will surely make the capacitance sensor read a much higher value.
Is there data available for the humidity dependence of the relative
permittivity of air at frequencies from 20 Hz to e.g. 10 kHz?
This graph from here suggests that you won't get a change in dielectric permittivity of water until you are in the GHz region: -
And this graph shows that the dielectric permittivity of water is constant down to 1 MHz. I think it's reasonable to assume that it will stay constant down to DC: -
And this graph from here pretty much nails it down to about 10 Hz: -