Consider the following circuit. Assume there is \$1 \mathrm{V}\$ on each
of the capacitors to start with.

simulate this circuit – Schematic created using CircuitLab
What happens when I press the button? The capacitors get shorted
together. Thus the voltages across them must be equal.
Now when I said each capacitor has \$1\mathrm{V}\$ at the start, I meant
the voltage measured from the + terminal to the - terminal of each
capacitor. But for the voltages across the connected capacitors to be
equal, the voltages have to be measured across the same pair of
nodes. That means from + to - for one capacitor but from - to plus for
the other. You missed this point in your calculations.
But what is the voltage across the capacitors when the button is
pressed? By symmetry, it has to be \$0 \mathrm{V}\$.
Let's try using the conservation of charge. Let the final voltage be
\$V_x\$. Then
$$
C_1 V_x + C_2 \left(-V_x\right) = C_1 \cdot 1 \mathrm{V} + C_2 \cdot 1
\mathrm{V}
$$
Now since \$C_1\$ and \$C_2\$ are equal, the equation cannot hold for
any \$V_x\$. Where's the problem?
According to Wikipedia, "In physics, charge conservation is the
principle that the total electric charge in an isolated system never
changes." If we take this circuit as our closed system, the total charge
will be (ignoring stray capacitance) the charges on both plates of both
\$C_1\$ and \$C_2\$. A capacitor stores opposite charges on its plates. What
charge conservation gives us is in fact:
$$
\underbrace{C_1 V_x}_{\text{one plate of one capacitor}} +
\underbrace{C_1 \left(-V_x\right)}_{\text{the other plate of one
capacitor}} +
\underbrace{C_2 \left(-V_x\right)}_{\text{one plate of the other
capacitor}} +
\underbrace{C_2 V_x}_{\text{the other plate of the other capacitor}} =
\underbrace{C_1 \left(1\mathrm V\right)}_{\text{one plate of one
capacitor}} +
\underbrace{C_1 \left(-1 \mathrm V\right)}_{\text{the other plate of one
capacitor}} +
\underbrace{C_2 \left(1 \mathrm V\right)}_{\text{one plate of the other
capacitor}} +
\underbrace{C_2 \left(-1 \mathrm V\right)}_{\text{the other plate of the
other capacitor}}
$$
The problem with the way I used charge conservation at first, and also
the way you used it, was that we took only one plate of each capacitor
as the system. These systems are not closed. They are connected by wires
to the other halves of the capacitors on which charge can
flow. Conservation of charge does not apply to systems that are not
closed.
What do we do now? We can take a node, which is some wires that are
connected directly, as our system. It is an open system. Its total
charge is conserved not because no charge can flow across its boundary,
but because even little charge accumulated in it will repel more charge
from getting in. How strong this effect is is measured by stray
capacitance, which is outside the scope of this post. Again, Wikipedia
says:
This law is also called Kirchhoff's first law, Kirchhoff's point rule,
or Kirchhoff's junction rule (or nodal rule).
The principle of conservation of electric charge, combined with the very
large repulsive Coulomb forces that will occur if charge "piles up"
anywhere, imply that:
At any node (junction) in an electrical circuit, the sum of currents
flowing into that node is equal to the sum of currents flowing out
of that node
or equivalently
The algebraic sum of currents in a network of conductors meeting at
a point is zero.
Let \$V_1\$ and \$V_2\$ be the voltages at nodes 1 and 2 after the
switch is switched. Their voltages before the switch is switched are
\$V_\mathrm{ref}\$ and \$V_\mathrm{in}\$.

simulate this circuit
The net charge that flows out of node 1 has to be \$0\$, so we have:
$$
\left(C_1 + C_3\right) \left[\left(V_1-V_2\right) -
\left(V_\mathrm{ref}-V_\mathrm{in}\right)\right] +
C_2 \left[\left(V_1-V_2\right) - \left(-V_\mathrm{in}\right)\right] +
C_5 \left(V_1-V_\mathrm{ref}\right) = 0
$$
The net charge that flows in to node ground has to be 0, so we have:
$$
C_4 \left(V_2 - V_\mathrm{in}\right) +
C_5 \left(V_1 - V_\mathrm{ref}\right) = 0
$$
You can also do the same to node 2. The result will be the first
equation subtract the second, so it will be redundant.
Solve these equations, and you get:
$$
V_1 = \frac{C_1 + C_3 + C_4 \parallel C_5 + \frac{C_5}{C_4 + C_5} C_2}
{C_1 + C_3 + C_4 \parallel C_5 + C_2} V_\mathrm{ref}
$$
You can see that it will always be less than \$V_\mathrm{ref}\$, and
that it gets closer to \$V_\mathrm{in}\$ as \$C_5\$ gets larger. Your
intuition is correct.
If you prefer working with one unknown instead of two, you can change
node 2 to ground.