One way to solve that problem is to use the (unilateral) Laplace transform, which allows to take initial conditions into account. For an RC high-pass circuit you have
with \$U_1(s)\$ the input voltage, \$U_2(s)\$ the output voltage, and \$U_C(s)\$ the voltage across the capacitor. If \$u_C(0)\$ is the initial voltage across the capacitor, the current through the capacitor is
With \$(2)\$ and \$I(s)=U_2(s)/R\$ you can express \$U_C(s)\$ in terms of \$U_2(s)\$:
For the zero-input response we set \$U_1(s)=0\$ and obtain from \$(1)\$ and \$(3)\$
with \$\tau=RC\$. In the time domain, \$(5)\$ corresponds to
This zero-input response \$(6)\$ is indeed identical to the one of an RC low-pass filter (maybe apart from the sign, depending on the definition of \$u_c(t)\$), but this is no surprise because if you short-circuit the input, both circuits become equivalent.