Look for the bold block of text near the end, for the TLDR.
This "path of least resistance" stuff drives me bonkers. There's only one way to describe this situation properly, and it's using Kirchhoff's Voltage Law (KVL), Kirchhoff's Current Law (KCL), and Ohm's law. Once you have applied and understood those, this question will never come up again. Bare with me, this will be worth it.
simulate this circuit – Schematic created using CircuitLab
Applying KCL to this is trivially simple. At junction X, current \$I_1\$ comes in from above, and can leave either right, \$I_0\$, or down, \$I_2\$.
Just like water in a junction of pipes, what goes in must come out:
$$ I_1 = I_0 + I_2 $$
The MCU input has an extremely high impedance, drawing no current at all, so:
$$ I_0 = 0 $$
This means that \$I_1\$ and \$I_2\$ are equal:
$$
\begin{aligned}
I_1 &= I_0 + I_2 \\ \\
I_1 &= 0 + I_2 \\ \\
I_1 &= I_2 \\ \\
&= I
\end{aligned}
$$
In other words, current through those two resistors is the same, and I shall call it \$I\$.
Now we apply KVL, which states that the voltages across R1 and R2 must add up to equal the voltage source (power supply) Vs:
$$ V_{R1} + V_{R2} = V_S $$
Using Ohm's law we can find \$V_{R1}\$ and \$V_{R2}\$:
$$
\begin{aligned}
V_{R1} = I \times R_1 \\ \\
V_{R2} = I \times R_2 \\ \\
\end{aligned}
$$
Combining those with the KVL equation from before:
$$
\begin{aligned}
V_{R1} + V_{R2} &= V_S \\ \\
IR_1 + IR_2 &= V_S \\ \\
I(R_1 + R_2) &= V_S \\ \\
I &= \frac{V_S}{R_1 + R_2} \\ \\
\end{aligned}
$$
Keep that last line in mind for later, we'll be using it a lot to find current in all three of your scenarios.
Lastly our output is the potential at X (also named OUT), which by KVL is higher than ground (0V) by an amount equal to the voltage \$V_{R2}\$ across R2:
$$
\begin{aligned}
V_{OUT} &= V_X \\ \\
&= 0V + V_{R2} \\ \\
&= V_{R2} \\ \\
&= IR_2 \\ \\
V_{OUT} &= IR_2 \\ \\
\end{aligned}
$$
That last line is also critically important, and is directly related to your question. Maybe you see why already.
Alternatively, using KVL again, we can state that node X is lower in potential than node CC, by amount \$V_{R1}\$:
$$
\begin{aligned}
V_{OUT} &= V_{CC} - V_{R2} \\ \\
&= V_{CC} - IR_2 \\ \\
\end{aligned}
$$
That's saying the same thing, really, but it will be useful when analysing your second and third scenarios.
Our super-important-equations are therefore:
$$
\begin{aligned}
I &= \frac{V_S}{R_1 + R_2} \\ \\
V_{OUT} &= IR_2 \\ \\
V_{OUT} &= V_{CC} - IR_1 \\ \\
\end{aligned}
$$
Let's examine your first (left) scenario, replacing those two resistors with an open gap and a switch which can have 0Ω resistance, or infinity (∞Ω), depending on its state:
simulate this circuit
On the left, switch open, current \$I\$ and output potential \$V_{OUT}\$ are:
$$
\begin{aligned}
I &= \frac{V_S}{R_1 + R_2} \\ \\
&= \frac{5V}{\infty\Omega + \infty\Omega} \\ \\
&= 0A \\ \\
V_{OUT} &= IR_2 \\ \\
&= 0A \times \infty \Omega \\ \\
&= \rm{undefined}
\end{aligned}
$$
The output node OUT is just floating in space, not connected to anything. That's where we get the term "floating" from. There's no current anywhere, and no clearly defined potential at OUT, which is bad, as you noted already, but not destructive.
In practice, this floating situation turns node OUT into an antenna, which is capacitively coupled to any source of changing potential elsewhere, such as nearby digital signal lines, mains power cabling etc. That makes the potential tend to flap up and down uncontrolled, which is bad.
Above right you have this scenario, where the switch is closed:
$$
\begin{aligned}
I &= \frac{V_S}{R_1 + R_2} \\ \\
&= \frac{5V}{\infty\Omega + 0\Omega} \\ \\
&= 0A \\ \\
V_{OUT} &= IR_2 \\ \\
&= 0A \times 0\Omega \\ \\
&= 0V \\ \\
\end{aligned}
$$
That's well defined, not problematic in any way.
Your middle scenario:
simulate this circuit
This is slightly more interesting, because we can't use \$V_{OUT}=IR_2\$ (try it), but we can use \$V_{OUT}=V_{CC}-IR_1\$. On the left:
$$
\begin{aligned}
I &= \frac{V_S}{R_1 + R_2} \\ \\
&= \frac{5V}{0\Omega + \infty\Omega} \\ \\
&= 0A \\ \\
V_{OUT} &= V_{CC} - IR_1 \\ \\
&= 5V - 0A \times 0\Omega \\ \\
&= +5V \\ \\
\end{aligned}
$$
On the right (switch closed):
$$
\begin{aligned}
I &= \frac{V_S}{R_1 + R_2} \\ \\
&= \frac{5V}{0\Omega + 0\Omega} \\ \\
&= \infty A \\ \\
\end{aligned}
$$
Obviously that's bad, and incorrect since in reality supply impedance and wiring and switch resistance will limit maximum current, but something will break. There's no point even trying to find \$V_{OUT}\$.
Now the last scenario, with a resistor:
simulate this circuit
On the left, with the switch open:
$$
\begin{aligned}
I &= \frac{V_S}{R_1 + R_2} \\ \\
&= \frac{5V}{10k\Omega + \infty\Omega} \\ \\
&= 0A \\ \\
V_{OUT} &= V_{CC} - IR_1 \\ \\
&= 5V - 0A \times 10k\Omega \\ \\
&= +5V \\ \\
\end{aligned}
$$
There it is, right there, mathematical proof that the "pull-up" resistor is causing \$V_{OUT}=V_{CC}=+5V\$ when the switch is open. With no current through it, that resistor has no potential difference across it. Therefore there is no difference in potential between its two ends, nodes CC and OUT.
On the right, switch closed:
$$
\begin{aligned}
I &= \frac{V_S}{R_1 + R_2} \\ \\
&= \frac{5V}{10k\Omega + 0\Omega} \\ \\
&= 500\mu A \\ \\
V_{OUT} &= IR_2 \\ \\
&= 500\mu A \times 0\Omega \\ \\
&= 0V \\ \\
\end{aligned}
$$
Or, we get the same \$V_{OUT}\$ like this:
$$
\begin{aligned}
V_{OUT} &= V_{CC} - IR_1 \\ \\
&= 5V - 500\mu A \times 10k\Omega \\ \\
&= 0V \\ \\
\end{aligned}
$$
