One of the first questions you have to ask when dealing with op-amps is "which model shall I use?". In this question, I'll go with the one that says that an op-amp has infinite gain, no loading on the input pins no matter what, and an output that saturates at the rail voltages (so, ground and +5V, in this case).
If, when the power is turned on, whatever transients that happen cause the \$v^-\$ input to be higher than the \$v^+\$ input when the op-amp starts working, the op-amp will pull the output down to ground. Then it will stay there forever, because \$v^+\$ has been pulled lower than \$v^-\$ and there's nothing to change that circumstance.
If, when the power is turned on, whatever transients that happen cause the \$v^-\$ input to be lower than the \$v^+\$ input when the op-amp starts working, the op-amp will pull the output up to +5V. Then it will stay there forever, because \$v^+\$ has been pulled higher than \$v^-\$ and there's nothing to change that circumstance.
Google "comparator" and "hysteresis" for more detailed information.