Your choice of resistor values will hinge on a few factors:
Power. You want to keep things as cool as possible, and you want to avoid wasting energy, so that your power supply requirements are easy to meet, and your batteries last as long as possible. This usually means that you should aim for large resistances, to keep currents small, and to ensure that power dissipation in any resistors is low.
Current. Many things are trying to impose potential difference across those resistors, and you need to work with currents that those devices are easily able to supply. This places a lower limit on resistances, and favours larger values over small. Where op-amps are concerned, their input requirements must also be considered. They may sink/source some small current too, and resistances must be chosen to keep signal currents appreciably larger than this, to avoid compromising op-amp behaviour. That places an upper limit on resistances you use in the op-amp's periphery.
Interference. Unwanted currents will be induced in every path of your circuit, from various sources. They can arise from changing magnetic fields nearby, or from capacitive coupling to nearby mains wiring, or due to radio signals being picked up like an antenna, among others. These changing/oscillating currents may be small, but if they pass through a large resistance, that resistance will develop a significant potential difference according to Ohm,'s law \$V=I\times R\$. To mitigate this, we usually aim to keep resistances as small as possible.
Noise. Every component in a system introduces thermal noise, including resistors. Sadly, this noise increases with the resistance, and if noise is a big concern, then again you would favour small resistances.
In other words, your choice of resistance values is always a balancing act, to satisfy the requirements of your circuit in terms of power, current ability of various elements, noise and immunity to interference.
As an example, take a typical inverting op-amp configuration (OA2, R1, R2), whose input is provided by a prior op-amp stage (OA1):
simulate this circuit – Schematic created using CircuitLab
Here OA1 is applying +5V at its output, and the inverting amplifier of gain \$-\frac{R_2}{R_1} = -2\$ should therefore produce an output of −10V. Let's calculate the corresponding input and output currents \$I_{IN}\$ and \$I_{OUT}\$, using Ohm's law:
$$ I_{IN} = \frac{(+5V)-(0V)}{100\Omega} = 50mA $$
$$ I_{OUT} = \frac{(0V)-(-10V)}{200\Omega} = 50mA $$
Considering that the largest current most op-amps are able to source or sink is about 10mA, clearly the outputs of both OA1 and OA2 are overloaded. The voltages we expect to be present in this design will not be achieved, because the op-amps are simply unable to comply.
If R1 and R2 are increased by, say, a factor of ten (1kΩ and 2kΩ respectively), then the currents involved are reduced by that same factor, to 5mA, which is within the capabilities of the op-amps, and will work just fine.
You could argue that increasing R1 and R2 by a factor of 10000 (to 1MΩ and 2MΩ) would also work, but then noise and interference become significant. Worse, \$I_{IN}\$ becomes so small that it is comparable with the op-amp's own input current (0.1μA in my example above):
$$ I_{IN} = \frac{(+5V)-(0V)}{1M\Omega} = 5\mu A $$
The closer \$I_{IN}\$ becomes to 0.1μA, the less "ideal" the behaviour of the system, and the less linear the relationship between input and output potentials.
Megohms is too much, hundreds-of-ohms is too little, and so here you would be considering values in the kilohms to hundreds of kilohms range. If this circuit is to be battery powered, then the higher end of this spectrum is better. If power supply isn't an issue, then the lower the better, kilohms is fine.
It just so happens that currents we tend to work with in signal-level systems like this are often in the tens of microamps to tens of milliamps range, and the voltages are usually just a few volts. Using Ohm's law alone, you can see that these conditions correspond to typical resistances in such typical applications, in the kilohms.