This diagram may help.
Figure 1. Figuring out the required voltage drop across the current-limiting resistor for a green LED at 20 mA. Source: LEDnique.
The graph shows the VF (forward voltage) of various LEDs at currents between 0 and 50 mA. We can see that at 20 mA the green LED will drop about 2.25 V. You are feeding from a 5 V supply so that means that the voltage drop across R is 5 - 2.25 = 2.75 V.
From Ohm's law we get \$ R = \frac {V}{I} = \frac {2.75}{0.02} = 137 \ \Omega \$. Pick the nearest standard value.
You can easily work out the resistor values for each of the other colours too.
For other currents slide the diode and resistor vertically to the desired value.
An alternate approach using the same graph is to draw the load-lines for a range of resistors.
Figure 2. Various 5 V resistor load-lines overlaid on IV curves.
[OP used] (1) blue: 5 kΩ, (2) yellow: 350 Ω, (3) red: 150 Ω, (4) orange: 1 kΩ, (5) green: 50 Ω.
Let's plot these points on the load lines.
Figure 3. The OP's resistor values found to give reasonably even brightness on a range of colours.
The plots indicate to me that there is a very large discrepancy in the efficiency (or possibly the optical focus) of the LEDs. If they were all the same efficiency the points should be close to the same height off the horizontal axis. From the results it appears that the blue is super-high efficiency but the green (which is in the most sensitive region of human vision) is terrible.