Transfer functions always seem to take the form $$F(s) = F(j\omega)$$
No not really..
The transfer function, usually denoted \$ H(s) \$, is generally speaking defined for all \$ s= \sigma + j\omega \$ in the complex plane (s-domain). When analyzing things like BIBO-stability and causality of an LTI system we need to cosider the entire s-domain.
As an example if the transfer function has a pole at \$ \bar s = \bar \sigma +
j \bar \omega \$, ie. \$ H(\bar s) \rightarrow \infty \$.
Then \$ \bar \sigma > 0 \$ means that the system is unstable (it has a pole in the right-half-plane).
What is the justification for dropping sigma, or the real part of \$ s \$?
There is no justification for "dropping" the real part \$ \sigma \$, which we don't generally do, this is the wrong phrasing.
The reason you usually see the transfer function evaluated only at imaginary values of \$ s \$ is that this corresponds to a real input excitation, ie. a sinusoid.
Or in plain English; real-world signals lie on the \$ \sigma = 0 \$ line of the complex plane..
A lot of theory goes into explaining the s-domain in detail, I am very much brushing over it here, it is just to give you an idea of the concept.