Working with very limited space (In Altium) and need to place vias to make routes between the red solder castellation plates on the top layer to the respective connector pins on the bottom layer, both shown in the image. ..., how small can I effectively make the diameters of my vias?
Other answers have given good information answering the question as asked, but this might be an X-Y problem. You have to get the signal from the top to the bottom in limited space, so you've decided the solution is to make the vias as small as possible.
I'm assuming you can't just move the component away from the board edge to give you more space, because that's a pretty obvious solution also.
But have you considered just using via-in-pad technology? To do this, you would drill an ordinary-sized (8 mil / 0.2 mm or larger) via through the middle of the component pad. Then the via is filled with epoxy and copper is plated over it so that the pad doesn't have a void in it. This does add some cost, but it is probably not as much as trying to drill holes below 8 mils.
Any shop capable of drilling 8 mil (or lower) vias these days should also be capable of doing via-in-pad.
How To
The usual way to produce via-in-pad is to place the vias in Altium with a unique size, for example 8.1 mils instead of 8 mils. Then a fabrication note something like "vias indicated as 8.1 mils are via-in-pad plated over". If you are designing for high volume production you can add a spec for the flatness of the resulting pad and for the type of the epoxy fill.