How do I make a via that is filled with a non-conductive material in EAGLE PCB?
Your CAD tool doesn't have to really know anything about which vias are filled.
You can just add an instruction to your fab drawing, for example:
Fill all vias less than .35 mm diameter with nonconductive epoxy.
Fill all vias in designated region "A" with nonconductive epoxy [accompanied by a call-out in the drawing to region "A"].
Vias specified as 0.251 mm diameter to be drilled as 0.25 mm and filled with nonconductive epoxy
If you also want these vias plated over, you should add something like "...and plate over with minimum 18 um copper" after any of these instructions.
If in doubt, ask your fab shop about their capabilities and the preferred wording of such instructions.