I wouldn't recommend it.
So, here's my theory:
SD card contacting depends on spring contacts in the SD card holder.
If your potting compound creeps into the gap between SD card and holder contacts before it hardens, it's game over.
But even if it's too viscous to creep in before it sets, chances are you now have removed all springiness from the contacts. Now, imagine your PCB gets colder: the potting beneath the contacts shrink. Since they are mechanically restricted by the potting in the direction of shrinkage, they might not be sliding along the SD card contacts while doing so, but simply go "away" from these contacts (my hypothesis here is that the potting compound will have similar thermal properties as the PCB, and thus lower thermal expansion per Kelvin; thus, it will "drag" the contacts towards the PCB surface, and not along the axes they will usually be following when contracting). That way, you might be breaking your contact.
Are you sure that microSD is the technology of choice if you need non-exchangeable memory? Wouldn't using a simple eMMC chip not be cheaper than soldering on a complex SD card holder?