I am developing a startup, and we are planning on using the SIMCOM SIM800F module for cellular communications on our device. If we use this module do we have to go through the FCC Certification process? Does the kind of antenna we use matter to the FCC (in-house PCB trace, or commercial rubber ducky)?
The only way I know to avoid the full FCC certification is to get a module that has an antenna already attached, whether an antenna connector, chip antenna or PCB trace. For example this Bluetooth module which I have used in the past comes with a chip antenna, in the upper left corner:
(I know you are using a GSM, but the same principal applies.) Note the FCC ID on the top of the module, this means the module has been pre-certified, because it has a chip antenna plus the relevant circuitry which allows the module to be tested as a whole. I have seen this even with modules that have an antenna connector, such as SMA or U.FL. Even with the pre-certification, you will have to go through some testing (to check, for example, that your microcontroller or other circuitry doesn't emit spurious emissions), but not as much.
The FCC doesn't care what kind of antenna you are using, as long as it passes the tests. I have never worked with a product that had a detachable antenna, but in that case, I imagine you would have to have something in your user instructions cautioning them not to attach an antenna with a gain more than such and such a limit.
Now in your case, the SIM800F module just has a pin that is to be connected to an antenna connector, or PCB or chip antenna. The datasheet for the SIM800F doesn't mention anything about FCC certification. So you will have to go through the more exhaustive testing.
One more note: if you have more than one radio in your device, such as cellular plus Bluetooth, or cellular plus BLE, etc. then all bets are off and you have to go through the full testing to make sure they play together nicely, whether the individual modules are pre-certified or not.