To rephrase, if only one port on the splitter is used, is the signal drop to that one port the same as when all four ports are used?
You should not be using it that way! This splitter/filter has no in put for power so it is a completely passive device meaning, the power which comes out must be smaller than the power which goes in. Ideally all signal power that feeds into it should get out but that's not going to happen, there will always be some signal loss, no matter what.
It is also designed to only split that power properly (as intended) when all outputs have a connection to a device with an antenna input.
If you leave one or more outputs open then the signal will reflect (as it cannot go anywhere, it goes back) and that disturbs the signal on all outputs. This doesn't cause any damage as the signals are very low in power. But you might suffer from ghost images on a TV or general bad reception, so be aware of that! But as a temporary solution until you connect everything properly, it is OK.
Is it the splitter itself which diminishes the signal, or is it the extra devices attached to it drawing part of the voltage away?
As I mention above, the splitter cannot amplify the signal so what power comes in is devided between all outputs. "Drawing voltage away" does not happen and is not the issue. The splitter must be connected properly and only then will it divide the signal equally.
So connecting only one port to a TV and leaving the rest open will actually result in the worst signal at the TV. You get the best signal with no splitter (cable straight into TV). Using the splitter properly (all ports in use) is the middle solution, you would get a decent signal everywhere provided that the signal going into the splitter is strong enough.
Sidenote 1:
There's the issue of characteristic impedance.
My bet is that the splitter you choose is designed for 75 ohms systems like TVs radios and cable modems. My bet is also that your 4G amplifier has a 50 ohms system as that is what is commonly used for cell phones and nearly anything related to it. The fact that the splitter does not mention 50 Ohms or 75 Ohms worries me.
Like using the splitter in the wrong way, the 50 Ohms vs 75 Ohms will not damage anything physically but it will harm the signals such that things are unlikely to work like you want. My bet is that the money you're spending on this will be wasted. You should stick with the products that the 4G extender's manufacturer sells as that is designed to work together. If they don't sell what you need then you're out of luck.
Sidenote 2:
Do realize that what you're trying to do might not work (properly). You have to be sure that the 4G (?) signals you're trying to distribute might not be in the supported frequency range of this splitter. It would be helpful to know which frequency band you are using. I know that 850 MHz is a band used for 4 G but also 1700 MHz up to 1900 MHz is used. And even though the splitter optimistically mentions 2500 MHz, in practice these cheap devices have quite poor performance at such a high frequencies.
Then there's the cables you will be using. Even at 800 MHz a decent cable will attenuate the signal, at 1700 MHz and higher signal losses can be very bad. Use the best quality (lowest loss) cable you can afford.
So if you're using 4G at 850 MHz, then what you intend to do might work. At the higher frequency bands maybe it will not work so well or not work at all.
