I have finished writing a project with separate VHDL files. Most of the components have connections directly to the FPGA ( which I have instantiated and connected in the top level file) but a few do not. to connect these "internal" components appropriately can I instantiate them in the top level file and connect them only to signals?(like one would in a test bench) or do I have to instantiate each entity where needed in different low level vhd files?
Anything you can put in the other files you can also put in the top level file.
So there is no reason you can't instantiate any components/entities you want in the top level.
Note that the synthesizer will trim out any logic that doesn't somehow connect to a port. Because if it doesn't touch a port then it doesn't do anything in the real world. Note that the connection to the port may be via intermediate signals, through other components, or through other logic. It doesn't matter, so long as it somehow connects.
You can certainly connect an instantiated component only to internal signals. As long as those signals go somewhere else also.