I am guessing that you have found what appears to be a serial port on a router and you are assuming that the router is running Linux and that the serial port is the console to the Linux OS? And the fact that this is a router has nothing to do with listening in on the console serial port. That this could be a Linux media box for the purposes of this question.
What might be done is to probe each pin with a scope. The Ground pin will be 0 volts and the TX (with respect to the "router") will likely have activity on it each time you apply power to the "router". The RX (with respect to the "router") may also appear to have 0 volts. To differentiate between RX and Ground you can use a large resistor (perhaps 1Kohms) connected to the positive voltage rail. When this and the scope is applied to the ground pin you should see nothing but 0 volts. But when this and the scope is applied to the RX pin you should see the voltage of the positive rail.
Take care not to short any pins directly to ground or the positive rail when testing.
(Added later after being asked if a multimeter instead of a scope could be used:)
If you do not have an scope you can do these tests using a multimeter.
Consider that both he multimeter's input and likely the RX input are both high impedance. So can be influenced easily by a week current. As is the case with an approximately 1000ohm resistor connected to the positive rail.
However, the ground pin, being directly connected to ground is low impedance. And can not be influenced by the resistor.
Both these cases can be detected by the multimeter. When the multimeter and resistor are connected at the RX pin, the multimeter will read a continuous positive voltage. When the multimeter and resistor are connected to the ground pin, the multimeter will read 0 volts.
The TX pin is a bit tricky if only using a multimeter. It is being driven and should not be influenced by the resistor. However we do not know if it is being driven high or low at any given moment. The best we can do here is assume it is changing. So we will look for a difference in average voltage. We expect the TX pin to be active on power up. So we center or testing on this pin after applying power to the board.
We can use the multimeter to perform and additional test to verify our findings. By measuring the impedance to ground and the positive rail when the power if off, we can make further inferences as to the function of the pins.
The easiest pin to identify using impedance testing is ground. It should have dozens to thousands of ohms resistance to the positive rail. But no resistance to ground.
The RX pin should have dozen to thousands of ohms resistance to both the positive rail and ground. But we are already fairly sure about this pin from the above multimeter resistor test.
The TX pin will again be difficult. It will likely have resistance like the RX pin.
Now consider both multimeter tests. It as easy to find the RX pin because we could influence it with a week current. But it may or may not have been easy to tell the difference between the ground and TX pins. Especially if the TX pin was actively low most of the time. However, it was easy to find the ground pin by measuring it had zero impedance to ground. That leaves the last pin which we can assume is the TX pin.