In measuring relay contact resistance, should I use 4 wire measurement or calculate it using the voltage across the relay and current?
They are the same thing. A 4 wire resistance measurement measures the voltage across the unknown, while applying a current through a different set of contacts to the voltage measurement.
You can do this manually on the bench with a current power supply, voltmeter and ammeter, or you can use a '4 wire resistance measurement meter' which integrates those elements, and bundles 'ease of use' into it as well.
With an unpackaged relay, you could measure the voltage directly across the contacts, and get the pure contact resistance. If it's a packaged relay, the best you are going to be able to measure is the contact resistance plus the connections from the contacts to the closest you can get the voltage leads, the package terminals.