I am going to make some assumptions here about the clocks in your measurement equipment being much better than both tcxos, otherwise it will be hard to tell which of those is causing deviations. Just for a simple illustration, say you have two unknown oscillators that should be 10MHz, but one is really off my much. You use one for your frequency counters reference, and measure the other. In one case the counter says 9.9MHz, in the other 10.1MHz. Not much gained here. Always keep this and similar issues in mind.
For a more in depth look, google up some guidelines about oscillator measurements, applications notes AN10007 and AN10033 from sitime seem to contain some useful tips for measurement setups. Also informing yourself about allan variance might be useful.
You always need to keep in mind what the requirements for your product are. Measuring one to deviate by 2.1ppm and the other by 1.8ppm doesn't mean much if your application is fine with anything under 10ppm.
For long term stability tests, you need something to compare to. Either you have a high end ocxo in one of your frequency counters, or you pick your best tcxo and discipline it by gps. Both should have very good mid-term stabilities, so you can then run your frequency counter with those as references, pick some high enough gate time, and collect the data of your frequency counter for long enough so you are confident that it will be meaningful for your application, then compare those for the new and old part.
For short term stability (which blurs over to jitter) you need to decide which kind of instability you are interested in, and there are tons of ways to measure jitter and similar, that all depend on your equipment abilities.
You could configure a spectrum analyzer for a rather long sweep time around the frequency you are interested in and compare the results of the tcxos. Phase noise will be visible as side bands here (if your analyzers LO is good enough). If your scope is good enough you can use its abilities to measure a certain amount of clock cycles and build a histogram out of that and compare both (some scopes have even a feature for histograms). Sometimes just using the infinite persistance of a scope can tell you something about the jitter.
Additionally it might make sense to compare the output waveforms and if they are usable or at the edge case of what your circuit can handle. A lot of cheapos on the market have rather weird clipped sine waves.
So since the sole responsibility of an oscillator is to generate a certain waveform (usually sine) with a precise frequency, there is not much more to measure here: Deviation from that intended waveform, as well as deviation from the frequency.
Of course, since you mentioned it, if your product requirements are such that it has to work within certain temperatures, make use of the temperature chamber, but only after you are confident in your measurement setup on the bench, and your abilities to read the data acquired.