I have a PCIe reference clock generator chip, ASVMPHC-100.000MHZ-LR (datasheet), but it generates a sinusoidal waveform at 100 MHz with an amplitude of ~750 mV. Should I be running this through a NOT gate or something to generate a square wave before sending it to the PCIe slot? If so, what logic level should I use?
Your oscillator chip has HCSL outputs. You can't simply hook up one of the differential HCSL outputs to a high impedance oscilloscope probe and expect anything useful to come out. Both output pins are significant -- it's the difference in voltage between them that constitutes the clock signal. Looking at them in isolation is not very helpful here.
You need to properly terminate the outputs (look for HCSL section), and in order to look at it you need a differential probe. In a pinch you can use two 50 Ohm probes and use Math Subtract mode on a two channel 'scope. Your oscilloscope and probe combination must have at least 450MHz bandwidth for you to see anything that resembles a square wave.
Alas, something in your question seems very fishy: you'll need to use your 100MHz clock to clock your PCIe PHY chip, which will then generate the signal for the REFCLK+/- pair on the PCIe bus. You may need to buffer the output of the oscillator, depending on how many devices are attached to it, and whether the bare oscillator fulfills the jitter and transition time requirements of whatever it feeds into. The buffers have PLL loops that regenerate the clock, reduce jitter, etc. You should be looking at the PCIe specification and the datasheets of all the chips involved while doing all of this work. What is it that you're trying to accomplish?