I'm trying to add the MIPS command SRA to the following one cycle CPU: If I knew how many bits needed to be shifted I could easily replace the lower bits with 0's, but that's never a known till the instruction is being executed. What would be a way to implement this so it doesn't matter how far the bits need to be shifted?
Use a single cycle rotator + masking (for shift operation). The rotator is like a coupled 32:32 mux (32 parallel 32:1 muxes with the same select signals and inputs, except the inputs are shifted by 1-bit for each subsequent 32:1 mux). The masking unit is a 32-bit thermometer decoder.
If you want this to work in a single cycle you may need to spend some time at the rotator (since it depends on both data and selects). If you will not ever need the rotation (as opposed to just shifting), you can save a lot of time on the critical path by hard wiring the rotator's second (and any subsequent stages) stage LSB (which in a rotator scheme would be the MSB of the previous stage) to either '0' or '1'; whichever you wish to shift in at the LSB.
The critical path will almost certainly be through the selects to the rotator (since it must both reach the first stage for the data as well as fan out to 32 gates).
The masking unit (aka 32 bit thermometer decoder) can be implemented requiring only the select signals and can be as simple as an AND with each bit (before or after the rotator; the ideal timing depends on the arrival times of the selects vs. the data signals).
For example, see the barrel shifter which relies completely on combinational logic (and thus able to be completed in a single cycle, assuming the logic is fast enough).