You can do a bit better than with a less-than-perfect pulse / step!\$\newcommand{\F}[1]{\mathcal{F}\left\{#1\right\}}\DeclareMathOperator{\rect}{rect}\DeclareMathOperator{\sinc}{sinc}\$
To put a bit of math to what Andy and a citizen wrote:
It's clear that your method of measuring the impulse response \$h\$ of a system works with an actual impulse, as the in/output relation of an LTI system is the convolution \$*\$ of the input with the output:
$$\delta(t)*h(t)=h(t)$$
Now, this works in frequency domain, too, because due to the properties of Fourier transform, the convolution of two signals in time domain is equivalent to multiplication of their Fourier transforms:
$$\F{\delta}(f)\F{h}(f) = 1(f)H(f)=H(f)$$
So, indeed, since the Fourier transform of the diract delta is a constant 1, you get the frequency response of the system when considering your measurements in frequency domain.
Now, let's go the route Andy proposes: use a rectangular pulse \$\rect_T(t)\$ with width \$T\$. In frequency domain we now get
$$\F{\rect_T}(f)\F{h}(f) = T\sinc(Tf)H(f),$$
and as you can see in Andy's excellent illustration, that is "good enough" if the part of the spectrum you care about is happening around the peak of the sinc in a narrow frequency window (lowpass region) that allows you to ignore the fact that the sinc, in fact, is anything but a the constant 1 that we got from the dirac impulse. Also, the longer the duration \$T\$ our rectangle, the narrower the frequency span in which our measurement really resembles the impulse response.
So, we want our rectangle to be as quickly OFF/ON/OFF as possible, right? That's going to be a bit of a problem:
- the shorter the rectangle, the less energy in it, the worse measurement and system noise are relative, and your estimate quality goes down
- Most systems are only linear for some limited input range, e.g. your system might just explode if you're trying to put through a physical pulse of 100 kV peak-to-peak, so increasing the amplitude of the rectangle to get more energy through is not an option (it also makes the rectangle harder to generate, which probably means it will be less narrow, back to the original problem)
Now, Realization:
What matters is not the "sharpness" of a pulse, what matters is how flat I can make it look in the spectrum. Sadly, since the pulse's time domain and the pulse's spectrum are linked, it seems these two things do go hand in hand. But, we can do post processing.
Imagine a signal \$s(t)\$ that has an autocorrelation function \$\phi_{ss}(\tau)=\delta(\tau)\$, i.e. it's perfectly white.
An autocorrelation function is just the (expected, but this is deterministic) convolution with itself's time inverse. So, what happens if we first measure the system, giving us \$g(t) = s(t)*h(t)\$, and then convolve the result with \$s'(t) = s(-t)\$?
\begin{align}
s'(t)*\left(s(t)*h(t)\right) &= \underbrace{s'(t)*s(t)}_{=\delta(t)}*h(t)\\
\F{s'(t)*\left(s(t)*h(t)\right)} &= \F{{s'*s}*h}\\
&= \F{{s'*s}}\F{h}\\
&= \F{{\delta}}\F{h}\\
&= 1H(f)\\
&=H(f)
\end{align}
So, the trick is to use a wideband signal \$s(t)\$, which occupies all the spectrum that you care about, and put that through your system, and then correlate the thing against the input signal. Of course, since your signal is finitely long, and can also not be infinitely wideband, you never actually get the \$\delta\$ autocorrelation, but for most systems (and physical things that excite systems), it's much easier to excite with something whose amplitude doesn't have to change incredibly much in very short time.
Note how you can make \$s(t)\$ arbitrarily long, solving the signal-to-noise-ratio problem that we had with the rectangle.
While there are analog ways of generating a white-as-possible signal and correlating with it, it's rarely done in analog domain: generation of white sequences in a computer is nothing but using an appropriate random number generator to compute a couple samples (or using known special sequences), and correlation/convolution is a simple algorithm.