This is probably a very basic question but I could not find the answer to it online. I just want to make sure - is the roughness caused by electromagnetic interference?
Roughness means some variations of signal:
1. High frequency additive oscillations (or noise). Try using much lower time/div to see them. Try to limit input bandwidth to suppress them.
2. Low frequency addidive oscillations or amplitude modulation. Try using much higher time/div. Also, try single-shot trigger mode to suppress multiple waveform accumulation on the screen.
Some of what you're seeing is undoubtedly due to simple ringing.
The simple fact is that a perfect square wave requires infinite frequency, which (obviously) isn't possible.
With a limited bandwidth, the signal will show some ringing, so (even at best) it looks something like this:
The red line shows the theoretical wave. The green shows up to the fifth harmonic of the fundamental, and the blue line up to the fifteenth harmonic.
Note how even in a simulated picture like this with no noise added at all, we still get what looks like a much thicker line at the peaks.
I must admit that I'm somewhat (extremely?) puzzled by some of the comments. Let's consider this:
Now, I'm the first to admit that this scope is fairly old (probably older than some of the people who wrote comments). I don't think it's being restricted to a mere 600 MHz bandwidth is quite enough to qualify as a "horrible oscilloscope" (you'd certainly think 600 MHz would be enough to show a 1 KHz signal fairly accurately). It's showing the 1 KHz square wave generated by the scope itself, so I don't think that's somehow started to produce output at 100 MHz or something a whole lot faster than 1 KHz. I'm using a direct coax connection from the generator to the input.
Does somebody here honestly believe that a 1 meter coax cable is going to let huge amounts of noise in to cause the peaks to be bright, even though the rise and fall are so dim they're nearly invisible? If so, how do they believe that anybody would ever stand any chance at all of measuring anything when using a normal scope probe, which inevitably has at least a little unshielded wiring involved?
For those who are convinced that you just can't see the overshoot and ringing without infinite bandwidth, let's take a look at a somewhat zoomed in view:
Now remember: this is a direct coax connection, so what we're seeing is only what's left when the signal has been terminated quite carefully (50 Ohm cable going to a 50 Ohm input). Even so, there's a little overshoot and ringing--under almost any normal circumstances, we'd expect to see (quite a bit) more.
Now, it is almost certainly true that with a phosphor display like this oscilloscope uses, part of the difference in brightness is due to the speed at which the electron beam is moving at a given time. We set a specific speed at which the beam traverses the display (e.g., 1 millisecond). When it's traveling (approximately) horizontally, it spends a relatively large amount of time lighting up the phosphor in a fairly small area. When it gets to the rising or falling part of the wave, it's still moving at the same speed horizontally, but it's also moving at relatively high speed vertically--thus, it's only spending a very small amount of time exciting the phosphor along those sections.
With a raster display, it's pretty much up to the people writing the software to decide whether to simulate this effect or not. I haven't looked at the software of the particular oscilloscope used by the OP, but it wouldn't strike me as terribly surprising if they decided to imitate what a conventional trace of the same signal would look like.
That does not, however, render what I said previously incorrect--that some of what you're seeing is from ringing in the circuit. No matter how careful you are to eliminate it, there's undoubtedly going to be at least some noise as well--but in the setup I've shown above, we see roughly the same effect in a signal that should be extremely low in noise--far below anything that could possibly account for the effect shown.