Oh, tube amplifiers and fuses. One of my favorite topics:
So, your 125mA fuse blows, on the secondary side of the power transformer I guess.
On the secondary side of the transformer, you'll deal with voltages around 400V. A fuse rated at 125mA will blow at around 50 Watt power-usage. So you don't need "just a small tube amp" to exceed this current.
As you've told your tube glows during turn-on and then goes back to normal operation. That's because you'll likely have a bunch of filter-capacitors in your amp that - on turn-on - draw more than 125mA while they charge up. Drawing 125mA and more is quite common even if the tubes of the power-amp are still on standby.
Back then in the 70th high voltage electrolytic capacitors had a very high "ESR" or "equivalent series resitance". In other words each capacitor always have a resistor in series. This limits the current running into the capacitor. 125mA for the fuse may have been a good value back then because of the ESR.
Nowadays capacitors have a much lower ESR, so during powering up the amp a much higher current will run through the fuse.
During normal operation (aka rocking out) you will on average never draw more current than what happens during turn-on time.
That may be the reason why your fuse lights up. The main culpit is, that the inrush, or turn-on, current is much higher than the average current you'll see while the amp is running.
There are two sane ways to solve this problem:
If you want to retrofit an old amp, add some kind of inrush current limiting device between the power-supply and the power-amp section. A beefy NTC resistor will do this okay, but there are more elegant solutions than that.
If you're designed the amp or build something based on a circuit from the 70th to 80th, consider ramping up the B+ voltage slowly. Your custumers, fuses and filter-caps will thank you.