This is normal behaviour for a guitar amp, which is why there is an accepted cabling/switching/volume order when setting up.
If you have no other earthing/grounding issues, then plugging just a guitar lead in without attached guitar just turns the whole thing into a big crude arial. It will predominantly hum at about G♯ in EU or B♭ in the US [ 50Hz vs 60Hz mains].
You can make it far worse by adding yourself to this arial by touching the tip of the jack, or slightly improve it by touching the shield or body. Anything else you touch will add to the amount of mains hum being picked up by your new crude arial*. Plugging in a cable-less jack just makes less of an arial.
Plugging a guitar in completes the circuit so you [in theory] no longer just have a big, dumb arial; unless your guitar is poorly shielded. The usual trick is to plug the cable into the guitar first, then into the amp. It's also wise to turn the amp down first, or you get bangs & thumps as different parts of the plug contact all the wrong places before it finally settles into its 'click' locator.
Guitar amps are not actually the finest pieces of delicate electronics, unlike hifi or scientific equipment. They are designed to colour sound in weird & wonderful ways. Making them silent in operation is not high on the priority list, so long as they are reasonably quiet once fully connected. You might have noticed they can still make a lot of background noise anyway, once you're up at gig volume with a distorted sound.
*There's a whole slew of 'fixes' for hum, interference, ground loops/ground lifts, induced RFI… a lot of which borders on voodoo, so I'm not going to go anywhere near it.
A lot of these people here are experts in electronics… I'm just a guitarist of 50 years' experience, who learned a lot of this through simple empiricism. [I also these days use a totally hum-free guitar, based on transducers not magnetic pickups.. but that's a whole other story.]