The "white stuff" is a fast setting adhesive used to provide mechanical location for heavier components. Leave it alone as much as possible. I've provided additional details and some suggested alternatives below - although odds are you will not be replacing it.
Replacing the bulging capacitors MAY provide a fix but they may have been damaged by some other failure. Voltage rating of the new caps should be the same or somewhat more than the existing ones.
Capacitance of the new caps should ideally be as before, but larger is unlikely to cause a problem.
What IS this white stuff anyway ????:
That's sometime affectionately known as "Gorilla Snot" (hereafter "GS"). It's a relatively fast setting adhesive that is INTENDED to provide mechanical strength and/or rigidity to parts that should have been given it by proper means such as brackets or support rings, BUT they decide that they can get away with doing it on the cheap with this material.
They can and they can't. It tends to fail long term in many cases when stressed. If it doesn't fail it probably wasn't needed.
Sometimes used instead and no less effective is "hot melt" glue, applied at moderate temperature and sets as it cools. Like GS it too almost always fails in the medium to long term. Usually in months, sometimes weeks. It seems that the glue (aka melted plastic) hardens with time and loses any keying it has to most surfaces. It can be used with some utility by allowing it to pass through a hole or slot and to accumulate a "knob" or bulge on the other side, so that WHEN it releases the knob is too large to pass through the hole or slot. Such an arrangement could provide reasonable retention although maintaining tight positional tolerances may be difficult.
In my experience, a superior solution is to use a neutral cure silicone rubber. Extensive advice is available from manufacturers on what grades to use with various materials and conditions. Primers are available for some harder to bond materials. In most cases I have found that silicone rubbers provide superb long term seals and bonds. 20+ year sealing and retention in outdoor plumbing type use is common.
An exception I found was in attempting to bond a FR4 PCB backed PV panel into an ABS well in a portable light housing. Over time the ABS/silicone-rubber join would debond, just as GS and hot-melt glue do. In that case it was possible to key the rubber into holes in the housing so that retention knobs were formed, as mentioned above.