In a nut shell, quality, quality, quality.
The first thing you do is to use high reliablity parts. NASA specifies 4 quality levels starting with commercial (the lowest grade), moving to '883B (a mil standard); then QML level Q, and finally QML level V. With each step up in level, the screening requirements become more stringent; the paper trail more onerous; and the cost ever increasing.
With increasing quality levels comes lower predicted failure rates. This means that when you do your reliability prediction (or more accurately, your probability of mission success), your Ps increases with better quality parts.
Adequate derating also plays into this, particularly with new technology or new parts for which there is no history. We are sometimes told to use a 100 V MOSFET for a 20V application because of this.
Redundancy helps a lot. But with redundancy comes added complexity and more parts, which actually degrades the serial failure rate.
With any hi-rel design, you need to do an analysis to identify and mitigate, to the extent possible, any single point failures (SPfs). An SPF is a failure that would degrade or cause the loss of the entire function, or mission. SPF analysis is particularly important when redundancy is employed because you do not want a single failure to cause both the primary and redundant set of hardware to not work.
Finally, on those Voyager missions, I'll bet they were designed for an 8 or 10 year mission life, not 40.
While you cannot test your way into a highly reliable system, testing plays a big part in weeding out marginal parts. All of our assemblies go through some type of environmental stress screening, which includes functional testing over the expected temperature range, and temperature cycling, both powered and un-powered. Systems destined for space go through testing in a thermal vacuum (TVAC) chamber. There also may be vibration or shock testing, but these are usually done on a test article.
EDIT 2 8/6/2020 - Added blurb on temperature swings
Several who have responded to this question mentioned temperature and its effects on reliability. So I thought I would expound on this a bit more.
Semiconductors exhibit a failure rate that approximately doubles for every 10 deg C increase in temperature. There are papers out there arguing whether 2X is the right value; that maybe it should be 1.8, or 2.5, or some other amount. But for purposes of this discussion I’ll use 2X as it is a value that’s “accepted” by industry, the government, and the reliability disciplines.
With that out of the way, it makes sense that, from a reliability standpoint, you want to keep your electronics as cool as possible. 85 deg C operating temperature is better than 95 deg C, and 75 deg C is better than 85 deg C.
But in addition to the operating temperature, be it average or peak, there is also the temperature swing, or variation. Temperature swings are bad from a reliability standpoint in that it is temperature changes that stress interconnects, particularly those involving IC’s or even discrete semiconductors. These temperature changes induce a stress on the interconnect between the component and the board due to the differences in Coefficients of Thermal Expansion (CTEs) between the component and the board. For example a typical FR4 PCB has a CTE of ~15 ppm, while a BGA package might have a CTE closer to 6 ppm. These differences in CTE cause a stress to be exerted on the solder joints that attaches the part to the board as the temperature changes. These stresses are proportional to the changes in temperature and the size of the package and over time, given enough temperature cycles, can lead to a fracture of the solder joint or attachment to the board.
Leaded parts, such as the old 14/16/20 pin flat packs are much more forgiving in this environment than are rigidly attached packages such as Ball Grid Arrays (BGAs) because the leads of the former provide a significant amount of compliance that reduces the stress on the solder joint.
The reason for bringing all this up is that what we usually care about is the reliability of the system as a whole, or more properly the Probability of Mission Success (Ps) of the system. Because of how temperature changes and the average operating temperature affect various aspects of the system’s reliability, it may turn out that it’s better to operate a system at a constant higher temperature (say 85 deg C) as opposed to letting the temperature swing from 10 deg C to 70 deg C on a regular basis.