# Measurement of relay characteristics so can detect before failure

I am working with a very large safety system which was designed around 30 years ago. It has a system of relays interlocked together in a logic configuration so that it will failsafe if any relay itself fails. There are probably around 20000 relays in this system altogether, all on rack mounting cards.

The cards are in about 50 racks in groups of 20, spread out over a large space approximately 600m (2000ft) square. Access to some of the racks is difficult.

The relays are PCB mounted Panasonic S4-DC24V, to give an idea of the type.

The number of operations is low (est. less than 10 ops per day) and most of the time (est. > 95%) these relays are on. The reason for this is these cards are driven by door/hatch contacts and fixed (as in bolted and stay there for weeks) guarding.

The question- is there a parameter we could measure that could predict a failure? The obvious one is contact resistance, but the circuit prevents that (see note 1 below). An option is switching time, but I have not seen any reference to this as a predictor of relay life. Relay temperature I guess would not work either, as the contact load is low.

These cards are tested using ATE as part of a safety procedure for function. Failures are rare but are happening (see note 2), and can be a problem. You can assume replacing all the cards is not an option (see note 3), but gradual replacement based on a tested value is quite possible.

Just to add, it would make sense to have a cost/benefit analysis on cost of a breakdown versus cost of X cards and go from there. I don't have this information yet. If this affects what you suggest, you could assume a value range, i.e. 1 breakdown costs us the same as X new cards and specify X.

Note 1 - We have a ATE that checks the rack function with a few hundred tests taking about 5 minutes. In this rack, the statement is true. If we took the cards out and put them in a dedicated fixture, we can measure contact resistance, albeit resistance of 1 to 4 sets of contacts due to the routing in the circuit. If that is what we need to do, then I can put forward a case for it. I have also found a requirement that each rack should be tested every 12 months, but since the test rig was broken (my first job has been to fix it) I am not convinced this has been happening. To be fair, a new manager is concerned about restoring a correct system so that is why I am here.

Note 2 - I haven't got detailed info on the failure mode- it gets recorded as "the card does not work".

Note 3 - The facility is used 24/7 with short shutdown periods. The department is understaffed so things are reactive.

Note 4 - The system may be replaced or partially replaced in about 4 years, this means we are trying to avoid changing large amounts of it.

EDIT 2023-09-18 - 3 removed cards has been found. The test rig indicates they switch late (i.e. >200ms, but do switch), so they are not welded contacts. Welding is unlikely as the contacts are limited by fuses to <1A when the relays can switch much more that that.

• What failure modes are found? Commented Sep 14, 2023 at 18:40
• just curious: 20,000 relays in a "large safety system"! Are they all in the same location? Spread out in a factory? Commented Sep 14, 2023 at 18:43
• "Number of operations is low" .. once per hour? Once per day? Is it repetitive in any pattern? Commented Sep 14, 2023 at 18:53
• Any estimation of the failure rate? Of a population of 20,000, how many fail in a year? When a relay fails, what happens? Does something else simply take over (fully redundant) or does something get shut down (production line or whatever) for safety and you lose money? Commented Sep 15, 2023 at 9:49
• Are the relays in sockets? If a failure occurs, is it easy to know what the failure is and where to replace the card? I was wondering if it was possible to reduce MTTR (or cost) rather than think about MTBF (of current system). If MTTR is sufficiently small, no one cares. (Of course this might not be possible.) Another thought: could you stress test them? Ie, take out of service (run at 5 Hz, 3A, 60°C, vibration etc) and that might kill off the weak ones. I note datasheet says operations should be ~10^5 = 27 years @ 10 ops/day. Commented Sep 15, 2023 at 10:08

One useful technique is to measure absolutely everything, and then to see what correlates with failures (assuming that whoever is doing the maintenance logs stuff properly).

As a specific example, a difference in release time depending on the current flowing through the contacts might indicate that their tips are degrading and likely to weld.

• I like this as I use this in many situations. In this case, we don't have maintenance logs to look at. Release time is a good one, as it is in the relay datasheet with a max value. Commented Sep 15, 2023 at 9:53
• After a lot of discussions here it was decided to measure on and off time automatically in the existing test rig, add these to the report, and flag up any values at the limit of the range. So this is the closest answer. But jonathanjo has really helped a lot with the problem too. Thank you to all who replied to this problem. Commented Oct 2, 2023 at 7:47

The question- is there a parameter we could measure that could predict a failure?

Have you looked at the simplest: age.

I did some work on the statistics for a system with approx 1,500 pumps, each operating about 8 hours/day. (Different in that this system had negligible safety impact.) We had individual data on each pump; after something like 6 years we had a very good aging curve.

We're now on about the fourth wave of replacements, which we do at an age where the pumps start failing at a much faster rate. From memory we picked something like "point at which failure rate is 10x the base rate." We picked 10 because there was a knee there. Certainly it was arbitrary, but it was also pretty effective and cheap to do.

If your relay cards aren't all the same age, perhaps you pick some age at which you replace the oldest ones.

EDIT to add: the real problem with this is that "remaining lifetime under normal operating conditions" has a very high variance. So you need to swap out quite a lot to get the benefit. And in OP's case, failure propagation is bad in that one failure stops the whole system.

• This is a good answer and thank you for the context where you have used it. In this case we don't know the exact age of the cards (but we could guess from the IC data codes). Another factor is added as note 4 to the question. Commented Sep 15, 2023 at 9:38
• @REPuzzle Counting the number of operations and comparing that to the datasheet values would also be an indicator, but that can't be retrofitted, you can do some estimations based on the data you have, but not sure how useful that really is. And you have to know the circuit well enough to get the right point from the lifetime diagram. I post this as a comment because I think age is closely related to number of operations for a relay (specific to an application of course). Commented Sep 15, 2023 at 10:54
• @Arsenal Data sheet values might not help much: Pickering says "some [relatively uncommon] failures are related to the length of time that a relay is in operation (particularly when a relay is switching very low signal levels or when a relay is not operated very often and oxidation forms on the contacts)." also counting "is not a good indicator on its own. Load conditions alone can impact a relay's operating life by more than three orders of magnitude." Commented Sep 15, 2023 at 11:06

Usually relays fail because the contacts wear out. It's strictly a function of the stress on the contact (voltage, current, inductance being switched, inrush current etc.) and the number of operations. These are sealed type so they tend to contaminate themselves over time if they are called upon to switch much of anything. The seals are also not perfect and can leak external contaminants (the datasheet explicitly warns against silicone, so any conformal coating had best be something like acrylic).

So if those parameters are known perhaps you can preemptively swap out some of the relays for new ones. It sounds like you have a large enough sample to gather data even without detailed analysis, especially if the circuits are repetitive.

A more "high tech" approach would be to try to detect the arcing EMI signature when the contacts don't close completely. While perhaps an interesting project, success would not be assured and the odds would depend heavily on the details of the application.

• From what you are saying it sounds that contact resistance is the way to go, even if we need to go card by card. Maybe I should get an old card and measure it. That is a thought. Commented Sep 15, 2023 at 9:45

I would think you could do a continuity test to see if the relay is working and to also test if it's switching before use. AFIAK it is the contacts that fail, and the coils should remain intact if not stressed beyond their rated limits.

One thing that the datasheet says is the relays will develop NOx in a humid environment if there is arcing when the relays are switched, so I would try and keep the humidity low.

• Good point about the NOx, thank you for that. The humidity is controlled in this area, which is a good start. The ATE mentioned above effectively does the tests you mention as part of a complete rack test cycle. I added a note to it as a result of your answer so that was a good catch on your part. Commented Sep 15, 2023 at 9:41
• Apparently there is coil aging especially with temperature. Omron, p26, says "The life of a coil can be predicted if the temperature in the conditions that the coil is operated under is known. A total of 40,000 hours at 120°C is used as a reference point for most polyurethane copper wire coils" Commented Sep 15, 2023 at 10:40

You say that you can't measure contact resistance in the circuit (I suppose because it has to operate permanently?)

Do you know the current trough the contact (or are you able to measure it with a clamp probe on cables)? If so, you can simply measure the voltage across the relays, and deduce the resistance.

Or if you have cards you know to be good (or the same card you suppose to be OK for now), then you can just measure voltage and compare later on to detect variations.

NB : depending on the relays, and the current, voltage across the contacts might be quite low, so you might need a voltmeter with sub-mV resolution

• I have added a note on this, it is possible to measure contacts if individual cards are removed from racks, but the rack ATE will not allow in situ measurement. Current through the contact is 20mA as they switch the coils of 2 ST2-24VDC-F relays each. The data sheet for the card relay lists (up to) 6V @ 1A as the contact test current for a resistance of 50mOhm. So if can get a PSU a standard multimeter should do it. That is a good idea about comparison, especially as we have new value limits on the datasheet. Commented Sep 15, 2023 at 9:51

Another method used to check relay operation is to test operation/characteristics at less than their rated voltage.

A system with thousands of relays was used at a major airport and the various buildings had battery backup power supplies. These power systems were built with mains power supplies that trickle charged the batteries i.e. the batteries were just 'floating' in normal service. Catch was nobody was performing an adequate level of routine matenance on the batteries. When the the power supplies in one building were switched off the battery voltage in that building started to rapidly sag in minutes. Needless to say some marginal relay circuits failed as the voltage dropped. At that point the actaual panic was over the state of the batteries.

• That is an interesting angle. If a relay was failing as it could not switch on, for whatever reason, a lower voltage may cause that failure mode early. So if an automated supply was used, maybe it could cycle from a low voltage to see the lowest voltage the relay would switch at. Commented Sep 18, 2023 at 16:46