I am working on a competition where we are required to detect partial discharges in given signals. Each signal represents a measurement of some power-line voltage.

I know that partial discharges appear as bursts of pulses at the peaks of the sinusoidal wave, but the given signals have a lot of noise. For example, these are two signals that are labeled to not have a partial discharge:

And these are two signals that are labeled to have a partial discharge:

I couldn't spot a pattern that distinguishes signals with partial discharge from other signals.

I used wavelet transform and a Butterworth filter on the signals to cancel some noise and to remove the sinusoidal shape so for example, the first signal above (which has no partial discharge) becomes like this:

and the last signal (which has partial discharge) becomes as below:

But I still can't find a pattern.

My question is: how to spot partial discharge in noisy signals like the given signals? Is there a signal-processing operation or calculation (e.g.: filter) that makes it easier to spot partial discharges?

  • 4
    \$\begingroup\$ Convolution if you can isolate and figure out what a discharge actually is? \$\endgroup\$ – DKNguyen Feb 19 '19 at 2:16
  • 1
    \$\begingroup\$ The first partial-discharge plot has a very ringy event at t=100,000; ;use that time-sequence as the correlation-template, as "Toor" suggests. \$\endgroup\$ – analogsystemsrf Feb 19 '19 at 2:54
  • 2
    \$\begingroup\$ What is the ADC sampling rate? \$\endgroup\$ – Tony Stewart Sunnyskyguy EE75 Feb 19 '19 at 3:21
  • 1
    \$\begingroup\$ @ElliotAlderson It is a Kaggle competition. The 1st place takes $12,000. I don't think I will win but I aim to have a good rank. And if you help me you don't win the competition, but you can participate in it on your own. \$\endgroup\$ – ammar Feb 19 '19 at 5:00
  • 3
    \$\begingroup\$ Too much math to describe on here and it's in many places. You basically have a reference waveform flip it in the time axis and step it through your incoming waveform from right to left doing some operations for every time step which produces a graph of the correlation between the two signals. It's basically searching for a waveform in another waveform. You flip the reference waveform in time so when the waveforms run past each other, the earliest points in both waveforms meet first, and the latest points meet last Look at the animation first: en.wikipedia.org/wiki/Convolution \$\endgroup\$ – DKNguyen Feb 19 '19 at 5:05

UPDATE Imagine this is a $1k competition to a $trillion global power infrastructure problem with ageing grid transformers and switchgear that can explode from dissolved combustible gasses from H2 to Acetylene that requires your data analysis skills.

Here you can see a unique impulse (PD) response of the system due to PD events in high voltageenter image description here

After all the denoising and event identification when you count repetition rate of events and map a histogram of phase coherence you can readily see it is somewhat stochastic (quasi-random with histogram peaks near 50and 230 deg and in this more on the negative cycle. (230deg)

enter image description here

Design Goal

  • Optimal Receiver design theory.

  • Take the spectrum of each data stream with an FFT

    • use the one without PD as the spectral shape for your Bandstop filter.
    • use the one with PD to compare the difference and use the difference as your Bandpass Filter.
  • choose a 10th order Bessel filter for bandstop and band pass separately.

    • compute what your spectral SNR becomes and see if it is > 30 dB for good error rates.

PD can be over 1GHz and even in the optical, UV and higher spectrum depending on the bandwidth of your sensors and how close to the PD activity it is. After all PD is the result of detonation of a tiny void of homogeneous dielectric that may have a slightly lower dielectric constant and possible a lower breakdown voltage BDV.


simulate this circuit – Schematic created using CircuitLab

There are hundreds of ways to detect PD and to do de-noising and thousands of research articles on the web to demonstrate this but the channel spectrum of "Noise" must be characterized 1st so that the PD signal can be extracted by some wavelet definitions with line phase-PD amplitude histograms.

But as I understand this is a statistical analysis competition, not a measurement competition. The data is poor here, which makes it a challenge.

Below is a PD event I captured on 33kV 5MVA transformer in a factory without power ON doing an offline Hipot PDIV test to the primary bushing using a wire loop around the Ground strap to a long coax going into a 50 Ohm load on the LeCroy scope.

The PDIV was below normal operating voltage so the unit was taken out of service in spite of perfect operational power study parameters but DGA indicated near 4% LEL after 6 mos service.


The peak voltage I captured was -10.8V @ 50 Ohms using 300MHz BW with a rise time < 1ns and PW50= 9 ns from a low power signal generator putting out 20kVac 60 Hz. THis was tested on a 5MVA transformer ramping up a HiPot test to PDIV. It failed below the normal operating volge of 33kVac 60Hz.

enter image description here

Normally PD clusters around the rising edge of each half sine where dV/dt is high before peak voltage but may have other patterns which indicate different failure modes. It may start triggering 1 pulse every few minutes which is the Inception Voltage (PDIV) and can rapidly increase to several pulses per half cycle like any relaxation unijunction oscillator due to several stress factors like heat, oil pump pressure and common mode delta voltage but the Root Cause is particle size contamination in the insulation that may be invisible to the naked eye.

This is Fourier spectrum of a pulse, and the primary spectrum (1st hump) goes up to Tr=0.35/BW with a null energy at f=/t(PW50) then harmonics replicate this pattern from rise time.
i.e. the sine wave that fits in the pulse width will have minimal energy but will have harmonics above and fundamental below.

You may only need to denoise the low frequency noise (< 10MHz?)and capture the spectrum above this.

enter image description here


What is a PD? It is a very tiny capacitance of contaminant that detonates in a vast and much larger dielectric medium. The event may be large in voltage but small in <<1 pF such that Q=CV is the charge being measured. The event is a very small duration (ps) of ionization in a dielectric medium but due to low pass filtering effects of the medium with inductive core and windings, the impulse imparts a lower frequency ringing unless nearby UHF antennae are used to capture the events. In Oil, the PD microburst breaks down some molecules of a large hydrocarbon chain \$H_xC_y\$ to create the gas with the lowest activation energy 1st Hydrogen, \$H_2\$. Then if the energy is larger it generates Methane and so on up into the pyrotechnic energy levels where it can generate Acetylene gas which has the highest activation energy and a precursor to grid transformer explosions.

This is why early PD detection is so vital

In dry insulation, it tends to convert gas void microbubbles or contaminants surrounded by epoxy, mica or other plastic insulation materials and creates a carbon chain which increasing spreads the energy until it fuses out or grows with age and activation levels.

But the initial event is like a abrupt pulse discharge but the measured signal passes thru the LC network of large core and grid inductance and insulating capacitance, so what is recorded is the LC network response of ringing, that depends on the geometry core inductance and dielectric capacitance where the underdamped resonance occurs. You could easily hear this on a SW radio nearby like lightning static with hum.

The equivalent charge is measured with calibrated PD equipment in nanoCoulombs based on the CV relative to standard test cap and standard impulse voltage.

The result is a group of varying resonant pulses that may vary widely in spectrum depending on the devive under test (DUT) and measurement sensor BW and DAQ data acquisition limitations. The best PD equipment has a spectrum > 3GHz. The waveform at the bottom of this answer used had an old 300MHz LeCroy scope that I had but I am sure that my sensor had a bandwidth 10x higher than this, and gave a nice 1st order response, unlike so many Rogowski coils that have a resonance from poor design. So in my case, the voltage was reduced by 10 and stretched out to 10x the rise time. The lack of resonance with 50 Ohm coax and termination, indicates a clean recording.

So the de-noising of PD using auto-correlation RMS averaging or real-time FFT is not effective but by characterizing the network with FFT you can design an Optimal Receiver as below where the maximum SNR is possible.

Here is a simulation of 50 Hz with a few handmade glitches up to the 256 harmonic of 50Hz up to 15kHz. Consider this as the line noise and look for PD Signal 2~10MHz. Just use a Bandstop filter and decide where to place it so that there is no group delay in the passband.

There is NO pattern !! except stochastic groupings. It would sound like a geiger counter because a large PD event in 1 location canbe a low voltage on the next cycle as the charge builds up and when the PDIV threshold exceeds results in a large impulse but not necessarily in the same phase from the same source.

Many PD event locations can exist like random lightning strokes so there may not be any recurring phase pulse but there may be a range of pulse repetition rates from various sources. Where lightning might be an atmospheric discharge that occurs every few seconds, Line induced PD is most stressful just after max dV/dt before peak voltage, but can still can occur later as the event is a particle being accelerated with enough kinetic energy to hit a conductor so there can be inertial effects in a fluid.

Events can occur from rain on the outside of bushings from dust and occur with dozens of spikes per cycle but since the particle mass and voltage creates a dynamic force since water has a very high dielectric constant the contamination tends to be very low and thus charges up more rapidly with an alternating electrostatic force.


enter image description here

Let's call the line frequency & load spectral noise from brushed motors and aggregate Triac powered devices as NOISE

Let's call your desired PD spectrum = SIGNAL.

You need to "de-noise" the data to maximize your SNR and reduce your probability of error detection. Let's choose a 30:1 SNR minimum for a target.

| improve this answer | |
  • 2
    \$\begingroup\$ @ Sunnyskyguy EE75 gotta love the details in this. Thank you. \$\endgroup\$ – analogsystemsrf Feb 19 '19 at 18:26
  • \$\begingroup\$ You posted a better answer than I did. You are showing in images what I wrote about in words (Energy density). I hope the OP can understand it. +1 \$\endgroup\$ – Sparky256 Feb 19 '19 at 23:04
  • \$\begingroup\$ I did not show any load energy noise nor compare with PD activity and you cannot tell from the OP's waveform by looking at the trace waveform or his unknown wavelet formula. The spike disturbance of load and PD looks similar at screen resolution since the 25 ns sampling interval does not distinguish spikes within a 20ms cycle on 1kpix image. It is the higher burst frequency of LC wavelet pulse that will hopefully be distinguished after we get a look at the FFT results or perhaps the PW50(ns) histogram results \$\endgroup\$ – Tony Stewart Sunnyskyguy EE75 Feb 19 '19 at 23:49
  • \$\begingroup\$ It is not "energy density" per se , rather it is unique spectral density bursts above the baseline noise. It can be even lower energy than the load disturbances but a result of impulse response from a micro detonation of insulation. \$\endgroup\$ – Tony Stewart Sunnyskyguy EE75 Feb 19 '19 at 23:55

The 'average' value of the noise with discharge is greater than the noise alone. Use a peak detector or sum the absolute value of the signal after the high-pass filter. Any 'average' filter must be of short duration, or it will miss the changes in noise density caused by the discharges.

A noise 'burst' as a discharge has more energy density for a given period of time than random noise alone, which has the same average energy dispersed over the same length of time. In fact random noise is fairly constant energy over enough time for it to average out to being invisible by RMS detection.

Unless the random noise is intense in both peak values and rate, it adds little energy to the signal, and a fine-tuned low pass filter could remove much of the out-of-band content, but in this case we want a high-pass filter first, to extract the random noise plus discharge noise.

Do not go by peak intensity, but by the ABS sum of peaks over a time equal to the discharge event. Though it is partially buried in random noise, noise that is NOT purely random will show up as an average increase in total energy density at a controlled rate, though the discharge event maybe partially randomized by software control.

If you are trying to capture and log these discharge events, record only events that have above average noise density per given time period. You will have to research to find out the average duration and rate of the discharge event, then use repeated sampling at this rate or a bit faster to capture most if not all discharge events.

| improve this answer | |
  • \$\begingroup\$ Denoising PD in a severe BW truncating data collection results in stochastic pings of LC resonance due to source impedance in a complex 3D model, so simple methods won't work rather some repetitive and many unique & common resonant bursts that may have no phase coherence yet tend to cluster slope before peaks. This is the challenge to find unique spectral noise signatures in channel of lower frequency load noise bursts, both contain some HF noise \$\endgroup\$ – Tony Stewart Sunnyskyguy EE75 Feb 19 '19 at 16:29
  • \$\begingroup\$ You don't want to take ABS value since PD correlation with phase is biased towards -ve voltage where -ve charged particles accelerate greater to grounded core than the conductors) due to exposed surface area of CRGOS core. See my plot #1 Correlation helps improve detection and identification of location of PD.or root cause (many) of insulation failure types. Also the noise burst is actually a resonant burst higher in frequency than distance loads. In UHF detectors it is just a 1st order UHF impulse response. Yet this data is all below Nyquist BW ~ 15MHz I assume with high order LPF 40Msps \$\endgroup\$ – Tony Stewart Sunnyskyguy EE75 Feb 19 '19 at 18:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.