How can error correction codes reduce bit error rate, for same amount of energy

Question

I am selfstudying error correction block codes and has a confusion about their performance.

Suppose we have \$r\$ information bits and we make \$k\$ coding bits in addition. I am given to understand that, for the same amount of energy per information bit, the coded system performs better compared to an uncoded system, where the channel is additive white Gaussian noise (AWGN) and modulation is BPSK.

This is demonstrated in a simulation, where the bit error rate (BER) is plotted against energy per information bit.

My question is, how does this come to be? When we code the \$r\$ bits, we in fact spread the energy over \$r+k\$ bits. Where as in uncoded case all the energy is used on \$r\$ bits alone. On the other hand the coded bit stream has \$r+k\$ dimensions whereas uncoded steam has \$r\$ dimensions.

Can someone please explain, mathematically, where the BER improvement comes from?

Answer 1

Think about the probability of an uncorrectable error occurring in each message. If you just send r bits, all errors are uncorrectable, so the chance of a successfully sent message is $$(1-P(bitError))^r$$ (in other words, all bits have to pass through with no error.) Now, say you spread the energy just a little to add k error correction bits, such that you can correct a single bit error. Then the chance of a single bit error does go up due to the weaker signal, but the chance of a successfully sent message becomes something like $$(1-P(higherBitError))^{r+k} + (r+k)(P(higherBitError)(1-P(higherBitError))^{r+k-1})$$ (In other words, it's successful if all bits pass through with no error as before. But it's also successful in the (r+k) possible situations where 1 bit has an error and the other r+k-1 bits don't.)

I'm not 100% sure my probability formulation is exactly correct, but you can see the idea: there are now more possible situations where you win. Depending on the values of r, k and how much the bit error probability increases, the error correction will often result in a higher probability of a successfully sent message.

Answer 2

When transmitting over a noisy channel, the mutual information between the input and the output depends on the chosen input ensemble[1]. One can choose an input ensemble to maximize this mutual information. When one has done so, the resulting maximum mutual information is known as the capacity of the channel. [2] The transformation that maps high-entropy data to that optimal input distribution is the error correcting code.

[1] An ensemble is an "alphabet" of possible input symbols along with associated probabilities (which do not have to be uniform)

[2] In the case of an Additive Gaussian White Noise channel, this capacity is given by the Shannon–Hartley theorem.