0
\$\begingroup\$

In a textbook that I'm reading, it mentioned the following dynamical processes that arise in information theory: "compression, decompression, noise, encoding and decoding error-correcting codes".

I'm confused about how encoding and decoding are different processes from compression and decompression as they seem the same to me?

\$\endgroup\$
4
  • 2
    \$\begingroup\$ Encode and decode are just more general. Compression specifically is to reduce bandwidth. You might encode something for reasons other than reducing bandwidth. For example, the data cannot be transmitted reliably as is, or sometimes not at all. \$\endgroup\$
    – DKNguyen
    Commented Aug 23, 2021 at 1:19
  • 2
    \$\begingroup\$ it's like fruit and apples ... encoding is fruit ... compression is apples \$\endgroup\$
    – jsotola
    Commented Aug 23, 2021 at 1:22
  • 2
    \$\begingroup\$ Encoding is the process of applying codes to data to transform it into a new, coded form. Compression is a way to reduce the redundancy in data, hopefully making it smaller. You can use encoding to perform compression, or use codes for some other purpose (robustness, compatibility, etc). \$\endgroup\$ Commented Aug 23, 2021 at 1:33
  • \$\begingroup\$ encoding and decoding is converting from one representation to another and reverting this. E.g. binary data is mapped to an modulation scheme, or e.g. an 8b10b codec to make your data DC free and embedd an clock. compression and decompression is another kind of encoding and decoding, but here you replace e.g. your binary data with a space optimized representation which eliminates redundant information. For real world applications normally several encoding and decoding steps are combined, compression and decompression might be one of them \$\endgroup\$
    – schnedan
    Commented Aug 24, 2021 at 10:55

4 Answers 4

1
\$\begingroup\$

These terms are often used in the same sentence but have different meanings.

Encode could be analog, ( like QPSK) or digital like RLL codes or encrypted or encoded for error detection/correction codes.

Expansion/Compression could also be analog like FM which expands BW to improve SNR or reduce dynamic range with compression, or an ADC/DAC with logarithmic codec like A or u Law of a very large dynamic range into 8 bits . It could also be splitting the wide BW signal into many sub-bands each equalized compressed with high bits/baud but with low group delay distortion within the sub-band then aggregated and expanded to the original bit rate. This is generally how many modems work.

\$\endgroup\$
1
\$\begingroup\$

Coding can mean different things in different contexts, but in the usual structured breakdown of a communications system:

typical communications system

image from: Sklar paper

there are two types of coding you could be referring to.

  1. Source coding - this is where we make use of things we know about the source to modify the bitstream, typically to reduce redundant bits. This can be lossless compression, (e.g. zip file), or lossy compression - jpeg, mpeg, voice coders for cellular etc.
  2. Channel Coding - here the bitstream is modified to perform better over the intended channel - it could include FEC, interleaving (to counter burst errors) and so on.

The Sklar paper (and his book) is a good introduction.

\$\endgroup\$
1
\$\begingroup\$

In the context of discussing compression,

  • Compression analyzes a data set to identify redundant information for removal, then removes it
  • Encoding is the overall process, which results in a reduced data set. It can also refer to a specific process of entropy coding

For example, JPEG processes 8x8 blocks of pixels using the Discrete Cosine Transform (a kind of 2-dimensional frequency analysis), then quantizes the low-value components (the quantization being the ‘removal’ step, this is what makes JPEG 'lossy'.) The remaining information is then entropy coded using a scheme called Huffman coding that further reduces the data size, but without further loss.

In general use though, ‘compression’ and ‘encoding’ are often used interchangeably. We will speak of h.264 codecs or compressors, but understand that we mean an h.264 compliant block that uses a number of techniques to compress the data, followed by entropy coding.

We also speak of u-LAW or a-LAW encoding, which itself is a kind of compression (non-linear quantization) done in one step, but doesn’t use entropy coding.

On the other hand, entropy encoding, used by itself without loss, is also compression. LZW is a well-known example of lossless compression.

After the compression pipe, other types of encoding can be applied to the data to condition it for the medium it's targeted for. Example: DVD disc which uses MPEG-2, which is in turn encoded with Reed-Solomon CIRC for error correction, which is then encoded to 8-16 channel code and finally laid down as pits on the disc. All different steps, but only the first step, MPEG-2, is compression / entropy. You can also make a DVD full of GIFs which are losslessly coded using LZW.

The takeaway is, look at the steps in the compression (and decompression) pipeline to understand how they relate to each other and where they would apply. In general, you have data reduction (lossy or lossless) to get your compressed payload, followed by error correction (if needed) and and then channel coding; with decompression being the reverse of these steps.

\$\endgroup\$
1
  • \$\begingroup\$ Added some more to the 'pipeline' concept since OP is touching on that, using DVD as an example. \$\endgroup\$ Commented Aug 23, 2021 at 22:35
1
\$\begingroup\$

The final item of that list is 'encoding and decoding error correcting codes', all as one dynamical process. It sounds like you might be interpreting it such that encoding, decoding, and error correcting codes are each their own individual dynamical process. If not, it is good to say it explicitly just in case.

To answer your question however, compression and decompression would be total opposite of encoding and decoding error correction codes.

Compression in the context of information theory typically means entropy coding, which is also known as lossless compression.

Anything that is information has a certain amount of information entropy, which is exactly the absolute minimum number of bits needed to transmit that information over a channel without any loss. It is not possible for any form of lossless compression to reduce information below this number, unless the compression cheats and some part of the information is actually built into the algorithm (which had to be sent to the other side of the channel at some point if they are to decompress the information sent).

Compression (and decompression) are ways of reducing some number of bits to a value that is closer to its true information entropy. It is impossible to do this in a universal way, because for every file that a given algorithm is able to reduce the size of, there exists another sequence of bits that the same algorithm will end up increasing the size of, and by the same amount.

The trick is that some forms of compression are much more likely to give good results on certain kinds of information, and you just try to avoid using it on kinds of information it would make larger.

Ultimately, compression/decompression is removing redundancies from information so that information can be represented with only as many bits as it actually needs. Or at least, get it closer to that ideal number of bits.

Encoding and decoding error correction codes, on the other had, is the exact opposite: it intentionally adds redundancies to information, increasing the size. But it is done so such that the redundancies are no longer just wasteful, but instead directly protect the information we care about.

Just like compression, error correction is an entire field in itself with all sorts of algorithms and techniques.

Error correction allows information being transmitted over noisy channels where there is a certain probability of each bit sent being flipped. The correction codes permit both the detection and correction of a certain number of errors per good bits received.

Contrast this with an .xz or .zip archive, where just one bit out of place will render the entire archive corrupt, resulting in total loss of information.

See, they really are about as opposite as one can get. Despite that, they are often used together - first compress data to reduce the size, then encode error correction codes into that data as it is transmitted over a channel. This increases the efficiency while making it fault tolerant at the same time.

Error correction is why digital television works, why CDs work, why hard drive work, and so many other things. The math behind them is also equal parts awe-inspiring and humbling.

Information theory is one of the most interesting, useful, and often over-looked fields. I also highly recommend reading about Claude Shannon, he was a very interesting person and truly is the father of information theory (and digital circuit design theory, as if information theory wasn't already enough!).

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.