Coding Theory

Tags: speech-processing

Find how much see page 9, information is in a signal.

Information that's relevent to coding theory is:

**Entropy** Information in a random variable (amount of information in a signal). \\[ H(x) = - \sum\_{x \in X} p(x){\log p(x)} \\] **Mutual Information** Amount of information that's common in two random variables. \\[ I(X;Y) = \sum\_{x, y} p(x, y){\log \frac{p(x,y)}{p(x) \times p(y)}} \\]

Entropy indicates ease of compression (due to redundancy). Mutual Information is used to find communication rate through a channel.

The amount of information needed to [see page 12, encode] a signal is defined by:

Digital signals are thus characterised by their data-rate (bits per second)

We can express essentially the same signal using [see page 12, less information] with lossy compression. Digital Speech Codecs use lossy schemes to do this by exploiting the source filter model (eg. removing high frequency sounds we can't even hear).

Digital Speech Coding

The information rate in speech is only around 100 bps, however we transmit speech using k/bps vocoders. The way to code signals at lower rates is to exploit redundancies in the signal, which for speech is done using predictive models.

The ultimate predictive model for speech is speech-recognition + speech-synthesis. Essentially: Convert the speech to text, store text, convert back to speech on read.