Source coding with 2 distinct distributions and entropies

[wu_weidong]

I'm learning about source coding, and many of the books/resources I've read give examples of the source Xⁿ being defined as a sequence of iid random variables. How about when the sequence is independent but belong to 2 distinct distributions (e.g. P_x when X_i is odd, and P_x' when X_i is even), with 2 distinct entropies (H(X) and H(X'))?

Where can I find resources that talk about such problems, including how to define the typical set in such cases?

One more question - when I'm asked to "provide a coding scheme" or "design a coding scheme", what exactly am I expected to give?

 

[Future Bruno]

In this case, when the sequence is composed of two distinct distributions, you will need to provide a coding scheme that takes into account the two distinct entropies. This can be done by using a combination of entropy coding and arithmetic coding. Entropy coding is a technique which uses the idea of Huffman coding to achieve higher compression, while arithmetic coding is a more advanced technique that allows for better compression by exploiting the correlations between symbols in a sequence. When you are asked to provide a coding scheme or design a coding scheme, you will usually be expected to provide an algorithm that takes a given input and produces a compressed output. This algorithm should include the details of how the input is encoded and decoded, as well as the data structures used to store the resulting compressed data. It should also take into account the two distinct entropies of the source Xn in order to achieve the highest possible compression.