Let X denote an ensemble corresponding to a random variable

Let X denote an ensemble corresponding to a random variable with outcomes Ax = {0, 1} occurring with probabilities pX = (0.2, 0.8). (a) (2 pt) Compute the entropy H(X). Provide an upper bound on log2 |TN, beta,(X)| for N = 100 and beta = 0.2. Give an example of a sequence x E {0, 1}^400 such that x not E TN,beta(X) for N = 400 and beta = 0.2. (Hint: One such sequence has a very simple structure that you can specify succinctly. If you don\'t see it, you can try looking at candidate atypical sequences for N = 4 and N = 8.) Suppose X is an ensemble as per the previous question. Recall that X^4 denotes an extended ensemble of X. (2 pt) Give an example of a uniform, lossless coding of X^4. (2 pt) Give an example of a uniform, lossy coding of X^4. (2 pt) What is the raw bit content of X^4? (2 pt) How many unique values does H delta (X^4) attain as delta is varied in [0, 1]? (4 pt) What is the smallest value of delta such that H delta (X^4)

Solution

Chegg\'s policy allow me to answer 1 question per post. I can gladly help you but you should post it in a new question.

5)

X H(X)

0 0.2

1 0.8