Let X be a binary input with 0 and 1 equally likely, so that H(X) = 1 bit. (a) Suppose the bit is subject to the bit-flip process described by Eq XXXXXXXXXXwith p = 0.1. Let Y be the output of this...

Let X be a binary input with 0 and 1 equally likely, so that H(X) = 1 bit.

(a) Suppose the bit is subject to the bit-flip process described by Eq. 20.45 with p = 0.1. Let Y be the output of this process. Calculate H(X : Y). By how much does this fall short of H(X)?

(b) Now suppose the bit is encoded into three bits according to Eq. 20.46, and that each of these bits is independently subject to Eq. 20.45. The output Y is the final value of the three output bits. Calculate H(X : Y) for p = 0.1.

Consider the three-bit code from Eq. 20.46. Let X be the input value (0 and 1 being equally likely) and Y be the output

(c) In each case, compare the actual probability of error with the bound on PE obtained from Fano’s inequality.