Question

A detective agency is looking to bust an arms sale. According to available intelligence, the sale is likely to happen in one of the three locations A, B, or C. Historical patterns indicate that P(A) (the likelihood that the sale will happen at A) is twice that of P(B), and likewise, P(B) = 2P(C). What is the probability distribution of the arms sale happening in these 3 places? Let us denote the location of the sale with a memoryless source X, i.e., X takes values ‘A’, ’B’, ’C’, with probabilities P(A), P(B), and P(C) respectively. Calculate it’s entropy H(X) in bits. Note that H(X) represents the amount of uncertainty in the location of the sale. The director of the agency is not very happy with the level of uncertainty. So he looks up recent covert transmissions for hints. He finds a recent coded message from agent Atomic Blonde which indicates that the sale will not happen at C. Let us denote the message from agent Blonde as Y , and in this case Y ="not C". Given this information, what is the new uncertainty H(X|Y = "not C")?

Answer 1

Answer:

Probability Distribution={(A, 4/7), (B, 2/7), (C, 1/7)}

H(X)=5.4224 bits per symb

H(X|Y="not C")=0.54902 bits per symb

Explanation:

P(B)=2P(C)

P(A)=2P(B)

But

P(A)+P(B)+P(C)=1

4P(C)+2P(C)+P(C)=1

P(C)=1/7

Then

P(A)=4/7

P(B)=2/7

Probability Distribution={(A, 4/7), (B, 2/7), (C, 1/7)}

iii

If X={A,B,C}

and P(Xi)={4/7,2/7,1/7}

where  Id =logarithm to base  2

Entropy, H(X)=-{P(A) Id P(A) +P(B) Id P(B) + P(C) Id P(C)}

=-{(1/7)Id1/7 +(2/7)Id(2/7) +(4/7)Id(4/7)}

=5.4224 bits  per symb

if P(C)  =0

P(A)=2P(B)

P(B)=1/3

P(A)=2/3

H(X|Y="not C")= -(1/3)Id(I/3) -(2/3)Id(2/3)

=0.54902 bits per symb