George makes the following argument: let *X* be a binary random variable that equals 1 if Iran has an active nuclear weapons program and *0* if not. Suppose that last month we knew that *P(X = 1) = p > 1/2*. Then we can measure our uncertainty about *X* via its entropy:

H(X) = h

_{b}(p) = – p log p – (1-p) log (1-p)

Here *h _{b}(p)* is the binary entropy function. Now let

*Y*be a random variable representing the NIE. We know that conditioning reduces entropy:

H(X | Y) ≤ H(X)

Let *p’* be our new probability that *X = 1* conditioned on the evidence *Y*. We cannot have *p’ < p*, because then *h _{b}(p’) > h_{b}(p)*, which is a contradiction. Therefore

*p’ ≥ p*and therefore the NIE shows that the chance Iran has an active nuclear program is even higher than before.

**Exercise:** Explain the error(s) in George’s argument.

**Extra Credit:** Write a short essay explaining why one should not abuse information theory for political ends.

Why does conditioning reduce entropy? I can see that if the variable conditioned on is included in the distribution whose entropy is being measured, this would seem plausible, but in general this seems very implausible. (I’m ignorant of the relevant theorems here – is this the equivocation that’s going on in this argument?)

Also, why would p’

Oops – I should have used the properly escaped html commands rather than the angle brackets – can you replace each < with < ? Or should i just re-post?

Hmmm. As you pointed out, the entropy decreases when p’ > p. However, the entropy can also decrease for some (but not all) values of p’ 0.5

Choose p’

Oh, oh. Looks like I should have typed the comment in html. Oh well. I hope the full (previous) comment shows up in your blog admin pages.

(The previewer seemed to suggest that my entire comment was ok.)