George and the NIE : the argument from information theory

George makes the following argument: let X be a binary random variable that equals 1 if Iran has an active nuclear weapons program and 0 if not. Suppose that last month we knew that P(X = 1) = p > 1/2. Then we can measure our uncertainty about X via its entropy:

H(X) = hb(p) = – p log p – (1-p) log (1-p)

Here hb(p) is the binary entropy function. Now let Y be a random variable representing the NIE. We know that conditioning reduces entropy:

H(X | Y) ≤ H(X)

Let p’ be our new probability that X = 1 conditioned on the evidence Y. We cannot have p’ < p, because then hb(p’) > hb(p), which is a contradiction. Therefore p’ ≥ p and therefore the NIE shows that the chance Iran has an active nuclear program is even higher than before.

Exercise: Explain the error(s) in George’s argument.

Extra Credit: Write a short essay explaining why one should not abuse information theory for political ends.


0 thoughts on “George and the NIE : the argument from information theory

  1. Why does conditioning reduce entropy? I can see that if the variable conditioned on is included in the distribution whose entropy is being measured, this would seem plausible, but in general this seems very implausible. (I’m ignorant of the relevant theorems here – is this the equivocation that’s going on in this argument?)

    Also, why would p’

  2. Hmmm. As you pointed out, the entropy decreases when p’ > p. However, the entropy can also decrease for some (but not all) values of p’ 0.5
    Choose p’

  3. Oh, oh. Looks like I should have typed the comment in html. Oh well. I hope the full (previous) comment shows up in your blog admin pages.

    (The previewer seemed to suggest that my entire comment was ok.)

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.