Zero Dark Thirty and Bayes’ theorem

A moviegoing colleague writes:

I just watched the movie Zero Dark Thirty about the hunt for Osama Bin Laden. What struck me about it was: (1) Bayes theorem underlies the whole movie; (2) CIA top brass do not know Bayes theorem (at least as portrayed in the movie).

Obviously one does not need to know physics to play billiards, but it helps with the reasoning.

Essentially, at some point the key CIA agent locates what she strongly believes is OBL’s hidding place in Pakistan. Then it takes the White House some 150 days to make the decision to attack the compound. Why so long? And why, even on the eve of the operation, were senior brass only some 60% OBL was there?

Fear of false positives is the answer. After all, the compound could belong to a drug lord, or some other terrorist. Here is the math:

There are two possibilities, according to movie: OBL is in a compound (C) in a city or he is in the mountains in tribal regions. Say P(OBL in C) = 0.5.

A diagnosis is made on the compound on the basis of X criteria (e.g. high walls, nobody enters or leaves except courier, etc.. ).

Say the “compound test” is positive (+) when all criteria are met and negative otherwise. The CIA agent is very sure this is OBL’s compound. In particular, say P(+|OBL in C)=0.9.

This would seem like a “slam dunk” but what if say P(+|OBL not in C)=0.6. Then, by Bayes Theorem, P(OB in C|+) is only 60%, which is what the senior brass thinks on the eve of the operation.

One can play around with the parameters, but I draw three observations:

First, I don’t know if this was the right assessment or not but what I gather from the discussion — as portrayed in the movie — is that top brass did not focus explicitly on debating the arguments to the Bayes formula. The evidence was not structured, but casually spoken in hunches and guesses.

Second, it seems the debacle on WMD in Iraq has made the CIA more worried of false positives!

Third, given what the film portrays about the compound, I think it belonging to a drug lord was pretty minimal, so I would say P(+|OBL not in C)=0.1. If so P(OB in C|+) is 90%. This suggests the White House was extremely risk averse: it seemed very likely they had OBL and not somebody else yet they sat on it for half a year (at substantial risk to homeland)!!!! (Alternatively, the fear of false positives was getting in the way of the top brass judgement (e.g. maybe they thought it was 90% but did not want to say it)).