LOST CAUSES IN STATISTICS I: Finite Additivity

Normal Deviate 2013-07-06

I decided that I’ll write an occasional post about lost causes in statistics. (The title is motivated by Streater (2007).) Today’s post is about finitely additive probability (FAP).

Recall how we usually define probability. We start with a sample space ${S}$ and a ${\sigma}$ -algebra of events ${{\cal A}}$ . A real-valued function ${P}$ on ${{\cal A}}$ is a probability measure if it satisfies three axioms:

(A1) ${P(A) \geq 0}$ for each ${A\in {\cal A}}$ .

(A2) ${P(S)=1}$ .

(A3) If ${A_1,A_2,\ldots}$ is a sequence of disjoint events then

$\displaystyle P\Bigl(A_1 \bigcup A_2 \bigcup \cdots \Bigr) = \sum_{i=1}^\infty P(A_i).$

The third axiom, countable additivity, is rejected by some extremists. In particular, Bruno de Finetti was a vocal opponent of (A3). He insisted that probability should only be required to satisfy the additivity rule for finite unions. If ${P}$ is only required to satisfy the additivity rule for finite unions, we say it is a finitely additive probability measure.

Axioms cannot be right or wrong; they are working assumptions. But some assumptions are more useful than others. Countable additivity is undoubtedly useful. The entire edifice of modern probability theory is built on countable additivity. Denying (A3) is like saying we should only use rational numbers rather than real numbers.

Proponents of FAP argue that it can express concepts that cannot be expressed with countably additive probability. Consider the natural numbers

$\displaystyle S = \{1,2,3,\ldots, \}.$

There is no countably additive probability measure that puts equal probability on every element of ${S}$ . That is, there is no uniform probability on ${S}$ . But there are finitely additive probabilities that are uniform on ${S}$ . For example, you can construct a finitely additive probability ${P}$ for which ${P(\{i\})=0}$ for each ${i}$ . This does not contradict the fact that ${P(S)=1}$ unless you invoke (A3).

You can also decide to assign probability 1/2 to the even numbers and 1/2 to the odd numbers. Again this does not conflict with each integer having probability 0 as long as you do not insist on countable additivity.

These features are considered to be good things by fans of FAP. To me, these properties make it clear why finitely additive probability is a lost cause. You have to give up the ability to compute probabilities. With countably additive probability, we can assign mass ${p(s)}$ to each element ${s\in S}$ and then we can derive the probability of any event ${A}$ by addition:

$\displaystyle P(A) = \sum_{s\in A}p(s).$

This simple fact is what makes probability so useful. But for FAP, you cannot do this. You have to assign probabilities rather than calculate them.

In FAP, you also give up basic calculation devices such as the law of total probability: if ${B_1,B_2,\ldots,}$ is a disjoint partition then

$\displaystyle P(A) = \sum_j P(A|B_j) P(B_j).$

This formula is, in general, not true for FAP. Indeed, as my colleagues Mark Schervish, Teddy Seidenfeld and Jay Kadane showed, (see Schervish et al (1984)), every probability measure that is finitely but not countably additive, exhibits non-conglomerability. This means that there is an event ${E}$ and a countable partition ${B_1,B_2,\ldots,}$ such that ${P(E)}$ is not contained in the interval

$\displaystyle \Bigl[\inf_j P(E|B_j),\ \sup_j P(E|B_j)\Bigr].$

To me, all of this suggests that giving up countable additivity is a mistake. We lose some of the most useful and intuitive properties of probability.

For these reasons, I declare finitely additive probability theory to be a lost cause.

Other lost causes I will discuss in the future include fiducial inference and pure likelihood inference. I am tempted to put neural nets into the lost cause category although the recent work on deep learning suggests that may be hasty. Any suggestions?

References

Schervish, Mark, Seidenfeld, Teddy and Kadane, Joseph. (1984). The extent of non-conglomerability of finitely additive probabilities. Zeitschrift f\”{u}r Wahrscheinlichkeitstheorie und Verwandte Gebiete, Volume 66, pp 205-226.

Streater, R. (2007). Lost Causes in and Beyond Physics. Springer.