Bayes, Theory, and Frequency

James Annan has another post on Bayesian inference, so, impressed by the underwhelming appreciation of my little joke, I will try a more explicit description of my current understanding of the issues. Here is James's example:

I have a max-min thermometer in my garden, which I reset each day. It measures the max and min, and let's assume that I know that the error on each day's measurement is well-characterised as a standard gaussian deviate - ie a sample from N(0,1). This morning I went out and saw that the min from the last night was -5C. What is the probability that the actual minimum was below -5C?

Most people will instinctively answer that the probability is 50%, reasoning that the error is just as likely to be positive as negative. Most people would be wrong. The probability depends on the prior distribution before you looked at the thermometer! This is an unavoidable consequence of how Bayes' Theorem works. The posterior is equal to the prior multiplied by the likelihood of the true temperature given the observation. Those who do not understand this will repeatedly get themselves into trouble when discussing probabilistic estimation.

To those who insist that 50% is the right answer, consider this situation: Say I know that the record low temperature ever recorded in my town (of which my garden is representative) is -4C. Will I still believe that there is a 50% chance that last night's temperature was below -5C? Of course not. Temperature records are virtually never broken by such a large margin. In this case, my prior probability density function has its mean (and the bulk of its support) above -5C, so even after measuring -5C, the posterior mean is above -5C. Even with a less extreme example - say the record low is -6C, but the typical February daily minimum temperature is -1C - a rational Bayesian will still (probably, based on the info supplied) conclude that the temperature was probably warmer than -5C.

OK, so James is applying Bayes Theorem.

Let us assume that we can consider some actual "real minimum temperature" t in his garden during the night, and lets call T the minimum temperature his thermometer measures. So what is the probability P(t|T) of t, given the measurement T? According to Bayes Theorem, it is given by:

P(t|T) = P(T|t)P(t)/Int(P(T|t')P(t'),t')

The ingredients of JA's deduction are the probability distribution P(T|t) for the measurement to be T, given that the real temperature is t, and the probability distribution P(t) of t. He has assumed that he knows P(T|t). He also knows that t = -4C was the previous record for low temperature and takes that as an indication that P(t=-5) is quite improbable.

Now, back to our dispute. I argued that when you can use Bayes theorem in this way, there is a theory and an at least implicit frequentist interpretation involved. What is P(t) but the frequentist interpretation of the local history? The theory comes from the idea that today's temperature resembles those of the past.

If we added another fact - say that all the surrounding communities had experienced record low tempertures that night, his story would change, and a different theory - approximate spatial continuity of temperature - would come into play, but it still look to me like there is a frequentist interpretation at bottom, and it doesn't look subjective to me.