Bayes's Theorem for Calculating Inverse Probabilities

On April 7, 1761 Thomas Bayes, an English clergyman and mathematician, died at the age of 59. Two years after his death, his paper, entitled "An Essay Towards Solving a Problem in the Doctrine of Chances" Thomas Bayes was published in the Philosophical Transactions of the Royal Society 53 (1763) 370-418. Bayes's paper enunciated Bayes's Theorem for calculating "inverse probabilities”—the basis for methods of extracting patterns from data in decision analysis, data mining, statistical learning machines, Bayesian networks, Bayesian inference.

"Whereas the ordinary rules of probability address such problems as 'what is the probability of drawing a yellow marble, if you draw three marbles from a sack containing 10 yellow marbles and 90 white marbles,' a Bayesian might ask the question, 'if I draw five marbles from a sack, and one is yellow and four are white, what is the probable distribution of the marbles in the sack?'  The advantage of inverse probability is that predictions can be continually refined as experience accumulates, so that if you draw five more marbles, and they are all white, that will change the probability prediction (and drawing a blue marble would drastically alter the situation), but Bayes’ theorem can easily accommodate any and all new information.  Bayes wrote his classic paper, 'An Essay towards solving a Problem in the Doctrine of Chances,' sometime in the late 1740s, but he never published it, for reasons unknown. After his death, his friend Richard Price found the paper among Bayes’ effects, and Price sent it for publication to John Canton of the Royal Society of London (apparently modifying the original paper considerably), and it appeared in the Philosophical Transactions in 1763. No one paid it the slightest attention. Ten years later, the Frenchman Pierre Simon Laplace independently discovered the rules of inverse probability, and although he later learned about Bayes’ paper and gave him priority, for the next century and a half Laplace got most of the credit (when credit was given at all--most statisticians did not consider Bayesian methods to be reputable, since they often involved making hunches and using gut feelings).  It wasn't until 1950 that the famous geneticist and mathematician R.A. Fisher first applied Bayes’ name to the methods of inverse probability, and since then, Bayes’ reputation has been gradually restored" (William B. Ashworth, Jr., email received on April 7, 2014.)

Hook & Norman, Origins of Cyberspace (2002) no. 1.

(This entry was last revised on April 7, 2014.)