A geometric visualisation of Bayes' theorem. In the table, the values 2, 3, 6 and 9 give the relative weights of each corresponding condition and case. The figures denote the cells of the table involved in each metric, the probability being the fraction of each figure that is shaded. This shows that P(A|B) P(B) = P(B|A) P(A) i.e. P(A|B) = P(B|A) P(A)/P(B) . Similar reasoning can be used to show that P(¬A|B) = P(B|¬A) P(¬A)/P(B) etc.
Contingency table
Hypothesis
Evidence
Satisfies hypothesis H
Violates hypothesis ¬H
Total
Has evidence E
P(H|E)·P(E) = P(E|H)·P(H)
P(¬H|E)·P(E) = P(E|¬H)·P(¬H)
P(E)
No evidence ¬E
P(H|¬E)·P(¬E) = P(¬E|H)·P(H)
P(¬H|¬E)·P(¬E) = P(¬E|¬H)·P(¬H)
P(¬E) = 1−P(E)
Total
P(H)
P(¬H) = 1−P(H)
1
Bayesian inference figures out the posterior probability from prior probability and the "likelihood function". The likelihood function comes from a statistical model of the data.
where
is a hypothesis that is changed by data (or evidence). There are usually many hypotheses. The point of the test is to see which hypothesis is more likely.
is the prior probability. It estimates the probability of a hypothesis before there is any evidence.
is the evidence, or data. It is any new data that is found.
is the posterior probability. This is what we want to know.
is the likelihood function.
is the marginal likelihood. It is the same for all possible hypotheses that are being tested. has to be greater than 0. If is 0, then you divide by zero.
Bolstad, William M. (2007) Introduction to Bayesian Statistics: Second Edition, John Wiley ISBN0-471-27020-2
Winkler, Robert L (2003). Introduction to Bayesian Inference and Decision (2nd ed.). Probabilistic. ISBN978-0-9647938-4-2. Updated classic textbook. Bayesian theory clearly presented.
Lee, Peter M. Bayesian Statistics: An Introduction. Fourth Edition (2012), John Wiley ISBN978-1-1183-3257-3
Carlin, Bradley P. & Louis, Thomas A. (2008). Bayesian Methods for Data Analysis, Third Edition. Boca Raton, FL: Chapman and Hall/CRC. ISBN978-1-58488-697-6.
Gelman, Andrew; Carlin, John B.; Stern, Hal S.; Dunson, David B.; Vehtari, Aki; Rubin, Donald B. (2013). Bayesian Data Analysis, Third Edition. Chapman and Hall/CRC. ISBN978-1-4398-4095-5.
O'Hagan, A. and Forster, J. (2003). Kendall's Advanced Theory of Statistics, Volume 2B: Bayesian Inference. Arnold, New York. ISBN0-340-52922-9.
Robert, Christian P (2007). The Bayesian Choice: From Decision-Theoretic Foundations to Computational Implementation (paperback ed.). Springer. ISBN978-0-387-71598-8.
Pearl, Judea. (1988). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, San Mateo, CA: Morgan Kaufmann.
A. Hajek and S. Hartmann: Bayesian Epistemology, in: J. Dancy et al. (eds.), A Companion to Epistemology. Oxford: Blackwell 2010, 93–106.
S. Hartmann and J. Sprenger: Bayesian Epistemology, in: S. Bernecker and D. Pritchard (eds.), Routledge Companion to Epistemology. London: Routledge 2010, 609–620.