Classical probability theory (and in particular, Bayesian statistics) provides us with a statistical theory of evidence which allows us to update the probabilities attached to certain facts in the light of new evidence. The fundamental notion is that of conditional probability:
P(H | E)
This is read as the probability that a hypothesis H is true given that we have observed evidence E for the hypothesis. So, using Rich &Knight's example, we might be interested in the probability that a patient has measles given the knowledge that they have spots:
P(patient-has-measles | patient-has-spots)
Sometimes we will know how likely some ``evidence'' is, if some hypothesis is true, but not the other way around. For example, we may know that 50%of people with measles have spots. We may also know that:
The only diseases that cause spots are measles, chickenpox and lassa fever.
60%of people with chickenpox have spots.
80%of people with lassa fever have spots.
There is a 1%chance of someone in a given population having measles (given no evidence for or against).
There is a 1%chance of them having chickenpox.
There is a 0.05%chance of them having lassa fever.
This can be represented more formally as:
P(spots | measles) = 0.5
P(spots | chickenpox) = 0.6
P(spots | lassa) = 0.8
P(measles) = 0.01
P(chickenpox) = 0.01
P(lassa) = 0.0005
From this we can calculate the probability that they have measles if they have spots (if we have no other evidence). Abbreviating things somewhat, for the above example we have:
Or using the actual probabilities:
P(m | s) = 0.5 x 0.01 / (0.5 x 0.01 + 0.6 x 0.01 + 0.8 x 0.0005)
= 0.44
(The general formula for Bayes theorem is given in Rich &Knight).
Bayes theorem is only valid if we know all the conditional probabilities relating to the evidence in question - if there are hundreds of different diseases causing spots we'd need to know the probabilities of spots in each case. This makes it hard to apply the theorem in practical AI applications - we just don't know of all the relevant diseases or their spot probabilities. Also, if we have multiple bits of evidence (maybe the patient has a fever too) then things get more complex. In fact, as we consider more and more evidence it quickly becomes computationally intractable to use Bayes theorem, quite apart from the problem of obtaining and representing all the conditional probabilities. Because of this, Bayes theorem is rarely used in its full glory. However, it is important as it is a well known, sound way of dealing with the probabilities of hypotheses given evidence, and as such provides a kind of standard for assessing other approaches.