Probability as an Extension of Logic

A truth table assigns each proposition one of two values and then computes the value of every compound from the values of its parts. The whole apparatus of the beginner tier rests on there being exactly two values and on the computation being possible.

Probability keeps the first idea and modifies the second. Each proposition receives a value, but the values are drawn from the whole interval between zero and one rather than from a two-element set, and the values one and zero turn out to behave exactly as truth and falsity did. The deductive apparatus is not replaced. It sits inside the probabilistic one as the limiting case where every value has gone to an extreme.

That is the sense in which probability extends logic, and this article works out what the extension costs and what it does not preserve. The interpretive question of what a probability is a probability of, which divides the subject into rival camps, is deliberately left aside; the structure below is common ground between all of them.

What the numbers attach to

Before values can be assigned, there has to be a settled collection of things to assign them to, and the collection has to be closed under the operations that will be performed on it.

The sample space is the set of possible outcomes, written $\Omega$, and an event is a subset of it: the event that a die shows an even number is the subset containing three of the six outcomes. The reason for demanding closure is that a probability assignment must handle whatever the operations produce, and an assignment defined on some events but not on their complements would be unusable.

There is a second reading of the same structure, and it is the one that matters for the connection to logic. Take the objects receiving values to be propositions rather than sets of outcomes, with conjunction in place of intersection, disjunction in place of union, and negation in place of complement. Nothing changes. The Boolean operations behave identically under both readings, and everything below can be stated either way. The set-theoretic version is standard in mathematics and the propositional version is standard in philosophy, and switching between them is a matter of translation rather than of substance.

The axioms

The first axiom fixes the bottom of the scale, the second fixes the top, and the third does all the work.

Several familiar facts follow at once. The probability of the complement of an event is one minus the probability of the event, since an event and its complement are exclusive and together exhaust the space. The empty event has probability zero. And for events that are not exclusive, the probability of the union is the sum of the two probabilities minus the probability of their overlap, since simple addition would count the overlap twice.

The gap between finite and countable additivity looks technical and has been a live dispute. Countable additivity is what makes the theory connect to measure theory and to the limit theorems, and mathematical practice assumes it without comment. It also has consequences some have found unwelcome, notably that no countably additive measure spreads probability evenly over a countably infinite set, so there is no such thing as picking a natural number entirely at random. Those who take probability to be a constraint on the beliefs of an agent rather than a piece of mathematics have sometimes declined the stronger axiom for exactly that reason, de Finetti being the best known.

Reading the axioms as a generalisation

The connection to the earlier articles is not an analogy. It can be stated as a set of theorems, and each one recovers something the deductive apparatus already had.

A tautology receives probability one, since a tautology is true on every interpretation and so corresponds to the whole sample space. A contradiction receives probability zero. If one proposition entails another, the first cannot receive a higher probability than the second, since the set of outcomes where the first holds is included in the set where the second does. And two logically equivalent propositions receive the same probability, since they correspond to the same set.

Deductive validity reappears as a special case. An argument is valid when every interpretation making the premises true makes the conclusion true, and in probabilistic terms this means that assigning probability one to each premise forces probability one on the conclusion. What the probabilistic framework adds is the ability to say something about arguments whose premises fall short of certainty, which the deductive framework cannot do at all, and the results there are less comfortable than one might expect. Uncertainty accumulates: an argument with ten premises each at probability nine tenths can leave its conclusion at little more than a third, even when the argument is deductively valid. The article on uncertain premises in the advanced tier takes that up.

One thing does not carry over, and it is the most important negative result in this article. Probability is not truth-functional. The value of a conjunction is not determined by the values of its conjuncts. If $P(A)$ and $P(B)$ are each one half, $P(A \cap B)$ may be one half, if the two events coincide; it may be zero, if they exclude each other; and it may be a quarter, if they are unrelated in the relevant sense. Nothing about the two separate values settles the matter, and no table of the sort used in the beginner tier can be written down.

This is why probability is a genuine extension rather than a relabelling. The two-valued case is truth-functional because the only way for two propositions each to have value one is for both to hold, and there is nothing further to specify. As soon as intermediate values are admitted, the relation between propositions becomes an additional parameter, and the theory has to carry information the truth table never needed.

Independence

The definition is the missing parameter from the previous section, pinned to one particular value. When events are independent, the conjunction's probability is fixed by the conjuncts' probabilities after all, which is why independence assumptions are so heavily relied on in practice: they restore something like truth-functionality and make problems tractable.

Two warnings attach to it. The first is that independence is not the same as having no causal connection, though the two often coincide. It is a numerical condition, and events with an evident causal link can satisfy it by coincidence of values while events with no connection at all can fail it through a common cause. The second is that independence for each pair in a collection does not give independence for the collection as a whole. Three events can be pairwise independent while the occurrence of two of them determines the third, and probabilistic arguments that slide from the pairwise condition to the stronger one are making a mistake that is easy to miss.

Independence is also what connects this material to conditional probability, treated elsewhere: two events are independent exactly when conditioning on one leaves the other's probability unchanged, which is the form in which the notion is usually applied.

Cox's route to the same place

Kolmogorov's axioms are a stipulation. They define a mathematical object, and they can be studied without any claim that they are the right constraints on anything. A different line of argument tries to derive them, and it is the line that most directly supports reading probability as an extension of logic.

The force of the result is that no separate justification of the axioms is needed. Anyone who wants a graded analogue of the truth values, who wants it to respect the Boolean structure of the propositions it grades, and who wants it to be consistent, has already committed to probability up to a change of scale. There is no third option waiting to be discovered, and alternative uncertainty formalisms have to reject one of the requirements rather than find a gap between them.

The argument has been read as showing that probability theory simply is the logic of plausible inference, extended from the two-valued case to the graded one, and Jaynes made that reading the organising idea of a large book. The reading is attractive and it should be held with some care. Halpern showed in 1999 that Cox's argument, as originally given, does not go through for domains with only finitely many propositions without further assumptions, and that the additional assumptions needed are strong enough to be worth stating rather than passing over. The theorem survives in repaired forms. What does not survive is the claim that it derives probability from requirements nobody could object to.

Where the numbers come from

The axioms constrain probability values without supplying any. Given that some event has probability one third, they determine a great deal; given nothing, they determine nothing. So there has to be some further principle taking one from a description of a situation to an initial assignment, and the oldest candidate is also the most troubled.

Applied to a die with no reason to think it loaded, the principle gives each face one sixth, and the answer is the one everybody wants. The difficulty is that the principle is sensitive to how the possibilities are described, and descriptions of the same situation that seem equally natural produce different answers.

The three constructions are worth following, because the paradox is easy to state and easy to underestimate.

Take a circle, and inscribe an equilateral triangle in it. A chord may be selected by fixing one endpoint on the circumference and choosing the other endpoint uniformly around the circumference; the chord is longer than the triangle's side when the second endpoint falls on the arc between the two far vertices, which is one third of the circumference, so the answer is one third. Alternatively a chord may be selected by choosing a radius, choosing a point uniformly along it, and drawing the chord perpendicular to the radius at that point; the chord is longer than the side when the point falls in the inner half of the radius, so the answer is one half. Or a chord may be selected by choosing its midpoint uniformly within the disc; the chord is longer than the side when the midpoint falls inside the circle of half the radius, whose area is a quarter of the whole, so the answer is one quarter.

Each construction picks a different quantity to spread uniformly, and nothing in the phrase "at random" says which. The principle of indifference, applied without a specification of the underlying quantity, is not a rule at all but an invitation to choose one and not say so.

Responses to this fall into three groups. Some hold that the question is ill-posed and that a random chord means nothing until a mechanism is specified, in which case the paradox is a warning against a form of words rather than an objection to the principle. Some seek an additional constraint, invariance under the symmetries of the problem being the most developed candidate, that singles out one construction as correct. And some abandon the principle entirely, taking initial probabilities to be inputs supplied from elsewhere rather than outputs of a rule. Which response is right depends on what probability is taken to be, and that is the interpretive question this article set aside at the start.

Next

The tools are now assembled. The remaining articles in this tier are about the language rather than in it, and they begin with the two results that everything else depends on. The next article states what soundness and completeness claim, and why the second is a great deal harder to believe than the first.