Epistemological short-circuits

I cannot therefore sidestep in any way the famous, but so badly formulated question: "Is probability subjective or objective?" In fact there is not, nor can there be, any such thing as probability in itself. There are only probabilistic models. In other words, randomness is in no way a uniquely defined, or even definable property of the phenomenon itself.

In a serious work, written by a competent author, on a scientific subject, an expression such as "this phenomenon is due to chance" constitutes simply, in principle, an elliptic form of speech. It really means "everything occurs as if this phenomenon were due to chance," or, to be more precise: "To describe, or interpret or formalize this phenomenon, only probabilistic models have so far given good results." It is then only a statement of fact. Eventually, the author may add: "And it does not appear to me that this situation is likely to change in the foreseeable future." There is then a personal stand, a methodological choice, which opens certain possibilities to research, but closes others. One understands that the author, who knows his subject, since he has practiced it for many years, wishes to spare his colleagues the trouble of entering a blind alley, thus saving them precious time.

 

There is a risk, because after all there is no reason to believe that tomorrow or in ten years time another researcher will not publish a deterministic theory which will explain beautifully and completely the phenomenon at hand, and we shall have perhaps missed the boat by following the implicit advice of our author: it is a real risk, but one which is normal and inherent to the practice of scientific work, since we must in any case make methodological choices.

 

In this purely operational sense, recourse to chance, that is, in reality, the decision to use probabilistic models, is perfectly legitimate and does not take us outside the framework of objectivity. Sometimes, however, and probably more often in works of popularization and philosophical synthesis than in purely scientific works, we are presented with quite a different interpretation. It is suggested, nay even affirmed, that Chance (sometimes with a capital c) acts decisively in its own right on the course of events. This is no longer a methodological choice. Chance is now hypothesized, supplied with positive attributes, set up as a deus ex machina. Under these circumstances, to attribute the phenomenon to Chance, is equivalent to attributing it to Providence, and both are foreign to scientific methodology.

Illegitimate use of scientific concepts beyond the limits within which they have an operative meaning is nothing else but a surreptitious passage into metaphysics.

Any given model, however well tested and corroborated, always necessarily contains theorems which do not correspond any more to empirical formulations, which cannot be controlled, and are not even controllable, beyond a certain limit. There always exists a threshold of realism, beyond which a mathematician can certainly pursue happily his deductions, but which a physicist must respect, lest he obtain first uncontrolled formulations, and later uncontrollable ones, that is, formulations which lack any objective meaning. In other words, they are "metaphysical" in the sense given to this word in the usage of objective science.

[...] nobody has ever applied either the theory of probability, or for that matter any other mathematical theory, to reality. One can only "apply" to reality real (physical, technical, etc.) operations, not mathematical operations. The latter only apply to mathematical models of the same nature as themselves. In other words, it is always to probabilistic models, and only to them, that we apply the theory of probability. [...] There is no probability in itself. There are only probabilistic models. The only question that really matters, in each particular case, is whether this or that probabilistic model, in relation to this or that real phenomenon, has or has not an objective meaning. As we have seen, this is equivalent to asking whether the model is falsifiable.

[...] sometimes we choose a unique and well-defined probability P from the beginning, and we then say that the model is completely specified; sometimes, however, we retain some room for maneuvering by just choosing a family P(a,b,...) of probabilities depending on a small number of parameters a,b,... In the latter case, we say that we have only chosen the type of model, and that we have left the problem of its specification, that is of the choice of the numerical values which should be attributed to the parameters a,b,... to a later stage. According to the viewpoint of "orthodox" statisticians the choice of the type of model constitutes a "hypothesis," while the problem of the specification of the numerical values of the parameters is called "statistical inference," or "estimation" of these parameters. Since this terminology carries with it implicit presuppositions concerning the real and objective existence of these parameters, we shall use the neutral and purely descriptive term of choice

The essential point, from the point of view of methodology, is to carefully distinguish between the two completely different roles we attribute to the same symbol p. On the one hand, p is a parameter of the model. On account of this, it may (or may not) have an objective meaning, and the assertion "p = 1/2" may (or may not) be falsifiable, that is, objective or empirical. On the other hand, the same symbol appears in the equality P(X10 = 1) = p. Here we are stating that the probability of success at the 10-th throw (of the present game) has a fixed numerical value, e.g. p=1/2: a singular and undecidable statement, which is therefore certainly devoid of any objective meaning, like all statements relating to the probability of a unique event.

The defining criterion, (in the strict sense) for the objectivity of a probabilistic model would thus be as follows: we agree to declare a model falsified if an event of zero probability (in the model) actually occurs (in reality).

According to the viewpoint of "orthodox" statistics, the most important problem one must then solve is that of "statistical inference", that is, the "estimation" of the unknown intensity theta of the Poisson process [modeling some forest dynamics]. For once this parameter is known one can calculate all other characteristics of the process. This point of view attributes, implicitly, a real and objective existence to the intensity parameter 'e': namely that even if our information allows us to arrive at no more than an approximate estimate of the "true" value of 'e', this does not change the fact that the latter exists somewhere in nature, and could be precisely measured, given perfect information. But in reality it is not at all certain that this contention has an operational sense: for in order to determine 'e' precisely, the forest would have to extend to infinity (and remain Poisson) while its real size is in fact limited. The presumed obviousness of the existence of 'e' is based on a summary identification of the model (the Poisson process) with reality (the forest). Such a confusion, which is quite common among statisticians, is essentially an epistemological short-circuit. For however well a model is adapted to its object, we never have a guarantee that all its characteristics will faithfully mirror objective properties of reality which can be put in a one-to-one correspondence with them.

The preceding analysis has highlighted three steps which have very different epistemological standings:

 

(1) there is first an epistemological choice: it has been decided to use probabilistic techniques to represent the phenomenon (the forest). This is a decision, not a hypothesis. It is a constitutive decision. (It "constitutes" the forest as an object of study, it defines the general framework within which we shall operate and determines the choice of the tools we use.) It is not an experimentally verifiable hypothesis (for it is neither true nor false to say that "this forest is a realization of a stochastic process", since there is no conceivable experiment or observation that could refute this proposition). At this level, we shall speak of a constitutive model (here a probabilistic one).

 

(2) We next encounter a hypothesis about the physical nature of the phenomenon studied - such as spatial homogeneity, absence or influence between neighboring regions - which leads to the choice of a generic model: the process is a Poisson process. Contrary to the preceding choice (which can only be justified by its efficiency and by the successes it leads to, and on which one can only pass judgement in the long run, after having dealt with a large number of cases) this second choice follows from a physical hypothesis which can be objectively tested. It can, therefore, be supported or rejected by the experimental data either through statistical tests, which can be very easily devised for this very simple particular case, or through some other method, including the judgement of the practitioner who knows that forest well. Resorting to the practitioner's intuition has no mystical connotations whatever to it; it simply reflects the epistemological priority that we give to reality over the mathematical or rather statistical model we have chosen to describe it.

 

We cannot overstress the capital importance of this step. For it is essentially here that we incorporate in the model hypotheses that have an objective meaning and that carry with them positive information which is not contained in the raw data. It is only because of this positive contribution that we can (apparently) extract from the data more than they really contain (i.e. a prediction as well as an estimation variance). The counterpart of this small-scale miracle is that our model is now vulnerable, and that our predictions can now be contradicted by experiment if the hypotheses on which we base them are not objectively valid. It should be made clear here that it is not enough to check (e.g. through tests) that these hypotheses are compatible with the data, that is, that they are verified over the sampled plots. We must also assume that they remain valid over the regions that have not been sampled, and we shall only know this after the fact. The choice of the model constitutes therefore an anticipatory hypothesis and always introduces a risk of radical error. This is why it is imperative that it takes into account not only the numerical data, but also all other available sources of in- formation (general knowledge about this type of phenomenon, the experience of practitioners, etc...).

 

In order to better localize the input of positive information and the consequent vulnerability of the model, it is often advantageous to distinguish two steps in the choice of the generic model: choice of a generic model in the wide sense, then choice of a particular type within this model. In the present case, the generic model (in the wide sense) would be for example "a stationary point process." The word "point" means that we have decided, as a first approximation, to treat each tree as a point in a plane. This is a (constitutive) decision that does not entail any input of positive information and therefore introduces no risk of experimental rejection. On the other hand, the word "stationary" is associated with a hypothesis of spatial homogeneity and entails an input of positive information as well as a vulnerability, but to a relatively small extent. By model type we shall mean a model that needs only the numerical value of a small number of parameters in order to be completely specified. Here the model type chosen is a "Poisson process" with only one indeterminate parameter. The choice of this type implies a very strong hypothesis, namely the lack of interaction between neighboring regions. It introduces a large amount of information, which enables us, among other things, to calculate the estimation variance. And it therefore introduces further risks of experimental rejection. This is a general rule: very often it is the choice of a type within the generic model that constitutes the crucial decision, the one that opens the largest number of operational possibilities, but also inevitably introduces the greatest risk of error.

 

(3) Finally the last step is the choice of the specific model or, as we shall also say, the specification of the model (here the choice of the numerical value for 'so many trees per hectare'). While Mathematical Statistics attributes an absolutely vital role to this third aspect of "model choice," to which it refers as statistical inference (i.e. numerical estimation of the parameters), it will only play a very minor role, or even none at all, here since the essential results (estimation and variance) will be expressed, in the final analysis, in terms of the experimental data alone, in a form which exploits choices (1) and (2) (and particularly the choice of the Poisson type) but not at all choice (3) of the numerical value [of the parameter]. For in reality it is only for terminological convenience and to clarify our ideas that we carry out the specification of the model, while choice (1) (the epistemo-logical choice) and choice (2) (choice of the generic model and its type) together with the numerical information provide us with an operational basis which is sufficient for solving the problem we are dealing with.

 

The precise distinction between these three aspects of model choice may appear elementary and uninteresting in a case as simple as that of the Poisson forest. But this will change rapidly as we examine more complicated models. Once the choice of the constitutive model (the epistemological decision) has been agreed upon, the crucial problem remains choice number (2), that of the generic model and more particularly that of model type. For once we have adopted a model that is suited to the experimental data, it is usually not difficult to specify it, if we really want to, on the basis of these same data. 

 

The important problem is therefore not that of statistical inference, but the choice of the generic model and of its type. Let us note carefully the fact that the problem is not just to "test a hypothesis" against the given data. This viewpoint, which is that of orthodox statistics, disregards the core of the problem. It is not only agreement with the available data that we need, but also the much stronger hypothesis that the chosen type is also compatible with the data which are as yet unavailable, that is, the unknown or inaccessible parts of the phenomenon. It is the latter hypothesis which explains the fertility of the method, and also, as we have seen, its vulnerability. Compatibility with the data is of course necessary, but it is never sufficient to insure us against an always possible disagreement with what is not given (and this is precisely what we are trying to estimate). Estimating and Choosing, Georges Matheron