In science, experimenter's regress refers to a loop of dependence between theory and evidence. In order to judge whether a new piece of evidence is correct we rely on theory-based predictions, and to judge the value of competing theories we rely on existing evidence. Cognitive bias affects experiments, and experiments determine which theory is valid. This issue is particularly important in new fields of science where there is no consensus regarding the values of various competing theories, and where the extent of experimental errors is not well known.
If experimenter's regress acts a positive feedback system, it can be a source of pathological science. An experimenter's strong belief in a new theory produces confirmation bias, and any biased evidence they obtain then strengthens their belief in that particular theory. Neither individual researchers nor entire scientific communities are immune to this effect: see N-rays and polywater.
Experimenter's regress is a typical relativistic phenomenon in the Empirical Programme of Relativism (EPOR). EPOR is very much concerned with a focus on social interactions, by looking at particular (local) cases and controversial issues in the context in which they happen. In EPOR, all scientific knowledge is perceived to be socially constructed and is thus "not given by nature".
In his article Son of seven sexes: The Social Destruction of a Physical Phenomenon, Harry Collins argued that scientific experiments are subject to what he calls "experimenter's regress".[1] The outcome of a phenomenon that is studied for the first time is always uncertain and judgment in these situations, about what matters, requires considerable experience, tacit and practical knowledge. When a scientist runs an experiment, and the experiment yields a result, they can never be sure whether this is the result which they had expected. The result looks good because they know that their experimental protocol was correct; or the result looks wrong, and therefore there must be something wrong with their experimental protocol. The scientist, in other words, has to get the right answer in order to know that the experiment is working, or know that the experiment is working to get the right answer.
Experimenter's regress occurs at the "research frontier" where the outcome of research is uncertain, for the scientist is dealing with "novel phenomena". Collins puts it this way: "usually, successful practice of an experimental skill is evident in a successful outcome to an experiment, but where the detection of a novel phenomenon is in question, it is not clear what should count as a 'successful outcome' – detection or non detection of the phenomenon" (Collins 1981: 34). In new fields of research where no paradigm has yet evolved and where no consensus exists as what counts as proper research, experimenter's regress is a problem that often occurs. Also, in situations where there is much controversy over a discovery or claim due to opposing interests, dissenters will often question experimental evidence that founds a theory.[2]
Because, for Collins, all scientific knowledge is socially constructed, there are no purely cognitive reasons or objective criteria that determine whether a claim is valid or not. The regress must be broken by "social negotiation" between scientists in the respective field. In the case of Gravitational Radiation, Collins notices that Weber, the scientist who is said to have discovered the phenomenon, could refute all the critique and had "a technical answer for every other point" but he was not able to convince other scientists and in the end he was not taken seriously anymore.[2]
The problems that come with "experimenter's regress" can never be fully avoided because scientific outcomes in EPOR are seen as negotiable and socially constructed. Acceptance of claims boils down to persuasion of other people in the community. Experimenter's regress can always become a problem in a world where "the natural world in no way constrains what is believed to be". Moreover, it is difficult to falsify a claim by replicating an experiment; aside from the practical issues of time, money, access to facilities, etc., an experimental outcome may depend on precise conditions, or tacit knowledge (i.e. unarticulated knowledge) that was not included in the published experimental methods. Tacit knowledge can never be fully articulated or translated into a set of rules.
Some commentators have argued that Collins's "experimenter's regress" is foreshadowed by Sextus Empiricus' argument that "if we shall judge the intellects by the senses, and the senses by the intellect, this involves circular reasoning inasmuch as it is required that the intellects should be judged first in order that the intellects may be tested [hence] we possess no means by which to judge objects" (quoted after Godin & Gingras 2002: 140). Others have extended Collins's argument to the cases of theoretical practice ("theoretician's regress"; Kennefick 2000) and computer simulation studies ("simulationist's regress"; Gelfert 2011; Tolk 2017).
^H.M. Collins (1981). "'Son of seven sexes', The Social Destruction of a Physical Phenomenon". Social Studies of Science. 11 (1): 33–62. doi:10.1177/030631278101100103. S2CID145126667.
H.M. Collins (1981). "'Son of seven sexes', The Social Destruction of a Physical Phenomenon". Social Studies of Science. 11 (1): 33–62. doi:10.1177/030631278101100103. S2CID145126667.
A. Gelfert (2011), "Scientific Models, Simulation, and the Experimenter's Regress", in Humphreys, Paul and Cyrille Imbert (ed.), Models, Simulations, and Representations, London: Routledge, pp. 145–167
D. Kennefick (2000). "Star Crushing: Theoretical Practice and the Theoreticians' Regress". Social Studies of Science. 30 (1): 5–40. doi:10.1177/030631200030001001. S2CID145568092.
