Digifesto

Tag: instrumental realism

instrumental realism and reproducibility in science and society

In Instrumental Realism, Ihde does a complimentary treatment of Ackerman’s Data, Instruments, and Theory (1985), which is positioned as a rebuttal to Kuhn. It is a defense of the idea of scientific progress, which is so disliked by critical scholarship. The key issue is are relativistic attacks on scientific progression that point out, for example, the ways in which theory shapes observation, which undermines the objectivity of observation. Ackerman’s rebuttal is that science does not progress through advance of theory, but rather through advance of instrumentation. Instruments allow data to be collected independently of theory. This creates and bounds “data domains”–fields of “data text” that can then be the site of scientific controversy and resolution.

The paradigmatic scientific instruments in Ackerman’s analysis are the telescope and the microscope. But it’s worthwhile thinking about what this means for the computational tools of “data science”.

Certainly, there has been a great amount of work done on the design and standardization of computational tools, and these tools work with ever increasing speed and robustness.

One of the most controversial points made in research today is the idea that the design and/or of these computational tools encodes some kind of bias that threatens the objectivity of their results.

One story, perhaps a straw man, for how this can happen is this: the creators of these tools have (perhaps unconscious) theoretical presuppositions that are the psychological encoding of political power dynamics. These psychological biases impact their judgment as they use tools. This sociotechnical system is therefore biased as the people in it are biased.

Ackerman’s line of argument suggests that the tools, if well designed, will create a “data domain” that might be interpeted in a biased way, but that this concern is separable from the design of the tools themselves.

A stronger (but then perhaps even harder to defend) argument would be that the tools themselves are designed in such a way that the data domain is biased.

Notably, the question of scientific objectivity depends on a rather complex and therefore obscure supply chain of hardware and software. Locating the bias in it must be extraordinarily difficult. In general, the solution to handling this complexity must be modularity and standardization: each component is responsible for something small and well understood, which provides a “data domain” available for downstream use. This is indeed what the API design of software packages is doing. The individual components are tested for reproducible performance and indeed are so robust that, like most infrastructure, we take them for granted.

The push for “reproducibility” in computational science is a further example of refinement of scientific instruments. Today, we see the effort to provide duplicable computational environments with Docker containers, with preserved random seeds, and appropriately versioned dependencies, so that the results of a particular scientific project are maintained despite the constant churn of software, hardware, and networks that undergird scientific communication and practice (let alone all the other communication and practice it undergirds).

The fetishization of technology today has many searching for the location of societal ills within the modules of this great machine. If society, running on this machine, has a problem, there must be a bug in it somewhere! But the modules are all very well tested. It is far more likely that the bug is in their composition. An integration error.

The solution (if there is a solution, and if there isn’t, why bother?) has to be to instrument the integration.

Instrumental realism — a few key points

Continuing my reading of Ihde (1991), I’m getting to the meat of his argument where he compares and constrasts his instrumental realist position with two contemporaries: Heelan (1989), whom Ihde points out is a double doctorate in physics and philosophy and so might be especially capable of philosophizing about physics praxis, and Hacking (1983), who is from my perspective the most famous of the three.

Ihde argues that he, Hacking, and Heelan are all more or less instrumental realists, but that Ihde and Heelan draw more from the phenomenological tradition, which emphasizes embodied perception and action, whereas Hacking is more in the Anglo-American ‘analytic’ tradition of starting from analysis of language. Ihde’s broader argument in the book is one of convergence: he uses the fact that many different schools of thought have arrived at similar conclusions to support the idea that those conclusions are true. That makes perfect sense to me.

Broadly speaking, instrumental realism is a position that unites philosophy of science with philosophy of technology to argue that:

  • That science is able to grasp, understand, theorize the real
  • That this reality is based on embodied perception and praxis. Or, in the more analytic framing, on observation and experiment.
  • That scientific perception and praxis is able to go “beyond” normal, every-day perception and praxis because of its use of scientific instruments, of which the microscope is a canonical example.
  • This position counters many simple relativistic threats to scientific objectivity and integrity, but does so by placing emphasis on scientific tooling. Science advances, mainly, by means of the technologies and infrastructures that it employs.
  • This position is explicitly embodied and materialist, counter to many claims that scientific realism depends on its being disembodied or transcendental.

This is all very promising though there are nuances to work out. Ihde’s study of his contemporaries is telling.

Ihde paints Heelan as a compelling thinker on this topic, though a bit blinkered by his emphasis on physics as the true or first science. Heelean’s view of scientific perception is that it is always both perception and measurement. Being what Ihde calls a “Euro-American” (which I think is quite funny), Ihde can describe him as therefore saying that scientific observation is both a matter of perception-praxis and a matter of hermeneutics–by which I mean the studying of a text in community with others or, to use the more Foucauldean term, “discourse”. Measurement, somewhat implicitly here is a kind of standardized way of “reading”. Ihde makes a big deal out of the subtle differences between “seeing” and “reading”.

To the extent that “discourse”, “hermeneutics”, “reading”, etc. imply a weakness of the scientific standpoint, they weigh against the ‘realism’ of instrumental realism. However, the term measurement is telling in that the difference between, say, different units of measurement of length, mass, time, etc. does not challenge the veracity of the claim “there are 24 hours in a day” because translating between different units is trivial.

Ihde characterizes Hacking as a fellow traveler, converging on instrumental realism when he breaks from his own analytic tradition to point out that experiment is one of the most important features of science, and that experiment depends on and is advanced by instrumentation. Ihde writes that Hacking is quite concerned about “(a) how an instrument is made, particularly with respect to theory-driven design, and (b) the physical processes entailed in the “how” or conditions of use.” Which makes perfect sense to me–that’s exactly what you’d want to scrutinize if you’d taking the ‘realism’ in instrumental realism seriously.

Ihde’s positions here, as the positions of his contemporaries, seem perfectly reasonable to me. I’m quite happy to adopt this view; it corresponds to conclusions I’ve reached in my own reading and practice and it’s nice to have a solid reference and term for it.

The questions that come up next are how instrumental realism applies to today’s controversies about science and technology. Just a handful of notes here:

  • I work quite a bit with scientific sofware. It’s quite clear to me that scientific software development is a major field of scientific instrumentation today. Scientists “see” and “do” via computers and software controls. This has made “data science” a core aspect of 21st century science in general, as it’s the part of science that is closest to the instrumentation. This confirms my long-held view that scientific software communities are the groups to study if you’re trying to understand sociology of science today.
  • On the other hand, it’s becoming increasingly clear in scientific practice that you can’t do software-driven science without the Internet and digital services, and these are now controlled by an oligopoly of digital services conglomerates. The hardware infrastructure–data centers, caching services, telecom broadly speaking, cloud computing hubs–goes far beyond the scientific libraries. Scientific instrumentation depends critically now on mass corporate IT.
  • These issues are compounded by how Internet infrastructure–now privately owned and controlled for all intents and purposes–is also the instrument of so much social science research. Don’t get me started on social media platforms as research tools. For me, the best resource on this is Tufekci, 2014.
  • The most hot-button, politically charged critique in the philosophy of science space is that science and/or data science and/or AI as it is currently constituted is biased because of who is represented in these research communities. The position being contested is the idea that AI/data science/computational social science etc. is objective because it is designed in a way that aligns with mathematical theory.
    • I would be very interested to read something connecting postcolonial, critical race, and feminist AI/data science practices to instrumental realism directly. I think these groups ought to be able to speak to each other easily, since the instrumental realists from the start are interested in the situated embodiment of the observer.
    • On the other hand, I think it would be difficult for the critical scholars to find fault in the “hard core” of data science/computing/AI technologies/instruments because, truly, they are designed according to mathematical theory that is totally general. This is what I think people mean when they say AI is objective because it’s “just math”. AI/data science praxis makes you sensitive to what aspects of the tooling are part of the core (libraries of algorithms, based on vetted mathematical theorems) and what are more incidental (training data sets, for example, or particular parameterizations of the general algorithms). If critical scholars focused on these parts of the scientific “stack”, and didn’t make sweeping comments that sound like they implicate the “core”, which we have every reason to believe is quite solid, they would probably get less resistance.
    • On the other hand, if science is both a matter of perception-praxis and hermeneutics, then maybe the representational concerns are best left on the hermeneutic side of the equation.

References

Hacking, I. (1983). Representing and Intervening: Introductory Topics in the Philosophy of Natural Science.

Heelan, P. A. (1989). Space-perception and the philosophy of science. Univ of California Press.

Ihde, D. (1991). Instrumental realism: The interface between philosophy of science and philosophy of technology (Vol. 626). Indiana University Press.

Tufekci, Z. (2014, May). Big questions for social media big data: Representativeness, validity and other methodological pitfalls. In Eighth International AAAI Conference on Weblogs and Social Media.