Digifesto

Bostrom’s Superintelligence: Definitions and core argument

I wanted to take the opportunity to spell out what I see as the core definitions and argument of Bostrom’s Superintelligence as a point of departure for future work. First, some definitions:

  • Superintelligence. “We can tentatively define a superintelligence as any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest.” (p.22)
  • Speed superintelligence. “A system that can do all that a human intellect can do, but much faster.” (p.53)
  • Collective superintelligence. “A system composed of a large number of smaller intellects such that the system’s overall performance across many very general domains vastly outstrips that of any current cognitive system.” (p.54)
  • Quality superintelligence. “A system that is at least as fast as a human mind and vastly qualitatively smarter.” (p.56)
  • Takeoff. The event of the emergence of a superintelligence. The takeoff might be slow, moderate, or fast, depending on the conditions under which it occurs.
  • Optimization power and Recalcitrance. Bostrom’s proposed that we model the speed of superintelligence takeoff as: Rate of change in intelligence = Optimization power / Recalcitrance. Optimization power refers to the effort of improving the intelligence of the system. Recalcitrance refers to the resistance of the system to being optimized.(p.65, pp.75-77)
  • Decisive strategic advantage. The level of technological and other advantages sufficient to enable complete world domination. (p.78)
  • Singleton. A world order in which there is at the global level one decision-making agency. (p.78)
  • The wise-singleton sustainability threshold. “A capability set exceeds the wise-singleton threshold if and only if a patient and existential risk-savvy system with that capability set would, if it faced no intelligent opposition or competition, be able to colonize and re-engineer a large part of the accessible universe.” (p.100)
  • The orthogonality thesis. “Intelligence and final goals are orthogonal: more or less any level of intelligence could in principle be combined with more or less any final goal.” (p.107)
  • The instrumental convergence thesis. “Several instrumental values can be identified which are convergent in the sense that their attainment would increase the chances of the agent’s goal being realized for a wide range of final goals and a wide range of situations, implying that these instrumental values are likely to be pursued by a broad spectrum of situated intelligent agents.” (p.109)

Bostrom’s core argument in the first eight chapters of the book, as I read it, is this:

  1. Intelligent systems are already being built and expanded on.
  2. If some constant proportion of a system’s intelligence is turned into optimization power, then if the recalcitrance of the system is constant or lower, then the intelligence of the system will increase at an exponential rate. This will be a fast takeoff.
  3. Recalcitrance is likely to be lower for machine intelligence than human intelligence because of the physical properties of artificial computing systems.
  4. An intelligent system is likely to invest in its own intelligence because of the instrumental convergence thesis. Improving intelligence is an instrumental goal given a broad spectrum of other goals.
  5. In the event of a fast takeoff, it is likely that the superintelligence will get a decisive strategic advantage, because of a first-mover advantage.
  6. Because of the instrumental convergence thesis, we should expect a superintelligence with a decisive strategic advantage to become a singleton.
  7. Machine superintelligences, which are more likely to takeoff fast and become singletons, are not likely to create nice outcomes for humanity by default.
  8. A superintelligent singleton is likely to be above the wise-singleton threshold. Hence the fate of the universe and the potential of humanity is at stake.

Having made this argument, Bostrom goes on to discuss ways we might anticipate and control the superintelligence as it becomes a singleton, thereby securing humanity.

And now for something completely different: Superintelligence and the social sciences

This semester I’ll be co-organizing, with Mahendra Prasad, a seminar on the subject of “Superintelligence and the Social Sciences”.

How I managed to find myself in this role is a bit of a long story. But as I’ve had a longstanding curiosity about this topic, I am glad to be putting energy into the seminar. It’s a great opportunity to get exposure to some of the very interesting work done by MIRI on this subject. It’s also a chance to thoroughly investigate (and critique) Bostrom’s book Superintelligence: Paths, Dangers, and Strategies.

I find the subject matter perplexing because in many ways it forces the very cultural and intellectual clash that I’ve been preoccupied with elsewhere on this blog: the failure of social scientists and engineers to communicate. Or, perhaps, the failure of qualitative researchers and quantitative researchers to communicate. Whatever you want to call it.

Broadly, the question at stake is: what impact will artificial intelligence have on society? This question is already misleading since in the imagination of most people who haven’t been trained in the subject, “artificial intelligence” refers to something of a science fiction scenario, whereas to practitioner, “artificial intelligence” is, basically, just software. Just as the press went wild last year speculating about “algorithms”, by which it meant software, so too is the press excited about artificial intelligence, which is just software.

But the concern that software is responsible for more and more of the activity in the world and that it is in a sense “smarter than us”, and especially the fear that it might become vastly smarter than us (i.e. turning into what Bostrom calls a “superintelligence”), is pervasive enough to drive research funding into topics like “AI Safety”. It also is apparently inspiring legal study into the regulation of autonomous systems. It may also have implications for what is called, vaguely, “social science”, though increasingly it seems like nobody really knows what that is.

There is a serious epistemological problem here. Some researchers are trying to predict or forewarn the societal impact of agents that are by assumption beyond their comprehension on the premise that they may come into existence at any moment.

This is fascinating but one has to get a grip.

intelligibility

The example of Arendt’s dismissal of scientific discourse from political discussion underscores a much deeper political problem: a lack of intelligibility.

Every language is intelligible to some people and not to others. This is obviously true in the case of major languages like English and Chinese. It is less obvious but still a problem with different dialects of a language. It becomes a source of conflict when there is a lack of intelligibility between the specialized languages of expertise or personal experience.

For many, mathematical formalism is unintelligible; it appears to be so for Arendt, and this disturbs her, as she locates politics in speech and wants there to be political controls on scientists. But how many scientists and mathematicians would find Arendt intelligible? She draws deeply on concepts from ancient Greek and Augustinian philosophy. Are these thoughts truly accessible? What about the intelligibility of the law, to non-lawyers? Or the intelligibility of spoken experiences of oppression to those who do not share such an experience?

To put it simply: people don’t always understand each other and this poses a problem for any political theory that locates justice in speech and consensus. Advocates of these speech-based politics are most often extraordinarily articulate and write persuasively about the need to curtail the power of any systems of control that they do not understand. They are unable to agree to a social contract that they cannot read.

But this persuasive speech is necessarily unable to account for the myriad mechanisms that are both conditions for the speech and unintelligible to the speaker. This includes the mechanisms of law and technology. There is a performative contradiction between these persuasive words and their conditions of dissemination, and this is reason to reject them.

Advocates of bureaucratic rule tend to be less eloquent, and those that create technological systems that replace bureaucratic functions even less so. Nevertheless each group is intelligible to itself and may have trouble understanding the other groups.

The temptation for any one segment of society to totalize its own understanding, dismissing other ways of experiencing and articulating reality as inessential or inferior, is so strong that it can be read in even great authors like Arendt. Ideological politics (as opposed to technocratic politics) is the conflict between groups expressing their interests as ideology.

The problem is that in order to function as it does at scale, modern society requires the cooperation of specialists. Its members are heterogeneous; this is the source of its flexibility and power. It is also the cause of ideological conflict between functional groups that should see themselves as part of a whole. Even if these members do see their interdependence in principle, their specialization makes them less intelligible. Articulation often involves different skills from action, and teaching to the uninitiated is another skill altogether. Meanwhile, the complexity of the social system expands as it integrates more diverse communities, reducing further the proportion understood by a single member.

There is still in some political discourse the ideal of deliberative consensus as the ground of normative or political legitimacy. Suppose, as seems likely, that this is impossible for the perfectly mundane and mechanistic reason that society is so complicated due to the demands of specialization that intelligibility among its constituents is never going to happen.

What then?

the state and the household in Chinese antiquity

It’s worthwhile in comparison with Arendt’s discussion of Athenian democracy to consider the ancient Chinese alternative. In Alfred Huang’s commentary on the I Ching, we find this passage:

The ancient sages always applied the principle of managing a household to governing a country. In their view, a country was simply a big household. With the spirit of sincerity and mutual love, one is able to create a harmonious situation anywhere, in any circumstance. In his Analects, Confucius says,

From the loving example of one household,
A whole state becomes loving.
From the courteous manner of one household,
A whole state becomes courteous.

Comparing the history of Europe and the rise of capitalistic bureaucracy with the history of China, where bureaucracy is much older, is interesting. I have comparatively little knowledge of the latter, but it is often said that China does not have the same emphasis on individualism that you find in the West. Security is considered much more important than Freedom.

The reminder that the democratic values proposed by Arendt and Horkheimer are culturally situated is an important one, especially as Horkheimer claims that free burghers are capable of producing art that expresses universal needs.

a refinement

If knowledge is situated, and scientific knowledge is the product of rational consensus among diverse constituents, then a social organization that unifies many different social units functionally will have a ‘scientific’ ideology or rationale that is specific to the situation of that organization.

In other words, the political ideology of a group of people will be part of the glue that constitutes the group. Social beliefs will be a component of the collective identity.

A social science may be the elaboration of one such ideology. Many have been. So social scientific beliefs are about capturing the conditions for the social organization which maintains that belief. (c.f. Nietzsche on tablets of values)

There are good reasons to teach these specialized social sciences as a part of vocational training for certain functions. For example, people who work in finance or business can benefit from learning economics.

Only in an academic context does the professional identity of disciplinary affiliation matter. This academic political context creates great division and confusion that merely reflects the disorganization of the academic system.

This disorganization is fruitful precisely because it allows for individuality (cf. Horkheimer). However, it is also inefficient and easy to corrupt. Hmm.

Against this, not all knowledge is situated. Some is universal. It’s universality is due to its pragmatic usefulness in technical design. Since technical design acts on everyone even when their own situated understanding does not include it, this kind of knowledge has universal ground (in violence, sadly, but maybe also in other ways.)

The question is whether there is room anywhere in the technically correct understanding of social organization (something we might see in Beniger) there is room for the articulation of what it supposed to be great and worthy of man (see Horkheimer).

I have thought for a long time that there is probably something like this describable in terms of complexity theory.

structuralism and/or functionalism

Previous entries detailing the arguments of Arendt, Horkheimer, and Beniger show these theorists have what you might call a structural functionalist bent. Society is conceived as a functional whole. There are units of organization within it. For Arendt, this social organization begins in the private household and expands to all of society. Horkheimer laments this as the triumph of mindless economic organization over genuine, valuable individuality.

Structuralism, let alone structural functionalism, is not in fashion in the social sciences. Purely speculatively, one reason for this might be that to the extent that society was organized to perform certain functions, more of those functions have been delegated to information processing infrastructure, as in Beniger’s analysis. That leaves “culture” more a domain of ephemerality and identity conflict, as activity in the sphere of economic production becomes if not private, opaque.

My empirical work on open source communities is suggestive (though certainly not conclusively so) that these communities are organized more for functional efficiency than other kinds of social groups (including academics). I draw this inference from the degree dissortativity of the open source social networks. Disassortativity suggests the interaction of different kinds of people, which is against homophilic patterns of social formation but which seems essential for economic activity where the interact of specialists is what creates value.

Assuming that society its entirety (!!) is very complex and not easily captured by a single grand theory, we can nevertheless distinguish difference kinds of social organization and see how they theorize themselves. We can also map how they interact and what mechanisms mediated between them.

Land and gold (Arendt, Horkheimer)

I am thirty, still in graduate school, and not thrilled about the prospects of home ownership since all any of the professionals around me talk about is the sky-rocketing price of real estate around the critical American urban centers.

It is with a leisure afforded by graduate school that I am able to take the long view on this predicament. It is very cheap to spend ones idle time reading Arendt, who has this to say about the relationship between wealth and property:

The profound connection between private and public, manifest on its most elementary level in the question of private property, is likely to be misunderstood today because of the modern equation of property and wealth on one side and propertylessness and poverty on the other. This misunderstanding is all the more annoying as both, property as well as wealth, are historically of greater relevance to the public realm than any other private matter or concern and have played, at least formally, more or less the same role as the chief condition for admission to the public realm and full-fledged citizenship. It is therefore easy to forget that wealth and property, far from being the same, are of an entirely different nature. The present emergence everywhere of actually or potentially very wealthy societies which at the same time are essentially propertyless, because the wealth of any single individual consists of his share in the annual income of society as a whole, clearly shows how little these two things are connected.

For Arendt, beginning with her analysis of ancient Greek society, property (landholding) is the condition of ones participation in democracy. It is a place of residence and source of ones material fulfilment, which is a prerequisite to ones free (because it is unnecessitated) participation in public life. This is contrasted with wealth, which is a feature of private life and is unpolitical. In ancient society, slaves could own wealth, but not property.

If we look at the history of Western civilization as a progression away from this rather extreme moment, we see the rise of social classes whose power is based on in landholding but in wealth. Industrialism and the economy based on private ownership of capital is a critical transition in history. That capital is not bound to a particular location but rather is mobile across international boundaries is one of the things that characterizes global capitalism and brings it in tension with a geographically bounded democratic state. It is interesting that a Jeffersonian democracy, designed with the assumption of landholding citizens, should predate industrial capitalism and be consitutionally unprepared for the result, but nevertheless be one of the models for other democratic governance structures throughout the world.

If private ownership of capital, not land, defines political power under capitalism, then wealth, not property, becomes the measure of ones status and security. For a time, when wealth was as a matter of international standard exchangeable for gold, private ownership of gold could replace private ownership of land as the guarantee of ones material security and thereby grounds for ones independent existence. This independent, free rationality has since Aristotle been the purpose (telos) of man.

In the United States, Franklin Roosevelt’s 1933 Executive Order 6102 forbade the private ownership of gold. The purpose of this was to free the Federal Reserve of the gold market’s constraint on increasing the money supply during the Great Depression.

A perhaps unexpected complaint against this political move comes from Horkheimer (Eclipse of Reason, 1947), who sees this as a further affront to individualism by capitalism.

The age of vast industrial power, by eliminating the perspectives of a stable past and future that grew out of ostensibly permanent property relations, is the process of liquidating the individual. The deterioration of his situation is perhaps best measured in terms of his utter insecurity as regards to his personal savings. As long as currencies were rigidly tied to gold, and gold could flow freely over frontiers, its value could shift only within narrow limits. Under present-day conditions the dangers of inflation, of a substantial reduction or complete loss of the purchasing power of his savings, lurks around the next corner. Private possession of gold was the symbol of bourgeois rule. Gold made the burgher somehow the successor of the aristocrat. With it he could establish security for himself and be reasonable sure that even after his death his dependents would not be completely sucked up by the economic system. His more or less independent position, based on his right to exchange goods and money for gold, and therefore on the relatively stable property values, expressed itself in the interest he took in the cultivation of his own personality–not, as today, in order to achieve a better career or for any professional reason, but for the sake of his own individual existence. The effort was meaningful because the material basis of the individual was not wholly unstable. Although the masses could not aspire to the position of the burgher, the presence of a relatively numerous class of individuals who were governed by interest in humanistic values formed the background for a kind of theoretical thought as well as for the type of manifestions in the arts that by virtue of their inherent truth express the needs of society as a whole.

Horkheimer’s historical arc, like many Marxists, appears to ignore its parallels in antiquity. Monetary policy in the Roman Empire, which used something like a gold standard, was not always straightforward. Inflation was sometimes a severe problem when generals would print money to pay the soldiers hat supported their political coups. So it’s not clear that the modern economy is more unstable than gold or land based economies. However, the criticism that economic security is largely a matter of ones continued participation in a larger system, and that there is little in the way of financial security besides this, holds. He continues:

The state’s restriction on the right to possess gold is the symbol of a complete change. Even the members of the middle class must resign themselves to insecurity. The individual consoles himself with the thought that his government, corporation, association, union, or insurance company will take care of him when he becomes ill or reaches the retiring age. The various laws prohibiting private possession of gold symbolize the verdict against the independent economic individual. Under liberalism, the beggar was always an eyesore to the rentier. In the age of big business both beggar and rentier are vanishing. There are no safety zones on society’s thoroughfares. Everyone must keep moving. The entrepreneur has become a functionary, the scholar a professional expert. The philosopher’s maxim, Bene qui latuit, bene vixit, is incompatible with the modern business cycles. Everyone is under the whip of a superior agency. Those who occupy the commanding positions have little more autonomy than their subordinates; they are bound by the power they wield.

In an academic context, it is easy to make a connection between Horkheimer’s concerns about gold ownership and tenure. Academic tenure is or was the refuge of the individual who could in theory develop themselves as individuals in obscurity. The price of this autonomy, which according the philosophical tradition represents the highest possible achievement of man, is that one teaches. So, the developed individual passes on the values developed through contemplation and reflection to the young. The privatization of the university and the emphasis on teaching marketable skills that allow graduates to participate more fully in the economic system is arguably an extension of Horkheimer’s cultural apocalypse.

The counter to this is the claim that the economy as a whole achieves a kind of homeostasis that provides greater security than one whose value is bound to something stable and exogenous like gold and land. Ones savings are secure as long as the system doesn’t fail. Meanwhile, the price of access to cultural materials through which one might expand ones individuality (i.e. videos of academic lectures, the arts, or music) decrease as a consequence of the pervasiveness of the economy. At this point one feels one has reached the limits of Horkheimer’s critique, which perhaps only sees one side of the story despite its sublime passion. We see echoes of it in contemporary feminist critique, which emphasizes how the demands of necessity are disproportionately burdened by women and how this affects their role in the economy. That women have only relatively recently, in historical terms, been released from the private household into the public world (c.f. Arendt again) situates them more precariously within the economic system.

What remains unclear (to me) is how one should conceive of society and values when there is an available continuum of work, opportunity, leisure, individuality, art, and labor under conditions of contemporary technological control. Specifically, the notion of inequality becomes more complicated when one considers that society has never been equal in the sense that is often aspired to in contemporary American society. This is largely because the notion of equality we use today draws from two distinct sources. The first is the equality of self-sufficient landholding men as they encounter each other freely in the polis. Or, equivalently, as self-sufficient goldholding men in something like the Habermasian bourgeois public sphere. The second is equality within society, which is economically organized and therefore requires specialization and managerial stratification. We can try to assure equality to members of society insofar as they are members of society, but not as to their function within society.

Horkheimer on engineers

Horkheimer’s comment on engineers:

It is true that the engineer, perhaps the symbol of this age, is not so exclusively bent on profitmaking as the industrialist or the merchant. Because his function is more directly connected with the requirements of the production job itself, his commands bear the mark of greater objectivity. His subordinates recognize that at least some of his orders are in the nature of things and therefore rational in a universal sense. But at bottom this rationality, too, pertains to domination, not reason. The engineer is not interested in understanding things for their own sake or the sake of insight, but in accordance to their being fitted into a scheme, no matter how alien to their own inner structure; this holds for living beings as well as for inanimate things. The engineer’s mind is that of industrialism in its streamlined form. His purposeful rule would make men an agglomeration of instruments without a purpose of their own.

This paragraph sums up much of what Horkheimer stands for. His criticism of engineers, the catalysts of industrialism, is not that they are incorrect. It is that their instrumental rationality is not humanely purposeful.

This humane purposefulness, for Horkheimer, is born out of individual contemplation. Though he recognizes that this has been a standpoint of the privileged (c.f. Arendt on the Greek polis), he sees industrialism as successful in bringing many people out of a place of necessity but at the cost of marginalizing and trivializing all individual contemplation. The result is an efficient machine with nobody in charge. This bodes ill because such a machine is vulnerable to being co-opted by an irrational despot or charlatan. Individuality, free of material necessity and also free of the machine that liberated it from that necessity, is the origin of moral judgement that prevents fascist rule.

This is very different from the picture of individuality Fred Turner presents in The Democratic Surround. In his account of how United States propaganda created a “national character” that was both individual enough to be anti-fascist and united enough to fight fascism, he emphasizes the role of art installations that encourage the view to stitch themselves synthetically into a large picture of the nation. One is unique within a larger, diverse…well, we might use the word society, borrowing from Arendt, who was also writing in the mid-century.

If this is all true, then this dates a transition in American culture from one of individuality to one of society. This coincides with the tendency of information organization traced assiduously by Beniger.

We can perhaps trace an epicycle of this process in the history of the Internet. In it’s “wild west” early days, when John Perry Barlow could write about the freedom of cyberspace, it was a place primarily occupied by the privileged few. Interestingly, many of these were engineers, and so were (I’ll assume for the sake of argument) but materially independent and not exclusively focused on profit-making. Hence the early Internet was not unlike the ancient polis, a place where free people could attempt words and deeds that would immortalize them.

As the Internet became more widely used and commercialized, it became more and more part of the profiteering machine of capitalism. So today we see it’s wildness curtailed by the demands of society (which includes an appeal to an ethics sensitive both to disparities in wealth and differences in the body, both part of the “private” realm in antiquity but an element of public concern in modern society.)

Arendt on social science

Despite my first (perhaps kneejerk) reaction to Arendt’s The Human Condition, as I read further I am finding it one of the most profoundly insightful books I’ve ever read.

It is difficult to summarize: not because it is written badly, but because it is written well. I feel every paragraph has real substance to it.

Here’s an example: Arendt’s take on the modern social sciences:

To gauge the extent of society’s victory in the modern age, its early substitution of behavior for action and its eventual substitution of bureaucracy, the rule of nobody, for personal rulership, it may be well to recall that its initial science of economics, which substitutes patterns of behavior only in this rather limited field of human activity, was finally followed by the all-comprehensive pretension of the social sciences which, as “behavioral sciences,” aim to reduce man as a whole, in all his activities, to the level of a conditioned and behaving animal. If economics is the science of society in its early stages, when it could impose its rules of behavior only on sections of the population and on parts of their activities, the rise of the “behavioral sciences” indicates clearly the final stage of this development, when mass society has devoured all strata of the nation and “social behavior” has become the standard for all regions of life.

To understand this paragraph, one has to know what Arendt means by society. She introduces the idea of society in contrast to the Ancient Greek polis, which is the sphere of life in Antiquity where the head of a household could meet with other heads of households to discuss public matters. Importantly for Arendt, all concerns relating to the basic maintenance and furthering of life–food, shelter, reproduction, etc.–were part of the private domain, not the polis. Participation in public affairs was for those who were otherwise self-sufficient. In their freedom, they would compete to outdo each other in acts and words that would resonate beyond their lifetime, deeds, through which they could aspire to immortality.

Society, in contrast, is what happens when the mass of people begin to organize themselves as if they were part of one household. The conditions of maintaining life are public. In modern society, people are defined by their job; even being the ruler is just another job. Deviation from ones role in society in an attempt to make a lasting change–deeds–are considered disruptive, and so are rejected by the norms of society.

From here, we get Arendt’s critique of the social sciences, which is essentially this: that is only possible to have a social science that finds regularities of people’s behavior when their behavior has been regularized by society. So the social sciences are not discovering a truth about people en masse that was not known before. The social sciences aren’t discovering things about people. They are rather reflecting the society as it is. The more that the masses are effectively ‘socialized’, the more pervasive a generalizing social science can be, because only under those conditions are there regularities there to be captured as knowledge and taught.

Hannah Arendt on the apoliticality of science

The next book for the Berkeley School of Information’s Classics reading group is Hannah Arendt’s The Human Condition, 1958. We are reading this as a follow-up to Sennett’s The Craftsman, working backwards through his intellectual lineage. We have the option to read other Arendt. I’m intrigued by her monograph On Violence, because it’s about the relationship between violence and power (which is an important thing to think about) and also because it’s comparatively short (~100 pages). But I’ve begun dipping into The Human Condition today only to find an analysis of the role of science in society. Of course I could not resist writing about it here.

Arendt opens the book with a prologue discussing the cultural significance of the Apollo mission. She muses at shift in human ambition that has lead to its seeking to leave Earth. Having rejected Heavenly God as Father, she sees this as a rejection of Earth as Mother. Poetic stuff–Arendt is a lucid writer, prose radiating wisdom.

Then Arendt begins to discuss The Problems with Science (emphasis mine):

While such possibilities [of space travel, and of artificial extension of human life and capabilities] still may lie in a distant future, the first boomerang effects of science’s great triumphs have made themselves felt in a crisis within the natural sciences themselves. The trouble concerns the fact that the “truths” of the modern scientific world view, though they can be demonstrated in mathematical formulas and proved technologically, will no longer lend themselves to normal expression in speech and thought. The moment these “truths” are spoken of conceptually and coherently, the resulting statements will be “perhaps not as meaningless as a ‘triangular circle,’ but much more so than a ‘winged lion'” (Erwin Schödinger). We do not yet know whether this situation is final. But it could be that we, who are earth-bound creatures and have begun to act as though we are dwellers of the universe, will forever be unable to unable to understand, that is, to think and speak about the things which nevertheless we are able to do. In this case, it would be as though our brain, which constitutes the physical, material condition of our thoughts, were unable to follow what we do, so that from now on we would indeed need artificial machines to do our thinking and speaking. If it should turn out to be true that knowledge (in the sense of know-how) and thought have parted company for good, then we would indeed become the helpless slaves, not so much of our machines as of our know-how, thoughtless creatures at the mercy of every gadget which is technically possible, no matter how murderous it is.

We can read into Arendt a Heideggerian concern about man’s enslavement of himself through technology, and equally a distrust mathematical formalism that one can also find in Horkheimer‘s Eclipse of Reason. It’s fair to say that the theme of technological menace haunted the 20th century; this is indeed the premise of Beniger‘s The Control Revolution, whose less loaded account described how the advance of technical control could be seen as nothing less or more than the continuing process of life’s self-organization.

What is striking to me about Arendt’s concerns, especially after having attended SciPy 2015, a conference full of people discussing their software code as a representation of scientific knowledge, is how ignorant Arendt is about how mathematics is used by scientists. (EDIT: The error here is mine. A skimming of the book past the prologue (always a good idea before judging the content of a book or its author…) makes it clear that this comment about mathematical formalism is not a throwaway statement at the beginning of the book to motivate a discussion of political action, but rather something derived from her analysis of political action and the history of science. Ironically, I’ve read her “speech” and interpreted it politically (in the narrow sense of implicating identities of “the scientist”, a term which she does seem to use disparagingly or distancingly elsewhere, when another more charitable reading (one that was more sensitive to how she is “technically” defining her terms (though I expect she would deny this usage)–“speech” being rather specialized for Arendt, not being merely ‘utterances’–wouldn’t be as objectionable. I’m agitated by the bluntness of my first reading, and encouraged to read further.)

On the one hand, Arendt wisely situates mathematics as an expression of know-how, and sees technology as an extension of human capacity not as something autonomous from it. But it’s strange to read her argue essentially that mathematics and technology is not something that can be discussed. This ignores the daily practice of scientists, mathematicians, and their intellectual heirs, software engineers, which involves lots of discussion about about technology. Often these discussions are about the political impact of technical decisions.

As an example, I had the pleasure of attending a meeting of the NumPy community at SciPy. NumPy is one of the core packages for scientific computing in Python which implements computationally efficient array operations. Much of the discussion hinged on whether and to what extent changes to the technical interface would break downstream implementations using the library, angering their user base. This political conflict, among other events, lead to the creation of sempervirens, a tool for collecting data about how people are using the library. This data will hopefully inform decisions about when to change the technical design.

Despite the facts of active discourse about technology in the mathematized language of technology, Arendt maintains that it is the inarticulateness of science that makes it politically dangerous.

However, even apart from these alst and yet uncertain consequences, the situation created by the sciences is of great political significance. Wherever the relevance of speech is at stake, matters become political by definition, for speech is what makes man a political being. If we would follow the advice , so frequently urged upon us, to adjust our cultural attitudes to the present status of scientific achievement, we wuld in all earnest adopt a way of life in which speech is no longer meaningful. For the sciences today have been forced to adopt a “language” of mathematical symbols which, though it was originally meant only as an abbreviation for spoken statements, now contains statements that in no way can be translated back into speech. The reason why it may be wise to distrust the political judgment of scientists qua scientists is not primarily their lack of “character”–that they did not refuse to develop atomic weapons–or their naivete–that they did not understand that once these weapons were developed they would be the last to be consulted about their use–but precisely the fact that they move in a world where speech has lost its power. And whatever men do or know or experience can make sense only to the extent that it can be spoken about. There may be truths beyond speech, and they may be of great relevance to man in the singular, that is, to man in so far as he is not a political being, whatever else he may be. Men in the plural, that is, men in so far as they live and move and act in this world, can experience meaningfulness only because they can talk with and make sense to each other and to themselves.

There is an element of truth to this analysis. But there is also a deep misunderstanding of the scientific process as one that somehow does not involve true speech. Here we find another root of a much more contemporary debate about technology in society reflected in recent concern about the power of ‘algorithms’. (EDIT: Again, after consideration, shallowly accusing Arendt of a “deep misunderstanding” at this stage is hubris. Though there does seem to be a connection between some of the contemporary debate about algorithms to Arendt’s view, it’s wrong to project historically backwards sixty years when The Human Condition is an analysis of the shifting conditions over the preceding two millennia.

Arendt claims early on that the most dramatic change in the human condition that she can anticipate is humanity’s leaving the earth to populate the universe. I want to argue that the creation of the Internet has been transformative of the human condition in a different way.)

I think it would be fair to say that Arendt, beloved a writer though she is, doesn’t know what she’s talking about when she’s talking about mathematical formalism. (EDIT: Again, a blunt conclusion. However, the role of formalism in, say, economics (though much debated) stands as a counterexample to Arendt in other ways.) And perhaps this is the real problem. When, for almost a century, theorists have tried to malign the role of scientific understanding in politics, it has been (incoherently) either on the grounds that it is secretly ideological in ways that have gone unstated, or (as for Arendt) that it is cognitively defective in a way that prevents it from participating in politics proper. (EDIT: This is a misreading of Arendt. It appears that what makes mathematical science apolitical for Arendt is precisely its universality, and hence its inability to be part of discussion about the different situations of political actors. Still, something seems quite wrong about Arendt’s views here. How would she think about Dwork’s “Fairness through awareness“?

The frustration for a politically motivated scientist is this: Political writers will sometimes mistake their own inability to speak or understand mathematical truths for their general intelligibility. On grounds of this alleged intelligibility they dismiss scientists from political discussion. They then find themselves apolitically enslaved by technology they don’t understand, and angry about it. Rather than blame their own ignorance of the subject matter, they blame scientists for being unintelligible. This is despite scientists intelligibility to each other.

An analysis of the politics of science will be incomplete without a clear picture of how scientists and non-scientists relate to each other and communicate. As far as I can tell, such an analysis is almost impossible politically speaking because of the power dynamic of the relation. Professional non-scientific intellects are loathe to credit scientists with an intellectual authority that they feel that they are not able to themselves attain, and scientific practice requires adhering to standards of rigor which give one greater intellectual authority; these standards by their nature require ahistorical analysis, dismissal of folk theorizing, etc. It has become politically impossible to ground an explanation of a social phenomenon on the basis that one population is “smarter” than another, despite this being a ready first approximation and one that is used in practice by the vast majority of people in private. Hence, the continuation of the tradition of treatises putting science in its place.

Follow

Get every new post delivered to your Inbox.

Join 1,033 other followers