Digifesto

There has been a lot of drama on the Internet about AI ethics, and that drama is in the mainstream news. A lot of this drama is about the shakeup in Google’s AI Ethics team. I’ve already written a little about this. I’ve been following the story since, and I would adjust my emphasis slightly. I think the situation is sadder than I did at first. But my take (and I’m sticking to it) is that what’s missing from the dominant narrative in that story, that it is about race and gender representation in tech, (a view quite well articulated in academic argot by Tao and Varshney (2021), by the way) is an analysis of the corporate firm as an organization, what it means to be an employee of a firm, and the relationship between firms and artificial intelligence.

These questions about the corporate firm are not topics that get a lot of traction on the Internet. The topics that get a lot of traction on the Internet are race and gender. I gather that this is something anybody with a blog, (let alone publishing news), discovers. I write this blog for an imagined audience of about ten people. It is mainly research notes to myself. But for some reason, this post about racism got 125 views this past week. For me, that’s a lot of views. Who are those people?

The other dramatic thing going in my corner of the Internet this past week is also about race and gender, and also maybe AI ethics. It is the reaction to Cade Metz’s NYT article “Silicon Valley’s Safe Space“, which is about the blog Slate Star Codex (SSC), rationalist subculture, libertarianism and … racism and sexism.

I learned about this kerfuffle in a backwards way. I learned about it through Glen Weyl’s engagement with SSC about technocracy, which I suppose he was bumping to ride on the shockwave created by the NYT article. In 2019, Weyl posted a critique of “technocracy” which was also a rather pointed attack on the rationalism community, in part because of its connection to “neoreaction”. SSC responded rather adroitly, in my opinion; Weyl responded. It’s an interesting exchange about political theory.

I follow Weyl on Twitter because I disagree with him rather strongly on niche topics in data regulation. As a researcher, I’m all about these niche topics in data regulation; I think they are where the rubber really hits the road on AI ethics. Weyl has published the view that consumer internet data should be treated as a form of labor. In my corner of technology policy research, which is quite small in terms of its Internet footprint, we think this is nonsense; it is what Salome Viljoen calls a propertarian view of data. The propertarian view of data is, for many important reasons, wrong. “Markets are never going to fix the AI/data regulation problem. Go read some Katharina Pistor, for christ’s sake!” That’s the main thing I’ve been waiting to say to Weyl, who has a coveted Microsoft Research position, a successful marketing strategy for his intellectual brand, perfect institutional pedigree, and so on.

Which I mean to his credit: he’s a successful public intellectual. This is why I was surprised to see him tweeting about rationalist subculture, which is not, to me, a legitimate intellectual topic, sanctioned as part of the AI/tech policy/whatever research track. My experiences with rationalism have all been quite personal and para-academic. It is therefore something of a context collapse — the social media induced bridging of social spheres — for me personally. (Marwick and Boyd, 2011; Davis and Jurgenson, 2014)

---

Context collapse is what it’s all about, isn’t it? One of the controversies around the NYT piece about SSC was that the NYT reporter, Metz, was going to reveal the pseudonymous author of SSC, “Scott Alexander”, as the real person Scott Siskind. This was a “doxxing”, though the connection was easy to make for anybody looking for it. Nevertheless, Siskind initially had a strong personal sense of his own anonymity as a writer. Siskind is, professionally, a psychotherapist, which is a profession with very strong norms around confidentiality and its therapeutic importance. Metz’s article is about this. It is also about race and gender — in particular the ways in which there is a therapeutic need for a space in which to discuss race and gender, as well as other topics, seriously, but also freely, without the major social and professional consequences that have come to be associated with it.

Smart people are writing about this and despite being a “privacy scholar”, I’m not sure I can say much that is smarter. Will Wilkinson’s piece is especially illuminating. His is a defense of journalism and a condemnation of what, I’ve now read, was an angry Internet mob that went after Metz in response to this SSC doxxing. It is also an unpacking of Siskind’s motivations based on his writing, a diagnosis of sorts. Spiers has a related analysis. A theme of these arguments against SSC and the rationalists is, “You weren’t acting so rationally this time, were you?”

I get it. The rationalists are, by presenting themselves at times as smarter-than-thou, asking for this treatment. Certainly in elite intellectual circles, the idea that participation in a web forum should give you advanced powers of reason that are as close as we will ever get to magic is pretty laughable. What I think I can say from personal experience, though, is that elite intellectuals seriously misunderstand popular rationalism if they think that it’s about them. Rather, popular rationalism is a non-elite social movement that’s often for people with non-elite backgrounds and problems. Their “smarter-than-thou” was originally really directed at other non-elite cultural forms, such as Christianity (!). I think this is widely missed.

I say this with some anecdata. I lived in Berkeley for a while in graduate school and was close with some people who were bona fide rationalists. I had an undergraduate background from an Ivy League university in cognitive science and was familiar with the heuristics and biases program and Bayesian reasoning. I had a professional background working in software. I should have fit in, right? So I volunteered twice at the local workshop put on by the Center for Applied Rationality to see what it was all about.

I noticed, as one does, that it’s mostly white men at these workshops and when asked for feedback I pointed this out. Eager to make use of the sociological skills I was learning in grad school, I pointed out that if they did not bring in more diversity early on, then because of homophily effects they may never reach a diverse audience.

At the time, the leadership of CFAR told me something quite interesting. It was that they had looked at their organizational goals and capacities and decided that where they could make the most impact was on teaching the skills of rational thought to smart people from, say, the rural midwest U.S.A. who would otherwise not get the exposure to this kind of thinking or what I would call a community of practice around it. Many of these people (much like Elizabeth Spiers, according to her piece) come from conservative and cloistered Christian backgrounds. Yudkowsky’s Harry Potter and the Methods of Rationality is their first exposure to Bayesian reasoning. They are often the best math students in their homogeneous home town, and finding their way into an engineering job in California is a big deal, as is finding a community that fills an analogous role to organized religion but does not seem so intellectually backwards. I don’t think it’s accidental the Julia Galef, who founded CFAR, started out in intellectual atheist circles before becoming a leader in rationalism. Providing an alternative culture to Christianity is largely what popular rationalism is about.

From this perspective, it makes more sense why Siskind has been able to cultivate a following by discussing cultural issues from a centrist and “altruistic” perspective. There’s a population in the U.S. that grew up in conservative Christian settings, now makes a living in a booming technology sector whose intellectual principles are at odds with those of their upbringing, is trying to “do the right thing” and, being detached from political institutions or power, turns to the question of philanthropy, codified into Effective Altruism. This population is largely comprised of white guys who may truly be upwardly mobile because they are, relative to where they came from, good at math. The world they live in, which revolves around AI, is nothing like the one they grew up in. These same people are regularly confronted by a different ideology, a form of left wing progressivism, which denies their merit, resents their success, and considers them a problem, responsible for the very AI harms that they themselves are committed to solving. If I were one of them, I, too, would want to be part of a therapeutic community where I could speak freely about what was going on.

---

This is several degrees removed from libertarian politics, which I now see as the line connecting in Weyl. Wilkinson makes the compelling case for contemporary rationalism originating in Tyler Cohen’s libertarian economist blogging and the intellectual environment at George Mason University. This spins out Robin Hanson’s Overcoming Bias blog, which spins out Yudkowsky’s LessWrong forum, which is where popular rationalism incubated. Weyl is an east coast libertarian public intellectual and it makes sense that he would engage other libertarian public intellectuals. I don’t think he’s going to get very far picking fights on the Internet with Yudkowsky, but I could be wrong.

Weyl’s engagement with the rationalist community does highlight for me two other missing elements in the story-as-told-so-far, from my readings on it. I’ve been telling a story partly about geography and migration. I think there’s also an element of shifting centers of cultural dominance. Nothing made me realize that I am a parochial New Yorker like living in California for five years. Rationalism remains weird to me because it is, today, a connection between Oxford utilitarian philosophers, the Silicon Valley nouveau riche, and to some extent Washington, D.C.-based libertarians. That is a wave of culture bypassing the historical intellectual centers of northeast U.S. Ivy League universities which for much of America’s history dominated U.S. politics.

To some extent, this speaks to the significance of the NYT story as well. It was not the first popular article about rationalists; Metz mentions the TechCrunch article about neoreactionaries (I’ll get to that) but not the Sam Frank article in Harpers, “Come With Us If You Want To Live” (2015), which is more ethnographic in its approach. I think it’s a better article. But the NYT has a different audience and a different standard for relevance. NYT is not an intellectual literary magazine. It is the voice of New York City, once the Center of the Universe. New York City’s perspective is particularly weighty, relevant, objective, powerful because of its historic role as a global financial center and marketing center. When the NYT notices something, for a great many people, it becomes real. NYT is at the center of a large public sphere with a specific geographic locus, in a way that some blogs and web forums are not. So whether it was justified or not, Metz’s doxing of Siskind was a significant shift in what information was public, and to whom. Part of its significance is that it was an assertion of cultural power by an institution tied to old money in New York City over a beloved institution of new money in Silicon Valley. In Bourdieusian terms, the article shifted around social and cultural capital in a big way. Siskind was forced to make a trade by an institution more powerful than him. There is a violence to that.

---

This force of institutional power is perhaps the other missing element in this story. Wilkinson and Frank’s pieces remind me: this is about libertarianism. Weyl’s piece against technocracy is also about libertarianism, or maybe just liberalism. Weyl is arguing that rationalists, as he understands them, are libertarians but not liberals. A “technocrat” is somebody who wants to replace democratic governance mechanisms, which depend on pluralistic discourse, with an expert-designed mechanism. Isn’t this what Silicon Valley does? Build Facebook and act like it’s a nation? Weyl, in my reading, wants an engaged pluralistic public sphere. He is, he reveals later, really arguing with himself, reforming his own views. He was an economist, coming up with mathematical mechanisms to improve social systems through “radical exchange”; now he is a public intellectual who has taken a cultural turn and called AI an “ideology”.

On the other end of the spectrum, there are people that actually would, if they could, build an artificial island and rule it via computers like little lords. I guess Peter Thiel, who plays a somewhat arch-villain role in this story, is like this. Thiel does not like elite higher education and the way it reproduces the ideological conditions for a pluralistic democracy. This is presumably why he backs Curtis Yarvin, the “neoreactionary” writer and “Dark Enlightenment” thinker. Metz goes into detail about this, and traces a connection between Yarvin and SSC; there’s leaked emails about it. To some people, this is the real story. Why? Because neoreaction is racist and sexist. This, not political theory, I promise you, is what is driving the traffic. It’s amazing Metz didn’t use the phrase “red pill” or “alt-right” because that’s definitely the narrative being extended here. With Trump out of office and Amazon shutting down Parler’s cloud computing, we don’t need to worry about the QAnon nutcases (which were, if I’m following correctly, a creation of the Mercers) but what about the right wing elements in the globally powerful tech sector, because… AI ethics! There’s no escape.

Slate Star Codex was a window into the Silicon Valley psyche. There are good reasons to try and understand that psyche, because the decisions made by tech companies and the people who run them eventually affect millions.

And Silicon Valley, a community of iconoclasts, is struggling to decide what’s off limits for all of us.

At Twitter and Facebook, leaders were reluctant to remove words from their platforms — even when those words were untrue or could lead to violence. At some A.I. labs, they release products — including facial recognition systems, digital assistants and chatbots — even while knowing they can be biased against women and people of color, and sometimes spew hateful speech.

Why hold anything back? That was often the answer a Rationalist would arrive at.

Metz’s article has come under a lot of criticism for drawing sweeping thematic links between SSC, neoreaction, and Silicon Valley with very little evidence. Noah Smith’s analysis shows how weak this connection actually is. Silicon Valley is, by the numbers, mostly left-wing, and mostly, by the numbers, not reading rationalist blogs. Thiel, and maybe Musk, are noteworthy exceptions, not the general trend. What does any of this have to do with, say, Zuckerburg? Not much.

The trouble is that if the people in Silicon Valley are left-wing, then there’s nobody to blame for racist and sexist AI. Where could racism and sexism in AI possibly come from, if not some collective “psyche” of the technologists? Better, more progressive leaders in Silicon Valley, the logic goes, would lead to better social outcomes. Pluralistic liberalism and proper demographic representation would, if not for the likes of bad apples like Thiel, steer the AI Labs and the big tech companies that use their products towards equitability and justice.

I want to be clear: I think that affirmative action for under-represented minorities (URMs) in the tech sector is a wonderful thing, and that improving corporate practices around their mentorship, etc. is a cause worth fighting for. I’m not knocking any of that. But I think the idea that this alone will solve the problems of “AI ethics” is a liberal or libertarian fantasy. This is because assuming that the actions of a corporation will have the politics of its employees is a form of ecological fallacy. Corporations do not work for their employees; they work, legally and out of fiduciary duty, to their shareholders. And the AI operated at the grand social scales that we are talking about are not controlled by any one person; they are created and operated corporately.

In my view, what Weyl (who I probably agree with more than I don’t), the earlier libertarian bloggers like Hanson, the AI X-risk folks like Yudkowsky, and the popular rationalist movement all get wrong is the way institutional power necessarily exceeds that of individuals in part because and through of “artificial intelligence”, but also through older institutions that distribute economic and social capital. The “public sphere” is not a flat or radical “marketplace of ideas”; it is an ecology of institutions like the New York Times, playing on ideological receptiveness grounded in religious and economic habitus.

---

Jeffrey Friedman is a dedicated intellectual and educator, who for years has been a generous intellectual mentor and facilitator of political thought. Friedman, like Weyl, is a critic of technocracy. In an early intellectual encounter that was very formative for me, he invited me to write a book review of Philip Tetlock Expert Political Judgment for his journal. In 2007, it was my first academic publication. The writing is embarrassing and I’m glad it is behind a paywall. In the article, I argue against Friedman and for technocracy based on the use of artificial intelligence. I have been in friendly disagreement on this point with Friedman ever since.

The line of reasoning from Friedman’s book, Power without Knowledge: A Critique of Technocracy (2019), that I find most interesting is about the predictability of individuals. Is it possible for a technocrat to predict society? This question has been posed by many different social theorists. I believe Friedman is unique in suggesting that (a) individuals cannot be predicted by a technocrat because (b) of how individual behavior is determined by their ideas, which are sourced so variously and combined in such complex ways that they cannot be captured by an external or generalizing observer. The unpredictability of society is an objective obstacle to the social scientific prediction that is required for technocratic policies to function effectively and as intended. Instead of a technocracy, Friedman advocates for an “Exitocracy”, based on the idea of Exit from Hirschman, which prioritizes a robust private sector in which citizens can experiment and find happiness, over technocratic or what others might call paternalist public policy. Some of the attractiveness of this model is that it depends on minimal assumptions about the rationality of agents, and especially the agency of technocrats, but still achieves satisficing results. Friedman’s exitocracy is, he argues, a ‘judicious’ technocracy, calibrated to realistic levels of expertise and public ignorance. In allowing for redistribution and centralized governance of some public goods, Friedman’s exitocracy stands as an alternative to more radically libertarian “Exit”-oriented proposals such as those of Srinivasan, which have been associated with Silicon Valley and, by dubious extension, the rationalist community.

At this time, I continue to disagree with Friedman. Academics are particularly unpredictable with respect to their ideas, especially if they do not value prestige or fame as much as their other colleagues. But most people are products of their background, or their habits, or their employers, or their social circles, or their socially structured lived experience. Institutions can predict and control people, largely by offering economic incentives. This predictability is what has made AI effective commercially — in uses by advertisers, for example — and it was what makes centralize public policy, or technocracy, possible.

But I could be wrong about this point. It is an empirical question how and under what conditions people’s beliefs and actions are unpredictable given background conditions. This amount of variability, which we might consider a kind of “freedom”, is a condition for whether technocracy, and AI, is viable or not. Are we free?

References

Davis, J. L., & Jurgenson, N. (2014). Context collapse: Theorizing context collusions and collisions. Information, communication & society, 17(4), 476-485.

Friedman, J. (2019). Power without knowledge: a critique of technocracy. Oxford University Press.

Kollman, K., Miller, J. H., & Page, S. E. (1997). Political institutions and sorting in a Tiebout model. The American Economic Review, 977-992.

Marwick, A. E., & Boyd, D. (2011). I tweet honestly, I tweet passionately: Twitter users, context collapse, and the imagined audience. New media & society, 13(1), 114-133.

Pistor, K. (2020). Rule by data: The end of markets?. Law & Contemp. Probs., 83, 101.

Tao, Y., & Varshney, K. R. (2021). Insiders and Outsiders in Research on Machine Learning and Society. arXiv preprint arXiv:2102.02279.

Lyotard’s 1979 treatise The Postmodern Condition tells a prescient story about the transformation of the university. He discusses two “metanarratives” used for the organization of universities: the German Humboldt model of philosophy as the central discipline, with all other fields of knowledge radiating out from it; and the French model of the university as the basis of education of the modern democratic citizen. Lyotard argues (perhaps speciously) that because of what the late Wittgenstein had to say about the autonomy of language games (there are no facts; there are only social rules) and because of cybernetics (the amalgamation of exact and applied sciences that had been turned so effectively towards control of human and machine), the metanarratives had lost their legitimacy. There was only “legitimation by performativity”, knowledge proving itself by virtue of its (technical) power, and “legitimiation by paralogy”, knowledge legitimizing itself through semantic disruption, creating pools of confusion in which one could still exist though out-of-alignment with prevailing cybernetic logics.

This duality–between cybernetics and paralogy–excludes a middle term identified in Habermas’s 1968 Knowledge and the Structure of Human Interests. Habermas identifies three “human interests” that motivate knowledge: the technical interest (corresponding to cybernetic performativity), the emancipatory interest (perhaps corresponding to the paralogic turn away from cybernetic performativity), and, thirdly, the hermeneutic interest. The latter is the interest in collective understanding that allows for collective understanding. As Habermas’s work matures, this interest emerges as the deliberative, consensual basis of law.

These frameworks for understanding knowledge and the university share an underlying pragmatism. Both Lyotard and Habermas seem to agree about the death of the Humboldt model: knowledge for its own sake is a deceased metanarrative. Knowledge for democratic citizens, the purportedly French model in Lyotard, was knowledge of shared historical narratives and agreement about norms for Habermas. Lyotard was pessimistic about the resilience of these kinds of norms under the pressure of cybernetics. Indeed, this tension between “smart technology” and “rule of law” remains today, expressed in the work of Hildebrandt. The question of whether technical knowledge threatens or delegitimizes legal/hermeneutic knowledge is still with us today.

These intellectual debates are perhaps ultimately about university politics and academic disciplines. If they are truly _ultimately_ about that, that marks their limitation. For what the pragmatist orientation towards knowledge implies is that knowledge does not exist for its own sake, but rather, in most cases, for its application. Philosophers can therefore only achieve so much by appealing to generalized interests. All real applications are contextualized.

Two questions unanswered by these sources (at least in what is assuredly this impoverished schematic of their arguments) are:

• Whence the interests and applications that motivate the university as socially and economically situated?
• What accounts for the tensions between the technical/performative disciplines and the hermeneutic and emancipatory ones?

In 1979, the same publication year of The Postmodern Condition, Pierre Bourdieu published Distinction: A Social Critique of the Judgement of Taste. While not in itself an epistemology, Bourdeiu’s method and conclusions provide a foundation for later studies of science, journalism, and the university. Bourdieu’s insight is that aesthetic taste–in art, in design, in hobbies, etc.–is a manifestation of socioeconomic class understood in terms of a multidimensional matrix of forms of capital–such as economic wealth, but also social status and prestigue, and social capital in knowledge and skills. Those with lots of wealth and low cultural capital–the nouveau riche–will value expensive, conspicuous consumption. Those with low wealth and high cultural capital–academics, perhaps–will value intricate works that require time and training to understand and so on. But these preferences exist to maintain the social structures of (multiply defined) capital accumulation.

A key figure in Bourdieu’s story is that of the petit bourgeoisie, the transitional middle class that has specialized their labor, created perhaps a small business, but has not accumulated capital in a way that secures them in the situation where they aspire to be. In today’s economy, these might include the entrepreneurs–those who would, by their labor, aspirationally transform themselves from laborers into capitalists. They would do this by the creation of technology–the means of productions, capital. Unlike labor applied directly to the creation of goods and services as commodities, capital technologies, commodified through the institution of intellectual property, have the potential to scale in use well beyond the effort of their creation and, through Schumpeterian disruption, make their creators wealthy enough to change their class position. On the other hand, there are those who prefer the academic lifestyle, who luxuriate in the study of literature and critique. Through the institutions of critical academia, these are also jobs that can be won through the accumulation of, in this case social and cultural, capital. By design, these are fields of knowledge that exist for their own sake. There are also, of course, law and social scientific disciplines that are helpful for the cultural formation of politicians, legislators, and government workers of various kinds.

Viewed in this way, we can start to see “human interests” not merely as transcendental features the general human condition, but rather as the expression of class and capital interests. This makes sense given the practical reality of universities getting most of their income through tuition. Students attend universities in order to prepare themselves for careers. The promise of a professional career allows universities to charge higher tuition. Where in the upper classes people choose to compete on intangible cultural capital rather than economic capital, universities maintain specialized disciplinary tracks in the humanities.

Notably, the emancipatory role of the humanities, lauded by Habermas, subtly lampooned (parhaps) by Lyotard, is in other works more closely connected to leisure. As early as 1947, Horkheimer, in Eclipse of Reason, points out that the kind of objective reason he sees as essential to the moral grounding of society that has been otherwise derailed by capitalism relies on leisure time that this a difficult class attainment. In perhaps cynical Bourdieusian terms, the ability to reflect on the world and decide, beyond the restrictions of material demands, on an independent or transcendent system of values is itself a form of cultural accumulation of the most rarified kind. However, as this form of cultural attainment is not connected directly to any means of production, it is perhaps a mystery what grounds it pragmatically.

There’s an answer. It’s philanthropy. The arts and humanities, the idealistic independent policy think tanks, and so on, are funded by those who, having accumulated economic capital and the capacity for leisurely thinking about the potential for a better word, have allocated some portion of their wealth towards “causes”. The competition for legitimacy between and among philanthropic causes is today a major site of politics and ideology. Most obviously, political parties and candidacy run on donations, which is in a sense a form of values-driven philanthropy. The appropriation of state funds, or not, for particular causes becomes a battlefield of all forms of capital at the end of the day

This is all understandable from the perspective that is now truly at the center of the modern university: the perspective of business administration. Ever since Herbert Simon, it has been widely known that the managerialist discipline and computational and cybernetic sciences are closely aligned. The economic sociology of Bourdieu is notable in that it is a successor to the sociology of Marx, but also a successor to the phenomenological approach of Kant, and yet is ultimately consistent with the managerialist view of institutions relying on skilled capital management. Disciplines or sub-disciplines that are peripheral to these core skillsets by virtue of their position in the network of capital flows are marginal by definition.

This accounts for much of interdisciplinary politics and grievance. The social structures described here account for the teleological dependency structure of different forms of knowledge: what it is possible to motivate, and with what. To the extent that a discipline as a matter of methodological commitment is unable to account for this social structure, it will be dependent on its own ability to perpetuate itself autonomously though the stupefication of its students.

There is another form of disciplinary dependency worth mentioning. It cuts the other way: it is the dependency that arises from the infrastructural needs of the knowledge institutions. This instrumental dependency is where this line of reasoning connects with Ihde’s instrumental realism as a philosophy of science. Here, too, there are disciplines that are blind to themselves. To the extent that a discipline is unable to account for the scientific advances necessary for its own work, it survives through the heroics of performative contradiction. There may be cases where an institution has developed enough teleological autonomy to reject the knowledge behind its own instrumentation, but in these cases we must be tempted to consider the knowledge claims of the former to be specious and pretensious. What purpose does fashionable nonsense have, if it rejects the authority of those that it depends on materially? “Those” here referring to those classes that embody the relevant infrastructural knowledge.

The answer is perhaps best addressed using the Bourdieusian insights already addressed: an autonomous field of discourse that denies its own infrastructure is a cultural market designed to establish a distinct form of capital, an expression of leisure. The rejection of performativity, or tenuous and ambiguous connection to it, becomes a class marker; synecdochal with leisure itself, which can then be held up as an esteemable goal. Through Lyotard’s analysis, we can see how a field so constructed might be successful through the rhetorical power of its own paralogic.

What has been lost, through this process, is the metanarrative of the university, most especially of the university as an anchor of knowledge in itself. The pragmatist cybernetic knowledge orientation entails that the university is subsumed to wider systems of capital flows, and the only true guarantee of its autonomy is philanthropic endowment which might perpetuate its ability to develop a form of capital that serves its own sake.

I’ve recently come across an interesting paper published at Scipy 2019, Dusen et al.’s “Accelerating the Advancement of Data Science Education” (2019) (link). It summarizes recent trends in data science education, as modeled by UC Berkeley’s Division of Data Science, which is now the Division of Computing, Data Science, and Society (CDSS). This is a striking piece to me as I worked at Berkeley on its data science capabilities several years ago and continue to be fascinated by my alma mater, the School of Information, as it navigates being part of CDSS.

Among other interesting points in the article, two are particularly noteworthy to me. The first is that the integration of data science into the social sciences appears to have continued apace. The article mentions that data science’s integration into the social science has continued apace. Economics, in particular, is well represented and supported in the extended data science curriculum.

The other interesting point is the emphasis on data science ethics as an essential pillar of the educational program. The writing in this piece is consistent with what I’ve come to expect from Berkeley on this topic, and I believe it’s indicative of broad trends in academia.

The authors of this piece are explicit about their “theory of change”. What is data science ethics education supposed to accomplish?

Including training in ethical considerations at all levels of society and all steps of the data science workflow in undergraduate data science curricula could play an important role in stimulating change in industry as our students enter the workforce, perhaps encouraging companies to add ethical standards to their mission statements or to hire chief ethics officers to oversee not only day-to-day operations but also the larger social consequences of their work.

The theory of change articulated by the paper is that industry will change if ethically educated students enter the workforce. They see a future where companies change their mission statements in accord with what has been taught in data science ethics courses, or hire oversight officials.

This is, it must be noted, broadly speculative, and implies that the leadership of the firms who hire these Berkeley grads will be responsive to their employees. However, unlike in some countries in Europe, the United States does not give employees a lot of say in the governance of firms. Technology firms, such as Amazon and Google, have recently proven to be rather unfriendly to employees that attempt to organize in support of “ethics”. This is for highly conventional reasons: the management of these firms tends to be oriented towards the goal of maximizing shareholder profits, and having organized employees advocating for ethical issues that interfere with business is an obstacle to that goal.

This would be understood plainly if economics, or economic history, was taught as part of “data science ethics”. But it’s not for some reason. Information economics, which would presumably be where one would start to investigate the way incentives drive data science institutions, is perhaps too complex to be included in the essential undergraduate curriculum, despite its being perhaps critical to understanding the “data intensive” social world we all live in now.

We forget today, often, that the original economists (Adam Smith, Alfred Marshall, etc.) were all originally moral philosophers. Economics has begun to be seen as a field designed to be in instrumental support of business practice or ideology rather than an investigation into the ethical consequences of social and material structure. That’s too bad.

Instead of teaching economic history, which would be a great way of showing students the ethical implications of technology, instead Berkeley is teaching Science and Technology Studies (STS) and algorithmic fairness! I’ll quote at length:

A recent trend in incorporating such ethical practices includes
incorporating anti-bias algorithms in the workplace. Starting from
the beginning of their undergraduate education, UC Berkeley students can take History 184D: Introduction to Science, Technology, and Society: Human Contexts and Ethics of Data, which covers the implications of computing, such as algorithmic bias. Additionally, students can take Computer Science 294: Fairness in Machine Learning, which spends a semester in resisting racial, political, and physical discrimination. Faculty have also come together to create the Algorithmic Fairness and Opacity Working Group at Berkeley’s School of Information that brainstorms methods to improve algorithms’ fairness, interpretability, and accountability. Implementing such courses and interdisciplinary groups is key to start the conversation within academic institutions, so students
can mitigate such algorithmic bias when they work in industry or
academia post-graduation.

Databases and algorithms are socio-technical objects; they emerge and evolve in tandem with the societies in which they operate [Latour90]. Understanding data science in this way and recognizing its social implications requires a different kind of critical thinking that is taught in data science courses. Issues such as computational agency [Tufekci15], the politics of data classification and statistical inference [Bowker08], [Desrosieres11], and the perpetuation of social injustice through algorithmic decision making [Eubanks19], [Noble18], [ONeil18] are well known to scholars in the interdisciplinary field of science and technology
studies (STS), who should be invited to participate in the development of data science curricula. STS or other courses in the social sciences and humanities dealing specifically with topics related to data science may be included in data science programs.

This is all very typical. The authors are correct that algorithmic fairness and STS have been trendy ways of teaching data science ethics. It is perhaps too cynical to say that these are trendy approaches to “data science ethics” because they are the data science ethics that Microsoft will pay for. Let that slip as a joke.

However, it is unfortunate if students have no better intellectual equipment for dealing with “data science ethics” than this. Algorithmic fairness is a fascinating field of study with many interesting technical results. However, as has been broadly noted by STS scholars, among others, the successful use of “algorithmic fairness” technology depends on the social context in which it is deployed. Often, “fairness” is achieved through greater scientific and technical integrity: for example, properly deducing cause and effect rather than lazily applying techniques that find correlation. But the ethical challenges in the workplace are often not technical challenges. They are the challenges of managing the economic incentives of the firm, and how these effect the power structures within the firm. (Metcalf et al., 2019) This is apparently not material that is being taught at Berkeley to data science students.

This more careful look at the social context in which technology is being used is supposed to be what STS is teaching. But, all too often, this is not what it’s doing. I’ve written elsewhere why STS is not the solution to “tech ethics”. Part of (e.g. Latourian) STS training is a methodological, if not intellectual, relativistic skepticism about science and technology itself (Carroll, 2006). As a consequence, it requires, of itself, to be a humanistic or anthropological field, using “interpretivist” methods, with weak claims to generalizability. It is, first and foremost, an academic field, not an applied one. The purpose of STS is to generate fascinating critiques.

There are many other social sciences that have different aims, such as the aim of building consensus around what social and economic conditions are in order to motivate political change. These social sciences have ethical import. But they are built around a different theory of change. They are aimed at the student as a citizen in a democracy, not as an employee at a company. And while I don’t underestimate the challenges of advocating for designing education to empower students as public citizens in this economic climate, it must nevertheless be acknowledge, as an ethical matter, that a “data science ethics” curriculum that does not address the politics behind those difficulties will be an anemic one, at best.

There is a productive way forward. It requires, however, interdisciplinary thinking that may be uncomfortable or, in the end, impossible for many established institutions. If students are taught a properly historicized and politically substantive “data science ethics”, not in the mode of an STS-based skepticism about technology and science, but rather as economic history that is informed by data science (computational and inferential thinking) as an intellectual foundation, then ethical considerations would need not be relegated to a hopeful afterthought invested in a theory of corporate change that is ultimately a fantasy. Rather, it would put “data science ethics” on a scientific foundation and help civic education justify itself as a matter of social fact.

Addendum: Since the social sciences aren’t doing this work, it looks like some computer scientists are doing it instead. This report by Narayanan provides a recent economic history of “dark patterns” since the 1970’s–an example of how historical research can put “data science ethics” in context.

References

Carroll, P. (2006). Science of Science and Reflexivity. Social Forces, 85(1), 583-585.

Metcalf, J., & Moss, E. (2019). Owning Ethics: Corporate Logics, Silicon Valley, and the Institutionalization of Ethics. Social Research: An International Quarterly, 86(2), 449-476.

Van Dusen, E., Suen, A., Liang, A., & Bhatnagar, A. (2019). Accelerating the Advancement of Data Science Education. Proceedings of the 18th Python in Science Conference (SciPy 2019)

On Sunday, New York State is closing all non-essential brick-and-mortar businesses and ordering all workforce who are able to work from home. Zoom meetings from home are now the norm for people working for both the private sector and government.

One might reasonably want to know whether the internet service providers (ISP) are operating normally during this period. I had occasion to call up Optimum yesterday and ask. I was told, very helpfully, “Were doing business as usual because we are like a utility.”

It’s quite clear that the present humane and responsible approach to COVID-19 depends on broad and uninterrupted access to the Internet to homes. The government and businesses would cease to function without it. Zoom meetings are performing the role that simple audio telephony once did. And executive governments are recognizing this as they use their emergency powers.

There has been a strain of “technology policy” thought that some parts of “the tech sector” should be regulated as utilities. In 2015, the FCC reclassified broadband access as a utility as part of their Net Neutrality decision. In 2018, this position was reversed. This was broadly seen as a win for the telecom companies.

One plausible political consequence of COVID-19 is the reconsideration of the question of whether ISPs are utilities or not. They are.

Quickly responding to Selbst, Elish, and Latonero’s “Accountable Algorithmic Futures“, Data and Society’s response to the Algorithmic Accountability Act of 2019…

The bill would empower the FTC to do “automated decision systems impact assessment” (ADSIA) of automated decision-making systems. The article argues that the devil is in the details and that the way the FTC goes about these assessments will determine their effectiveness.

The point of their article, which I found notable, is to assert the appropriate intellectual discipline for these impact assessments.

This is where social science comes in. To effectively implement the regulations, we believe that engagement with empirical inquiry is critical. But unlike the environmental model, we argue that social sciences should be the primary source of information about impact. Ethnographic methods are key to getting the kind of contextual detail, also known as “thick description,” necessary to understand these dimensions of effective regulation.

I want to flag this as weird.

There is an elision here between “the social sciences” and “ethnographic methods” here, as if there were no social sciences that were not ethnographic. And then “thick description” is implied to be the only source of contextual detail that might be relevant to impact assessments.

This is a familiar mantra, but it’s also plainly wrong. There’s many disciplines and methods within “the social sciences” that aren’t ethnographic, and many ways to get at contextual detail that does not involve “thick description”. There is a worthwhile and interesting intellectual question: what are the appropriate methods for algorithmic impact assessment. The authors of this piece assume an answer to that question without argument.