Digifesto

Tag: capitalism

Reading O’Neil’s Weapons of Math Destruction

I probably should have already read Cathy O’Neil’s Weapons of Math Destruction. It was a blockbuster of the tech/algorithmic ethics discussion. It’s written by an accomplished mathematician, which I admire. I’ve also now seen O’Neil perform bluegrass music twice in New York City and think her band is great. At last I’ve found a copy and have started to dig in.

On the other hand, as is probably clear from other blog posts, I have a hard time swallowing a lot of the gloomy political work that puts the role of algorithms in society in such a negative light. I encounter is very frequently, and every time feel that some misunderstanding must have happened; something seems off.

It’s very clear that O’Neil can’t be accused of mathophobia or not understanding the complexity of the algorithms at play, which is an easy way to throw doubt on the arguments of some technology critics. Yet perhaps because it’s a popular book and not an academic work of Science and Technology Studies, I haven’t it’s arguments parsed through and analyzed in much depth.

This is a start. These are my notes on the introduction.

O’Neil describes the turning point in her career where she soured on math. After being an academic mathematician for some time, O’Neil went to work as a quantitative analyst for D.E. Shaw. She saw it as an opportunity to work in a global laboratory. But then the 2008 financial crisis made her see things differently.

The crash made it all too clear that mathematics, once my refuge, was not only deeply entangled in the world’s problems but also fueling many of them. The housing crisis, the collapse of major financial institutions, the rise of unemployment–all had been aided and abetted by mathematicians wielding magic formulas. What’s more, thanks to the extraordinary powers that I loved so much, math was able to combine with technology to multiply the chaos and misfortune, adding efficiency and scale to systems I now recognized as flawed.

O’Neil, Weapons of Math Destruction, p.2

As an independent reference on the causes of the 2008 financial crisis, which of course has been a hotly debated and disputed topic, I point to Sassen’s 2017 “Predatory Formations” article. Indeed, the systems that developed the sub-prime mortgage market were complex, opaque, and hard to regulate. Something went seriously wrong there.

But was it mathematics that was the problem? This is where I get hung up. I don’t understand the mindset that would attribute a crisis in the financial system to the use of abstract, logical, rigorous thinking. Consider the fact that there would not have been a financial crisis if there had not been a functional financial services system in the first place. Getting a mortgage and paying them off, and the systems that allow this to happen, all require mathematics to function. When these systems operate normally, they are taken for granted. When they suffer a crisis, when the system fails, the mathematics takes the blame. But a system can’t suffer a crisis if it didn’t start working rather well in the first place–otherwise, nobody would depend on it. Meanwhile, the regulatory reaction to the 2008 financial crisis required, of course, more mathematicians working to prevent the same thing from happening again.

So in this case (and I believe others) the question can’t be, whether mathematics, but rather which mathematics. It is so sad to me that these two questions get conflated.

O’Neil goes on to describe a case where an algorithm results in a teacher losing her job for not adding enough value to her students one year. An analysis makes a good case that the cause of her students’ scores not going up is that in the previous year, the students’ scores were inflated by teachers cheating the system. This argument was not consider conclusive enough to change the administrative decision.

Do you see the paradox? An algorithm processes a slew of statistics and comes up with a probability that a certain person might be a bad hire, a risky borrower, a terrorist, or a miserable teacher. That probability is distilled into a score, which can turn someone’s life upside down. And yet when the person fights back, “suggestive” countervailing evidence simply won’t cut it. The case must be ironclad. The human victims of WMDs, we’ll see time and again, are held to a far higher standard of evidence than the algorithms themselves.

O’Neil, WMD, p.10

Now this is a fascinating point, and one that I don’t think has been taken up enough in the critical algorithms literature. It resonates with a point that came up earlier, that traditional collective human decision making is often driven by agreement on narratives, whereas automated decisions can be a qualitatively different kind of collective action because they can make judgments based on probabilistic judgments.

I have to wonder what O’Neil would argue the solution to this problem is. From her rhetoric, it seems like her recommendation must be prevent automated decisions from making probabilistic judgments. In other words, one could raise the evidenciary standard for algorithms so that they we equal to the standards that people use with each other.

That’s an interesting proposal. I’m not sure what the effects of it would be. I expect that the result would be lower expected values of whatever target was being optimized for, since the system would not be able to “take bets” below a certain level of confidence. One wonders if this would be a more or less arbitrary system.

Sadly, in order to evaluate this proposal seriously, one would have to employ mathematics. Which is, in O’Neil’s rhetoric, a form of evil magic. So, perhaps it’s best not to try.

O’Neil attributes the problems of WMD’s to the incentives of the data scientists building the systems. Maybe they know that their work effects people, especially the poor, in negative ways. But they don’t care.

But as a rule, the people running the WMD’s don’t dwell on these errors. Their feedback is money, which is also their incentive. Their systems are engineered to gobble up more data fine-tune their analytics so that more money will pour in. Investors, of course, feast on these returns and shower WMD companies with more money.

O’Neil, WMD, p.13

Calling out greed as the problem is effective and true in a lot of cases. I’ve argued myself that the real root of the technology ethics problem is capitalism: the way investors drive what products get made and deployed. This is a worthwhile point to make and one that doesn’t get made enough.

But the logical implications of this argument are off. Suppose it is true that “as a rule”, the makers of algorithms that do harm are made by people responding to the incentives of private capital. (IF harmful algorithm, THEN private capital created it.) That does not mean that there can’t be good algorithms as well, such as those created in the public sector. In other words, there are algorithms that are not WMDs.

So the insight here has to be that private capital investment corrupts the process of designing algorithms, making them harmful. One could easily make the case that private capital investment corrupts and makes harmful many things that are not algorithmic as well. For example, the historic trans-Atlantic slave trade was a terribly evil manifestation of capitalism. It did not, as far as I know, depend on modern day computer science.

Capitalism here looks to be the root of all evil. The fact that companies are using mathematics is merely incidental. And O’Neil should know that!

Here’s what I find so frustrating about this line of argument. Mathematical literacy is critical for understanding what’s going on with these systems and how to improve society. O’Neil certainly has this literacy. But there are many people who don’t have it. There is a power disparity there which is uncomfortable for everybody. But while O’Neil is admirably raising awareness about how these kinds of technical systems can and do go wrong, the single-minded focus and framing risks giving people the wrong idea that these intellectual tools are always bad or dangerous. That is not a solution to anything, in my view. Ignorance is never more ethical than education. But there is an enormous appetite among ignorant people for being told that it is so.

References

O’Neil, Cathy. Weapons of math destruction: How big data increases inequality and threatens democracy. Broadway Books, 2017.

Sassen, Saskia. “Predatory Formations Dressed in Wall Street Suits and Algorithmic Math.” Science, Technology and Society22.1 (2017): 6-20.

Alain Badiou and artificial intelligence

Last week I saw Alain Badiou speak at NYU on “Philosophy between Mathematics and Poetry”, followed by a comment by Alexander Galloway, and then questions fielded from the audience.

It was wonderful to see Badiou speak as ever since I’ve become acquainted with his work (which was rather recently, Summer of 2016) I have seen it as a very hopeful direction for philosophy. As perhaps implied by the title of his talk, Badiou takes mathematics very seriously, perhaps more seriously than most mathematicians, and this distinguishes him from many other philosophers for whom mathematics is somewhat of an embarrassment. There are few fields more intellectually rarified than mathematics, philosophy, and poetry, and yet somehow Badiou treats each fairly in a way that reflects how broader disciplinary and cultural divisions between the humanities and technical fields may be reconciled. (This connects to some of my work on Philosophy of Computational Social Science)

I have written a bit recently about existentialism in design only to falter at the actual definition of existentialism. While it would I’m sure be incorrect to describe Badiou as an existentialist, there’s no doubt that he represents the great so-called Continental philosophical tradition, is familiar with Heidegger and Nietzsche, and so on. I see certain substantive resonances between Badiou and other existentialist writers, though I think to make the comparison now would be putting the cart before the horse.

Badiou’s position, in a nutshell, is like this:

Mathematics is a purely demonstrative form of writing and thinking. It communicates by proof, and has a special kind of audience to it. It is a science. In particular it is a science of all the possible forms of multiplicity, which is the same thing as saying as it is the science of all being, or ontology.

Poetry, on the other hand, is not about being but rather about becoming. “Becoming” for Badiou is subjective: the conscious subject encounters something new, experiences a change, sees an unrealized potential. These are events, and perhaps the greatest contribution of Badiou is his formulation and emphasis on the event as a category. In reference to earlier works, the event might be when through Hegelian dialectic a category is sublated. It could also perhaps correspond to when existence overcomes being in de Beauvoir’s ethics (hence the connection to existentialism I’m proposing). Good poetry, in Badiou’s thought, shows how the things we experience can break out of the structures that objectify them, turning the (subjectively perceived) impossible into a new reality.

Poetry, perhaps because it is connected to realizing the impossible but perhaps just because it’s nice to listen to (I’m unclear on Badiou’s position on this point) is “seductive”, encouraging psychological connections to the speaker (such as transference) whether or not it’s “true”. Classically, poetry meant epic poems and tragic theater. It could be cinema today.

Philosophy has the problem that it has historically tried to be both demonstrative, like mathematics, and seductive, like poetry. It’s this impurity or tension that defines it. Philosophers need to know mathematics because it is ontology, but have to go beyond mathematics because their mission is to create events in subjectively experienced reality, which is historically situated, and therefore not merely a matter of mathematical abstraction. Philosophers are in the business of creating new forms of subjectivity, which is not the same as creating a new form of being.

I’m fine with all this.

Galloway made some comments I’m somewhat skeptical of, though I may not have understood them since he seems to build mostly on Deleuze and Lacan, who are two intellectual sources I’ve never gotten into. But Galloway’s idea is to draw a connection between the “digital”, with all of its connections to computing technology, algorithms, the Internet, etc., with Badiou’s understanding of the mathematical, and to connect the “analog”, which is not discretized like the digital, to poetry. He suggested that Badiou’s sense of mathematics was arithmetic and excluded the geometric.

I take this interpretation of Galloway’s as clever, but incorrect and uncharitable. It’s clever because it co-opts a great thinker’s work into the sociopolitical agenda of trying to bolster the cultural capital of the humanities against the erosion of algorithmic curation and diminution relative to the fortunes of technology industries. This has been the agenda of professional humanists for a long time and it is annoying (to me) but I suppose necessary for the maintenance of the humanities, which are important.

However, I believe the interpretation is incorrect and uncharitable to Badiou because though Badiou’s paradigmatic example of mathematics is set theory, he seems to have a solid enough grasp of Kurt Godel’s main points to understand that mathematics includes the great variety of axiomatic systems and these, absolutely, indisputably, include geometry and real analysis and all the rest. The fact that logical proof is a discrete process which can be reduced to and from Boolean logic and automated in an electric circuit is, of course, the foundational science of computation that we owe to Turing, Church, Von Neumann, and others. It’s for these reasons that the potential of computation is so impressive and imposing: it potentially represents all possible forms of being. There are no limits to AI, at least none based on these mathematical foundations.

There were a number of good questions from the audience which led Badiou to clarify his position. The Real is relational, it is for a subject. This distinguishes it from Being, which is never relational (though of course, there are mathematical theories of relations, and this would seem to be a contradiction in Badiou’s thought?) He acknowledges that a difficult question is the part of Being in the the real.

Meanwhile, the Subject is always the result of an event.

Physics is a science of the existing form of the real, as opposed to the possible forms. Mathematics describes the possible forms of what exists. So empirical science can discover which mathematical form is the one that exists for us.

Another member of the audience asked about the impossibility of communism, which was on point because Badiou has at times defended communism or argued that the purpose of philosophy is to bring about communism. He made the point that one could not mathematically disprove the possibility of communism.

The real question, I may be so bold as to comment afterwards, is whether communism can exist in our reality. Suppose that economics is like physics in that it is a science of the real as it exists for us. What if economics shows that communism is impossible in our reality?

Though it wasn’t quite made explicitly, here is the subtle point of departure Badiou makes from what is otherwise conventionally unobjectionable. He would argue, I believe, that the purpose of philosophy is to create a new subjective reality where the impossible is made real, and he doesn’t see this process as necessarily bounded by, say, physics in its current manifestation. There is the possibiliity of a new event, and of seizing that event, through, for example, poetry. This is the article of faith in philosophy, and in poets, that has established them as the last bastion against dehumanization, objectification, reification, and the dangers of technique and technology since at least Heidegger’s Question Concerning Technology.

Which circles us back to the productive question: how would we design a technology that furthers this objective of creating new subjective realities, new events? This is what I’m after.

Free software and capitalism

As the global capitalist economy tanks, and as I attend a free software conference, my mind alights on the subject of the role of free software in global capitalism.

My verdict: it is a radical departure.

Capitalism is an economic system whose foundation is the private ownership of the means of production.  Software is, among other things, a means of production.  Free software is not privately owned.*  So each successful free software project shifts the foundation of the economy towards…something else.

But what?

As Arnulf Christl has exhorted throughout the conference, the opposite of free software is proprietary software, not commercial software.  The proliferation of open source software has brought with it an open source industry that operates in the market just like other industries.

I’m certainly not the first to say this, but it seems high time for an economic theory that takes intellectual goods, and their tendency towards freedom, as fundamental instead of grafting them onto theories about trade in “normal,” material commodities.

* pace, licensing quibblers.