Digifesto

Transaction cost economics and privacy: looking at Hoofnagle and Whittington’s “Free”

As I’ve been reading about transaction cost economics (TCE) and independently scrutinizing the business model of search engines, it stands to reason that I should look to the key paper holding down the connection between TCE and privacy, Hoofnagle and Whittinton’s “Free: Accounting for the Costs of the Internet’s Most Popular Price” (2014).

I want to preface the topic by saying I stand by what I wrote earlier: that at the heart of what’s going on with search engines, you have a trade of attention; it requires imagining the user has have attention-time as a scarce resource. The user has a query and has the option to find material relevant to the query in a variety of ways (like going to a library). Often (!) they will do so in a way that costs them as little attention as possible: they use a search engine, which gives an almost instant and often high-quality response; they are also shown advertisements which consume some small amount of their attention, but less than they would expend searching through other means. Advertisers pay the search engine for this exposure to the user’s attention, which funds the service that is “free”, in dollars (but not in attention) to the users.

Hoofnagle and Whittington make a very different argument about what’s going on with “free” web services, which includes free search engines. They argue that the claim that these web services are “free” is deceptive because the user may incur costs after the transaction on account of potential uses of their personal data. An example:

The freemium business model Anderson refers to is popular among industries online. Among them, online games provide examples of free services with hidden costs. By prefacing play with the disclosure of personal identification, the firms that own and operate games can contact and monitor each person in ways that are difficult for the consumer to realize or foresee. This is the case for many games, including Disney’s “Club Penguin,” an entertainment website for children. After providing personal information to the firm, consumers of Club Penguin receive limited exposure to basic game features and can see numerous opportunities to enrich their play with additional features. In order to enrich the free service, consumers must buy all sort of enhancements, such as an upgraded igloo or pets for one’s penguin. Disney, like others in the industry, places financial value on the number of consumers it identifies, the personal information they provide, and the extent to which Disney can track consumer activity in order to modify the game and thus increase the rate of conversion of consumers from free players to paying customers.

There are a number of claims here. Let’s enumerate them:

  1. This is an example of a ‘free’ service with hidden costs to users.
  2. The consumer doesn’t know what the game company will do with their personal information.
  3. In fact, the game will use the personal information to personalize pitches for in-game purchases that ‘enrich’ the free service.
  4. The goal of the company is to convert free players to paying customers.

Working backwards, claim (4) is totally true. The company wants to make money by getting their customers to pay, and they will use personal information to make paying attractive to the customers (3). But this does not mean that the customer is always unwitting. Maybe children don’t understand the business model when they begin playing Penguin Club, but especially today parents certainly do. App Stores, for example, now label apps when they have “in-app purchases”, which is a pretty strong signal. Perhaps this is a recent change due to some saber rattling by the FTC, which to be fair would be attributable as a triumph to the authors if this article had influence on getting that to happen. On the other hand, this is a very simple form of customer notice.

I am not totally confident that even if (2), (3), and (4) are true, that that entails (1), that there are “hidden costs” to free services. Elsewhere, Hoofnagle and Whittington raise more convincing examples of “costs” to release of PII, including being denied a job and resolving identity theft. But being convincingly sold an upgraded igloo for your digital penguin seems so trivial. Even if it’s personalized, how could it be a hidden cost? It’s a separate transaction, no? Do you or do you not buy the igloo?

Parsing this through requires, perhaps, a deeper look at TCE. According to TCE, agents are boundedly rational (they can’t know everything) and opportunistic (they will make an advantageous decision in the moment). Meanwhile, the world is complicated. These conditions imply that there’s a lot of uncertainty about future behavior, as agents will act strategically in ways that they can’t themselves predict. Nevertheless, agents engage in contracts with some kinds of obligations in them in the course of a transaction. TCE’s point is that these contracts are always incomplete, meaning that there are always uncertainties left unresolved in contracts that will need to be negotiated in certain contingent cases. All these costs of drafting, negotiating, and safeguarding the agreement are transaction costs.

Take an example of software contracting, which I happen to know about from personal experience. A software vendor gets a contract from a client to do some customization. The client and the vendor negotiated some sort of scope of work ex ante. But always(!), the client doesn’t actually know what they want, and if the vendor delivers on the specification literally the client doesn’t like it. Then begins the ex post negotiation as the client tries to get the vendor to tweak the system into something more usable.

Software contracting often resolves this by getting off the fixed cost contracting model and onto a cost-and-materials contact that allows billing by hours of developer time. Alternatively, the vendor can internalize the costs into the contract by inflating the cost “estimates” to cover for contingencies. In general, this all amounts to having more contract and a stronger relationship between the client and vendor, a “bilateral dependency” which TCE sees as a natural evolution of the incomplete contract under several common conditions, like “asset specificity”, which means that the asset is specialized to a particular transaction (or the two agents involved in it). Another term for this is lock-in, or the presence of high switching costs, though this way of thinking about it reintroduces the idea of a classical market for essentially comparable goods and services that TCE is designed to mitigate against. This explains how technical dependencies of an organization become baked in more or less constitutionally as part of the organization, leading to the robustness of installed base of a computing platform over time.

This ebb and flow of contract negotiation with software vendors was a bit unsettling to me when I first encountered it on the job, but I think it’s safe to say that most people working in the industry accept this as How Things Work. Perhaps it’s the continued influence of orthodox economics that makes this all seem inefficient somehow, and TCE is the right way to conceptualize things that makes better sense of reality.

But back to the Penguins…

Hoofnagle and Whittington make the case that sharing PII with a service that then personalizes its offerings to you creates a kind of bilateral dependence between service and user. They also argue that loss of privacy, due to the many possible uses of this personal information (some nefarious), is a hidden cost that can be thought of as an ex post transaction cost that is a hazard because it has not been factored into the price ex ante. The fact that this data is valuable to the platform/service for paying their production costs, which is not part of the “free” transaction, is an indication that this data is a lot more valuable than consumers think it is.

I am still on the fence about this.

I can’t get over the feeling that successfully selling a user a personalized, upgraded digital igloo is such an absurd example of a “hidden cost” that it belies the whole argument that these services have hidden costs.

Splitting hairs perhaps, it seems reasonable to say that Penguin Club has a free version, which is negotiated as one transaction. Then, conditional on the first transaction, it offers personalized igloos for real dollars. This purchase, if engaged in, would be another, different transaction, not an ex post renegotiation of the original contract with the Disney. This small difference changes the cost of the igloo from a hidden transaction cost into a normal, transparent cost. So it’s no big deal!

Does the use of PII create a bilateral dependence between Disney and the users of Penguin Club? Yes, in a sense. Any application of attention to an information service, learning how to use it and getting it to be part of your life, is in a sense a bilateral dependence with a switching cost. But there are so many other free games to play on the internet that these costs seem hardly hidden. They could just be understood as part of the game. Meanwhile, we are basically unconcerned with Disney’s “dependence” on the consumer data, because Disney can get new users easily (unless the user is a “whale”, who actual pays the company). And “dependence” Disney has on particular users is a hidden cost for Disney, not for the user, and who cares about Disney.

The cases of identity theft or job loss are strange cases that seem to have more to do with freaky data reuse than what’s going on with a particular transaction. Purpose binding notices and restrictions, which are being normed on through generalized GDPR compliance, seem adequate to deal with these cases.

So, I have two conclusions:

(1) Maybe TCE is the right lens for making an economic argument for why purpose binding restrictions are a good idea. They make transactions with platforms less incomplete, avoiding the moral hazard of ex post use of data in ways that incurs asymmetrically unknown effects on users.

(2) This TCE analysis of platforms doesn’t address the explanatorily powerful point that attention is part of the trade. In addition to being concretely what the user is “giving up” to the platform and directly explaining monetization in some circumstances, the fact that attention is “sticky” and creates some amount of asset-specific learning is a feature of the information economy more generally. Maybe it needs a closer look.

References

Hoofnagle, Chris Jay, and Jan Whittington. “Free: accounting for the costs of the internet’s most popular price.” UCLA L. Rev. 61 (2013): 606.

Advertisements

Data isn’t labor because using search engines is really easy

A theme I’ve heard raised in a couple places recently, including Ibarra et al. “Should We Treat Data As Labor?” and the AI Now 2018 Report, is that there is something wrong with how “data”, particularly data “produced” by people on the web, is conceptualized as part of the economy. Creating data, the argument goes, requires labor. And as the product of labor, it should be protected according to the values and practices of labor movements in the past. In particular, the current uses of data in, say, targeted advertising, social media, and search, are exploitative; the idea that consumers ‘pay’ for these services with their data is misleading and ultimately unfair to the consumer. Somehow the value created by the data should be reapportioned back to the user.

This is a sexy and popular argument among a certain subset of intellectuals who care about these things. I believe the core emotional appeal of this proposal is this: It is well known that a few well-known search engine and social media companies, namely Google and Facebook, are rich. If the value added by user data were in part returned to the users, the users, who are compared to Google and Facebook not rich, would get something they otherwise would not get. I.e., the benefits for recognizing the labor involved in creating data is redistribution of surplus to The Rest of Us.

I don’t have a problem personally with that redistributive impulse. However, I don’t think the “data is labor” argument actually makes much sense.

Why not? Well, let’s take the example of a search engine. Here is the transaction between a user and a search engine:

  • Alice types a query, “avocado toast recipes”, into the search engine. This submits data to the company computers.
  • The company computers use that data to generate a list of results that it deems relevant to that query.
  • Alice sees the results, and maybe clicks on one or two of them, if they are good, in the process of navigating to the thing she was looking for in the first place.
  • The search engine records that click as well, in order to better calibrate how to respond to others making that query.

We might forget that the search engine is providing Alice a service and isn’t just a ubiquitous part of the infrastructure we should take for granted. The search engine has provided Alice with relevant search results. What this does is (dramatically) reduce Alice’s search costs; had she tried to find the relevant URL by asking her friends, organically surfing the web, or using the library, who knows what she would have found or how long it would take her. But we would assume that Alice is using the search engine because it gets her more relevant results, faster.

It is not clear how Alice could get this thing she wants without going through the motions of typing and clicking and submitting data. These actions all seem like a bare minimum of what is necessary to conduct this kind of transaction. Similarly, when I got to a grocery store and buy vegetables, I have to get out my credit card and swipe it at the machine. This creates data–the data about my credit card transaction. But I would never advocate for recognizing my hidden labor at the credit card machine is necessary to avoid the exploitation of the credit card companies, who then use that information to go about their business. That would be insane.

Indeed, it is a principle of user interface design that the most compelling user interfaces are those that require the least effort from their users. Using search engines is really, really easy because they are designed that way. The fact that oodles of data are collected from a person without that person exerting much effort may be problematic in a lot of ways. But it’s not problematic because it’s laborious for the user; it is designed and compelling precisely because it is labor-saving. The smart home device industry has taken this even further, building voice-activated products for people who would rather not use their hands to input data. That is, if anything, less labor for the user, but more data and more processing on the automated part of the transaction. That the data is work for the company, and less work for the user, indicates that data is not the same thing as user labor.

There is a version of this argument that brings up feminism. Women’s labor, feminists point out, has long been insufficiently recognized and not properly remunerated. For example, domestic labor traditionally performed by women has been taken for granted, and emotional labor (the work of controlling ones emotions on the job), which has often been feminized, has not been taken seriously enough. This is a problem, and the social cause of recognizing women’s labor and rewarding it is, ceteris paribus, a great thing. But, and I know I’m on dicey ground here, so bear with me, this does not mean that everything that women do that they are not paid to do is unrecognized labor in the sense that is relevant for feminist critiques. Case in point, both men and women use credit cards to buy things, and make telephone calls, and drive vehicles through toll booths, and use search engines, and do any number of things that generate “data”, and in most of these cases it is not remunerated directly; but this lack of remuneration isn’t gendered. I would say, perhaps controversially, that the feminist critique does not actually apply to the general case of user generated data much at all! (Though is may apply in specific cases that I haven’t thought of.)

So in conclusion, data isn’t labor, and labor isn’t data. They are different things. We may want a better, more just, political outcome with respect to the distribution of surplus from the technology economy. But trying to get there through an analogy between data and labor is a kind of incoherent way to go about it. We should come up with a better, different way.

So what’s a better alternative? If the revenue streams of search engines are any indication, then it would seem that users “pay” for search engines through being exposed to advertising. So the “resource” that users are giving up in order to use the search engine is attention, or mental time; hence the term, attention economy.

Framing the user cost of search engines in terms of attention does not easily lend itself to an argument for economic reform. Why? Because search engines are already saving people a lot of that attention by making it so easy to look stuff up. Really the transaction looks like:

  • Alice pays some attention to Gob (the search engine).
  • Gob gives Alice some good search results back in return, and then…
  • Gob passes on some of Alice’s attention through to Bob, the advertiser, in return for money.

So Alice gives up attention but gets back search results and the advertisement. Gob gets money. Bob gets attention. The “data” that matters is not the data transmitted from Alice’s computer up to Gob. Rather, the valuable data is the data that Alice receives through her eyes: of this data, the search results are positively valued, the advertisement is negatively valued, but the value of the bundled good is net positive.

If there is something unjust about this economic situation, it has to be due to the way consumer’s attention is being managed by Gob. Interestingly, those who have studied the value of ‘free’ services in attentional terms have chalked up a substantial consumer surplus due to saved attention (Brynjolfsson and Oh, 2012) This appears to be the perspective of management scientists, who tend to be pro-business, and is not a point repeated often by legal scholars, who tend to be more litigious in outlook. For example, legal scholarship has detailed the view of how attention could be abused through digital market manipulation (Calo, 2013).

Ironically for data-as-labor theorists, the search-engine-as-liberator-of-attention argument could be read as the view that what people get from using search engines is more time, or more ability to do other things with their time. In other words, we would use a search engine instead of some other, more laborious discovery mechanism precisely because it would cost us net negative labor. That absolutely throws a wrench in any argument that the users of search engines should be rewarded on dignity of labor grounds. Instead, what’s happened is that search engines are ubiquitous because consumers have undergone a phase transition in their willingness to work to discover things, and now very happily use search engines which, on the whole, seem like a pretty good deal! (The cost of being-advertised-to is small compared to the benefits of the search results.)

If we start seeing search engines as a compelling labor-saving device rather than a exploiter of laborious clickwork, then some of the disregard consumers have for privacy on search engines becomes more understandable. People are willing to give up their data, even if they would rather not, because search engines are saving them so much time. The privacy harms that come as consequence, then, can be seen as externalities to what is essentially a healthy transaction, rather than a perverse matter of a business model that is evil to the bone.

This is, I wager, on the whole a common sense view, one that I’d momentarily forgotten because of my intellectual milieu but now am ashamed to have overlooked. It is, on the whole, far more optimistic than other attempt to characterize the zeitgeist of new technology economy.

Somehow, this rubric for understanding the digital economy appears to have fallen out of fashion. Davenport and Beck (2001) wrote a business book declaring attention to be “the new currency of business”, which if the prior analysis is correct makes more sense than data being the new currency (or oil) of business. The term appears to have originated in an article by Goldhaber (1997). Ironically, the term appears to have had no uptake in the economics literature, despite it being the key to everything! The concept was understood, however, by Herbert Simon, in 1971 (see also Terranova, 2012):

In an information-rich world, the wealth of information means a dearth of something else: a scarcity of whatever it is that information consumes. What information consumes is rather obvious: it consumes the attention of its recipients. Hence a wealth of information creates a poverty of attention and a need to allocate that attention efficiently among the overabundance of information sources that might consume it.

(A digitized version of this essay, which amazingly appears to be set by a typewriter and then hand-edited (by Simon himself?) can be found here.)

This is where I bottom out–the discover that the line of thought I’ve been on all day starts with Herbert Simon, that the sciences of the artificial are not new, they are just forgotten (because of the glut of other information), and exhaustingly hyped. The attention economy discovered by Simon explains why each year we are surrounded with new theories about how to organize ourselves with technology, when perhaps the wisest perspectives on these topics are ones that will not hype themselves because their authors cannot tweet from the grave.

References

Arrieta-Ibarra, Imanol, et al. “Should We Treat Data as Labor? Moving beyond” Free”.” AEA Papers and Proceedings. Vol. 108. 2018.

Brynjolfsson, Erik, and JooHee Oh. “The attention economy: measuring the value of free digital services on the Internet.” (2012).

Calo, Ryan. “Digital market manipulation.” Geo. Wash. L. Rev. 82 (2013): 995.

Davenport, Thomas H., and John C. Beck. The attention economy: Understanding the new currency of business. Harvard Business Press, 2001.

Goldhaber, Michael H. “The attention economy and the net.” First Monday 2.4 (1997).

Simon, Herbert A. “Designing organizations for an information-rich world.” (1971): 37-72.

Terranova, Tiziana. “Attention, economy and the brain.” Culture Machine 13 (2012).

the make or buy decision (TCE) in the software and cybersecurity

The paradigmatic case of transaction cost economics (TCE) is the make-or-buy decision. A firm, F, needs something, C. Do they make it in-house or do they buy it from somewhere else?

If the firm makes it in-house, they will incur some bureaucratic overhead costs in addition to the costs of production. But they will also be able to specialize C for their purposes. They can institute their own internal quality controls. And so on.

If the firm buys it on the open market from some other firm, say, G, they don’t pay the overhead costs. They do lose the benefits of specialization, and the quality controls are only those based on economic competitive pressure on suppliers.

There is an intermediate option, which is a contract between F and G which establishes an ongoing relationship between the two firms. This contract creates a field in which C can be specialized for F, and there can be assurances of quality, while the overhead is distributed efficiently between F and G.

This situation is both extremely common in business practice and not well handled by neoclassical, orthodox economics. It’s the case that TCE is tremendously preoccupied with.


My background and research is in the software industry, which is rife with cases like these.

Developers are constantly faced with a decision to make-or-buy software components. In principle, they can developer any component themselves. In practice, this is rarely cost-effective.

In software, open source software components are a prevalent solution to this problem. This can be thought of as a very strange market where all the prices are zero. The most popular open source libraries are very generic , having little “asset specificity” in TCE terms.

The lack of contract between developers and open source components/communities is sometimes seen as a source of hazard in using open source components. The recent event-stream hack, where an upstream component was injected with malicious code by a developer who had taken over maintaining the package, illustrates the problems of outsourcing technical dependencies without a contract. In this case, the quality problem is manifest as a supply chain cybersecurity problem.

In Williamson’s analysis, these kinds of hazards are what drive firms away from purchasing on spot markets and towards contracting or in-house development. In practice, the role of open source support companies fills the role of being a responsible entity G that firm F can build a relationship with.

Williamson on four injunctions for good economics

Williamson (2008) (pdf) concludes with a description of four injunctions for doing good economics, which I will quote verbatim.

Robert Solow’s prescription for doing good economics is set out in three injunctions: keep it simple; get it right; make it plausible (2001, p. 111). Keeping it simple entails stripping away the inessentials and going for the main case (the jugular). Getting it right “includes translating economic concepts into accurate
mathematics (or diagrams, or words) and making sure that further logical operations are correctly performed and verified” (Solow, 2001, p. 112). Making it plausible entails describing human actors in (reasonably) veridical ways and maintaining meaningful contact with the phenomena of interest (contractual or otherwise).

To this, moreover, I would add a fourth injunction: derive refutable implications to which the relevant (often microanalytic) data are brought to bear. Nicholas Georgescu-Roegen has a felicitous way of putting it: “The purpose of science in general is not prediction, but knowledge for its own sake,” yet prediction is “the touchstone of scientific knowledge” (1971, p. 37).

Why the fourth injunction? This is necessitated by the need to choose among alternative theories that purport to deal with the same phenomenon—say vertical integration—and (more or less) satisfy the first three injunctions. Thus assume that all of the models are tractable, that the logic of each hangs together, and that agreement cannot be reached as to what constitutes veridicality and meaningful contact with the phenomena. Does each candidate theory then have equal claimsfor our attention? Or should we be more demanding? This is where refutable implications and empirical testing come in: ask each would-be theory to stand up and be counted.

Why more economists are not insistent upon deriving refutable implications and submitting these to empirical tests is a puzzle. One possibility is that the world of theory is set apart and has a life of its own. A second possibility is that some economists do not agree that refutable implications and testing are
important. Another is that some theories are truly fanciful and their protagonists would be discomfited by disclosure. A fourth is that the refutable implications of favored theories are contradicted by the data. And perhaps there are still other reasons. Be that as it may, a multiplicity of theories, some of which are
vacuous and others of which are fanciful, is an embarrassment to the pragmatically oriented members of the tribe. Among this subset, insistence upon the fourth injunction—derive refutable implications and submit these to the data—is growing.

References

Williamson, Oliver E. “Transaction cost economics.” Handbook of new institutional economics. Springer, Berlin, Heidelberg, 2008. 41-65.

Discovering transaction cost economics (TCE)

I’m in the process of discovering transaction cost economics (TCE), the branch of economics devoted to the study of transaction costs, which include bargaining and search costs. Oliver Williamson, who is a professor at UC Berkeley, won the Nobel Prize for his work on TCE in 2009. I’m starting with the Williamson, 2008 article (in the References) which seems like a late-stage overview of what is a large body of work.

Personally, this is yet another time when I’ve discovered that the answers or proper theoretical language for understanding something I am struggling with has simply been Somewhere Else all alone. Delight and frustration are pretty much evening each other out at this point.

Why is TCE so critical (to me)?

  • I think the real story about how the Internet and AI have changed things, which is the topic constantly reiterated in so many policy and HCI studies about platforms, is that they reduced search costs. However, it’s hard to make the case for that without a respectable theorization of search costs and how they matter to the economy. This, I think, what transaction cost economics are about.
  • You may recall I wrote my doctoral dissertation about “data economics” on the presumption (which was, truly, presumptuous) that a proper treatment of the role of data in the economy had not yet been done. This was due mainly to the deficiencies of the discussion of information in neoclassical economic theory. But perhaps I was a fool, because it may be that this missing-link work on information economics has been in transaction cost economics all along! Interestingly, Pat Bajari, who is Chief Economist at Amazon, has done some TCE work, suggesting that like Hal Varian’s economics, this is stuff that actually works in a business context, which is more or less the epistemic standard you want economics to meet. (I would argue that economics should be seen, foremost, as a discipline of social engineering.)
  • A whole other line of research I’ve worked on over the years has been trying to understand the software supply chain, especially with respect to open source software (Benthall 2016; Benthall, 2017). That’s a tricky topic because the idea of “supply” and “chain” in that domain are both highly metaphorical and essentially inaccurate. Yet there are clearly profound questions about the relationships between sociotechnical organizations, their internal and external complexity, and so on to be found there, along with (and this is really what’s exciting about it) ample empirical basis to support arguments about it, just by the nature of it. Well, it turns out that the paradigmatic case for transaction cost economics is vertical integration, or the “make-or-buy” decision wherein a firm decides to (A) purchase it from an open market, (D) produce something in-house, or (C) (and this is the case that transaction cost economics really tries to develop) engage with the supplier in a contract which creates an ongoing and secure relationship between them. Labor contracts are all, for reasons that I may go into later, of this (C) kind.

So, here comes TCE, with its firm roots in organization theory, Hayekian theories of the market, Coase’s and other theories of the firm, and firm emphasis on the supply chain relation between sociotechnical organizations. And I HAVEN’T STUDIED IT. There is even solid work on its relation to privacy done by Whittington and Hoofnagle (2011; 2013). How did I not know about this? Again, if I were not so delighted, I would be livid.

Please expect a long series of posts as I read through the literature on TCE and try to apply it to various cases of interest.

References

Benthall, S. (2017) Assessing Software Supply Chain Risk Using Public Data. IEEE STC 2017 Software Technology Conference.

Benthall, S., Pinney, T., Herz, J., Plummer, K. (2016) An Ecological Approach to Software Supply Chain Risk Management. Proceedings of the 15th Python in Science Conference. p. 136-142. Ed. Sebastian Benthall and Scott Rostrup.

Hoofnagle, Chris Jay, and Jan Whittington. “Free: accounting for the costs of the internet’s most popular price.” UCLA L. Rev. 61 (2013): 606.

Whittington, Jan, and Chris Jay Hoofnagle. “Unpacking Privacy’s Price.” NCL Rev. 90 (2011): 1327.

Williamson, Oliver E. “Transaction cost economics.” Handbook of new institutional economics. Springer, Berlin, Heidelberg, 2008. 41-65.

For fairness in machine learning, we need to consider the unfairness of racial categorization

Pre-prints of papers accepted to this coming 2019 Fairness, Accountability, and Transparency conference are floating around Twitter. From the looks of it, many of these papers add a wealth of historical and political context, which I feel is a big improvement.

A noteworthy paper, in this regard, is Hutchinson and Mitchell’s “50 Years of Test (Un)fairness: Lessons for Machine Learning”, which puts recent ‘fairness in machine learning’ work in the context of very analogous debates from the 60’s and 70’s that concerned the use of testing that could be biased due to cultural factors.

I like this paper a lot, in part because it is very thorough and in part because it tees up a line of argument that’s dear to me. Hutchinson and Mitchell raise the question of how to properly think about fairness in machine learning when the protected categories invoked by nondiscrimination law are themselves social constructs.

Some work on practically assessing fairness in ML has tackled the problem of using race as a construct. This echoes concerns in the testing literature that stem back to at least 1966: “one stumbles immediately over the scientific difficulty of establishing clear yardsticks by which people can be classified into convenient racial categories” [30]. Recent approaches have used Fitzpatrick skin type or unsupervised clustering to avoid racial categorizations [7, 55]. We note that the testing literature of the 1960s and 1970s frequently uses the phrase “cultural fairness” when referring to parity between blacks and whites.

They conclude that this is one of the areas where there can be a lot more useful work:

This short review of historical connections in fairness suggest several concrete steps forward for future research in ML fairness: Diving more deeply into the question of how subgroups are defined, suggested as early as 1966 [30], including questioning whether subgroups should be treated as discrete categories at all, and how intersectionality can be modeled. This might include, for example, how to quantify fairness along one dimension (e.g., age) conditioned on another dimension (e.g., skin tone), as recent work has begun to address [27, 39].

This is all very cool to read, because this is precisely the topic that Bruce Haynes and I address in our FAT* paper, “Racial categories in machine learning” (arXiv link). The problem we confront in this paper is that the racial categories we are used to using in the United States (White, Black, Asian) originate in the white supremacy that was enshrined into the Constitution when it was formed and perpetuated since then through the legal system (with some countervailing activity during the Civil Rights Movement, for example). This puts “fair machine learning” researchers in a bind: either they can use these categories, which have always been about perpetuating social inequality, or they can ignore the categories and reproduce the patterns of social inequality that prevail in fact because of the history of race.

In the paper, we propose a third option. First, rather than reify racial categories, we propose breaking race down into the kinds of personal features that get inscribed with racial meaning. Phenotype properties like skin type and ocular folds are one such set of features. Another set are events that indicate position in social class, such as being arrested or receiving welfare. Another set are facts about the national and geographic origin of ones ancestors. These facts about a person are clearly relevant to how racial distinctions are made, but are themselves more granular and multidimensional than race.

The next step is to detect race-like categories by looking at who is segregated from each other. We propose an unsupervised machine learning technique that works with the distribution of the phenotype, class, and ancestry features across spatial tracts (as in when considering where people physically live) or across a social network (as in when considering people’s professional networks, for example). Principal component analysis can identify what race-like dimensions capture the greatest amounts of spatial and social separation. We hypothesize that these dimensions will encode the ways racial categorization has shaped the social structure in tangible ways; these effects may include both politically recognized forms of discrimination as well as forms of discrimination that have not yet been surfaced. These dimensions can then be used to classify people in race-like ways as input to fairness interventions in machine learning.

A key part of our proposal is that race-like classification depends on the empirical distribution of persons in physical and social space, and so are not fixed. This operationalizes the way that race is socially and politically constructed without reifying the categories in terms that reproduce their white supremacist origins.

I’m quite stoked about this research, though obviously it raises a lot of serious challenges in terms of validation.

Is competition good for cybersecurity?

A question that keeps coming up in various forms, but for example in response to recent events around the ‘trade war’ between the U.S. and China and its impact on technology companies, is whether or not market competition is good or bad for cyber-security.

Here is a simple argument for why competition could be good for cyber-security: The security of technical products is a positive quality of them, something that consumers would like. Market competition is what gets producers to make higher quality products at lower cost. Therefore, competition is good for security.

Here is an argument for why competition could be bad for cyber-security: Security is a hard thing for any consumer to understand; since most won’t, we have an information asymmetry here and therefore a ‘market for lemons’ kind of market failure. Therefore, competition is bad for security. It would be better to have a well-regulated monopoly.

This argument echoes, though it doesn’t exactly parallel, some of the arguments in Pasquale’s work on Hamiltonian’s and Jeffersonian’s in technology platform regulation.

“the privatization of public functions”

An emerging theme from the conference on Trade Secrets and Algorithmic Systems was that legal scholars have become concerned about the privatization of public functions. For example, the use of proprietary risk assessment tools instead of the discretion of judges who are supposed to be publicly accountable is a problem. More generally, use of “trade secrecy” in court settings to prevent inquiry into software systems is bogus and moves more societal control into the realm of private ordering.

Many remedies were proposed. Most involved some kind of disclosure and audit to experts. The most extreme form of disclosure is making the software and, where it’s a matter of public record, training data publicly available.

It is striking to me to be encountering the call for government use of open source systems because…this is not a new issue. The conversation about federal use of open source software was alive and well over five years ago. Then, the arguments were about vendor lock-in; now, they are about accountability of AI. But the essential problem of whether core governing logic should be available to public scrutiny, and the effects of its privatization, have been the same.

If we are concerned with the reliability of a closed and large-scale decision-making process of any kind, we are dealing with problems of credibility, opacity, and complexity. The prospects of an efficient market for these kinds of systems are dim. These market conditions are the conditions of sustainability of open source infrastructure. Failures in sustainability are manifest as software vulnerabilities, which are one of the key reasons why governments are warned against OSS now, though the process of measurement and evaluation of OSS software vulnerability versus proprietary vulnerabilities is methodologically highly fraught.

Trade secrecy, “an FDA for algorithms”, a software bills of materials (SBOM) #SecretAlgos

At the Conference on Trade Secrets and Algorithmic Systems at NYU today, the target of most critiques is the use of trade secrecy by proprietary technology providers to prevent courts and the public from seeing the inner workings of algorithms that determine people’s credit scores, health care, criminal sentencing, and so on. The overarching theme is that sometimes companies will use trade secrecy to hide the ways that their software is bad, and that that is a problem.

In one panel, the question of whether an “FDA for Algorithms” is on the table–referring the Food and Drug Administration’s approval of pharmaceuticals. It was not dealt with in too much depth, which is too bad, because it is a nice example of how government oversight of potentially dangerous technology is managed in a way that respects trade secrecy.

According to this article, when filing for FDA approval, a company can declare some of their ingredients to be trade secrets. The upshot of that is that those trade secrets are not subject to FOIA requests. However, these ingredients are still considered when approval is granted by the FDA.

It so happens that in the cybersecurity policy conversation (more so than in privacy) the question of openness of “ingredients” to inspection has been coming up in a serious way. NTIA has been hosting multistakeholder meetings about standards and policy around Software Component Transparency. In particular they are encouraging standardizations of Software Bills of Materials (SBOM) like the Linux Foundation’s Software Package Data Exchange (SPDX). SPDX (and SBOM’s more generally) describe the “ingredients” in a software package at a higher level of resolution than exposing the full source code, but at a level specific enough useful for security audits.

It’s possible that a similar method could be used for algorithmic audits with fairness (i.e., nondiscrimination compliance) and privacy (i.e., information sharing to third-parties) in mind. Particular components could be audited (perhaps in a way that protects trade secrecy), and then those components could be listed as “ingredients” by other vendors.

The paradox of ‘data markets’

We often hear that companies are “selling out data”, or that we are “paying for services” with our data. Data brokers literally buy and sell data about people. There are other forms of expensive data sources or data sets. There is, undoubtedly, one or more data markets.

We know that classically, perfect competition in markets depends on perfect information. Buyers and sellers on the market need to have equal and instantaneous access to information about utility curves and prices in order for the market to price things efficiently.

Since the bread and butter of the data market is information asymmetry, we know that data markets can never be perfectly competitive. If it was, the data market would cease to exist, because the perfect information condition would entail that there is nothing to buy and sell.

Data markets therefore have to be imperfectly competitive. But since these are the markets that perfect information in other markets might depend on, this imperfection is viral. The vicissitudes of the data market are the vicissitudes of the economy in general.

The upshot is that the challenges of information economics are not only those that appear in special sectors like insurance markets. They are at the heart of all economic activity, and there are no equilibrium guarantees.