Digifesto

Category: Law

Regulating infoglut?

In the 20’s, many people were attracted for the first time in investing in the stock market. It was a time when fortunes were made and lost, but made more than they were lost, and so on average investors saw large returns. However, the growth in value of stocks was driven in part, and especially in the later half of the decade, by debt. The U.S. Federal Reserve chose to lower interest rates, making it easier to borrow money. When the interest rates on loans were lower than the rates of return on stocks, everybody from households to brokers began to take on debt to reinvest in the stock market. (Brooks, 1999)

After the crash of ’29, which left the economy decimated, there was a reckoning, leading to the Securities Act of 1933 and the Securities Exchange Act of 1934. The latter established the Securities and Exchange Commission (SEC), and established the groundwork for the more trusted financial institutions we have today.

Cohen (2016) writes about a more current economic issue. As the economy changes from being centered on industrial capitalism to informational capitalism, the infrastructural affordances of modern computing and networking have invalidated the background logic of how many regulations are supposed to work. For example, anti-discrimination regulation is designed to prevent decisions from being made based on protected or sensitive attributes of individuals. However, those regulations made most sense when personal information was relatively scarce. Today, when individual activity is highly instrumented by pervasive computing infrastructure, we suffer from infoglut — more information than is good for us, either as individuals or as a society. As a consequence, proxies of protected attributes are readily available for decision-makers and indeed are difficult to weed out of a machine learning system even when market actors fully intend to do so (see Datta et al., 2017). In other words, the structural conditions that enable infoglut erode rights that we took for granted in the absence of today’s network and computing systems.

In an ongoing project with Salome Viljoen, we are examining the parallels between the financial economy and the data economy. These economies are, of course, not fully distinct. However, they are distinguished in part by how they are regulated: the financial economy has over a century of matured regulations defining it and reducing system risks such as those resulting from a debt-financed speculative bubble; the data economy has emerged only recently as a major source of profit with perhaps unforeseen systemic risks.

We have an intuition that we would like to pin down more carefully as we work through these comparisons: that there is something similar about the speculative bubbles that led to the Great Depression and today’s infoglut. In a similar vein to prior work looking that uses regulatory analogy to motivate new thinking about data regulation (Hirsch, 2013; Froomkin, 2015) and professional codes (Stark and Hoffman, 2019), we are interested in how financial regulation may be a precedent for regulation of the data economy.

However, we have reason to believe that the connections between finance and personal data are not merely metaphorical. Indeed, finance is an area with well-developed sectoral privacy laws that guarantee the confidentiality of personal data (Swire, 2003); it is also the case that financial institutions are one of the many ways personal data originating from non-financial contexts is monetized. We do not have to get poetic to see how these assets are connected; they are related as a matter of fact.

What is more elusive, and at this point only a hypothesis, is that there is valid sense in which the systemic risks of infoglut can be conceptually understood using tools similar to those that are used to understand financial risk. Here I maintain an ambition: that systemic risk due to infoglut may be understood using the tools of macroeconomics and hence internalized via technocratic regulatory mechanisms. This would be a departure from Cohen (2016), who gestures more favorably towards “uncertainty” based regulation that does not attempt probabilistic expectation but rather involves tools such as threat modeling, as used in some cybersecurity practices.

References

Brooks, J. (1999). Once in Golconda: A true drama of Wall Street 1920-1938. John Wiley & Sons.

Cohen, J. E. (2016). The regulatory state in the information age. Theoretical Inquiries in Law17(2), 369-414.

Datta, A., Fredrikson, M., Ko, G., Mardziel, P., & Sen, S. (2017, October). Use privacy in data-driven systems: Theory and experiments with machine learnt programs. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (pp. 1193-1210).

Froomkin, A. M. (2015). Regulating Mass Surveillance as Privacy Pollution: Learning from Environmental Impact Statements. U. Ill. L. Rev., 1713.

Hirsch, D. D. (2013). The glass house effect: Big Data, the new oil, and the power of analogy. Me. L. Rev.66, 373.

Stark, L., & Hoffmann, A. L. (2019). Data is the new what? Popular metaphors & professional ethics in emerging data culture.

Swire, P. P. (2003). Efficient confidentiality for privacy, security, and confidential business information. Brookings-Wharton Papers on Financial Services2003(1), 273-310.

Surden, H. (2007). Structural rights in privacy. SMUL Rev.60, 1605.

Is there hypertext law? Is there Python law?

I have been impressed with Hildbebrandt’s analysis of the way particular technologies provide the grounds for different forms of institutions. Looking into the work of Don Ihde, who I gather is a pivotal thinking in this line of reasoning, I find the ‘postphenomenological’ and ‘instrumental realist’ position very compelling. Lawrence Diver’s work on digisprudence, which follows in this vein, looks generative.

In my encounters with with work, I have also perceived there to be gaps and discrepancies in the texture of the argument. There is something uncanny about reading material that is, perceptually, almost correct. Either I am in error, or it is.

One key difference seems to be about the attitude towards mathematical or computational formalism. This is chiefly, I sense, truly an attitude, in the sense of emotional difference. Scholars in this area will speak, in personal communication, of being “wary” or “afraid”. It’s an embodied reaction which orients their rhetoric. It is shared with many other specifically legal scholars. In the gestalt of these arguments, the legal scholar will refer to philosophies of science and/or technology to justify a distance between lived reality, lifeworld, and artifice.

Taking a somewhat different perspective, there are other ways to consider the relationship between formalism, science, and fact, even when taking seriously the instrumental realist position. It is noteworthy, I believe, that this field of scholarship is so adamantly Latourian, and that Latour has succeeded in anathematizing Bourdieu. I now see more clearly how Science of Science and Reflexivity, which was both a refutation of Latour and a lament of how the capture of institutional power (such as nation-state provided research funding) is a distortion to the autonomous and legitimizing processes of science, are really all one argument. Latour, despite the wrongness of so much of his early work which is now so widely cited, became a powerful figure. The better argument will only win in time.

Bourdieu, it should be noted, is an instrumental realist about science, though he may not have been aware of Ihde and that line of discourse. He also saw the connection between formalism and instrumentation which seems to elude the postphenomenologist legal scholars. Formalism and instrumentation are both a form of practical “automation” which, if we take the instrumental realists seriously (and we should) wind up enabling the body, understood as perception-praxis, to see and know in different ways. Bourdieu, who obviously has read Foucault but improves on him, accepts the perception-praxis view of the body and socializes it through the concept of the habitus, which is key to his analysis of the sociology of science.

But I digress. What I have been working towards is the framing of the questions in the title. To recap, Hildebrandt, in my understanding, makes a compelling case for how the printing press, as a technology, has had specific affordances that have enabled the Rule of Law that is characteristic of constitutional democracy. This Rule of Law, or some descendent of it, remains dominant in Europe and perhaps this is why, via the Brussells Effect, the EU now stands as the protector of individuals from the encroaching power of machine-learning powered technologies, in the form of Information and Communication Infrastructure (ICI).

This is a fine narrative, though perhaps rather specifically motivated by a small number of high profile regulatory acts. I will not suggest that the narrative overplays anybody’s hand; it is useful as a schematic.

However, I am not sure the analysis is so solid. There seem to be some missing steps in the historical analysis. Which brings me to my first question, which is: what about hypertext? Hypertext is neither the text of the printing press, nor is it a form of machine learning. It is instrumentally dependent on scientific and technological formalism: the HyperText Markup Language (HTML) and the HyperText Transfer Protocol are both formal standards, built instrumentally on a foundation of computation and networking theory and technology. And as a matter of contemporary perception and praxis, it is probably the primary way in which people engage in analysis of law and communication about the law today.

So, what about it? Doesn’t this example show a contradiction at the heart of this instrumental realist legal scholarship?

The follow-up question is about another class of digital “languages”: software source code. Python, for example. These, even more than HyperText, are formalism, with semantics guaranteed by a compiler. But these semantics are in a sense legislated via the Python Enhancement Proposal process, and of course any particular Python application or software practice may be designed and mandated through a wide array of institutional mechanisms before being deployed to users.

I would look forward to work on these subjects coming from Hildebrandt’s CUHOBICOL research group, but for the fact that these technologies (which may bely the ideology motivating the project!) are excluded by the very system of categories the project invokes to classify different kinds of regulatory systems. According to the project web site (written, like all web sites, in HyperText), there are three (only three?) kinds of normativity: text-driven normativity, based in the printing press; data-based normativity, the normativity of feedback once based in cybernetic engineering and now based in machine learning; and code-based normativity. The last category is defined in terms of code’s immutability, which is rather alien to anybody who writes software code and has to deal with how it changes all the time. Moreover, the project’s aim is to explore code-based normativity through blockchain applications. I understand that gesturing at blockchain technology is a nice way to spice up a funding proposal. But by seeing normativity in these terms, many intermediate technologies, and therefore a broad technical design space of normative technology, are excluded from analysis.

Managerialism and Habermas

Managerialism is an “in” topic recently in privacy scholarship (Cohen, 2019; Waldman, 2019). In Waldman’s (2019) formulation, the managerialism problem is, roughly: privacy regulations are written with a certain substantive intent, but the for-profit firms that are the object of these regulations interpret them either as a bothersome constraint on otherwise profitable activity, or else as means to the ends of profitability, efficiency, and so on themselves. In other words, the substance of the regulations are subjugated to the substance of the goals of corporate management. Managerialism.

This is exactly what anybody who has worked in a corporate tech environment would expect. The scholarly accomplishment of presenting these bare facts to a legal academic audience is significant because employees of these corporations are most often locked up by strict NDAs. So while the point is obvious, I mean that in the positive sense that it should be taken as an unquestioned background assumption from now on, not that it shouldn’t have been “discovered” by this field in a different way.

As a “critical” observation, it stands. It raises a few questions:

  • Is this a problem?
  • If so, for whom?
  • If so, what can be done about it?

Here the “critical” method reaches, perhaps, its limits. Notoriously, critical scholarship plays on its own ambiguity, dancing between the positions of “criticism”, or finding of actionable fault, and “critique”, a merely descriptive account that is at most suggestive of action. This ambiguity preserves the standing of the critical scholar. They need never be wrong.

Responding to the situation revealed by this criticism requires a differently oriented kind of work.

Habermas and human interests

A striking about the world of policy and legal scholarship in the United States is that nobody is incentivized to teach or read anything written by past generations, however much it synthesized centuries of knowledge, and so nothing ever changes. For example, arguably, Habermas’s Knowledge and Human Interests (KHI), originally published 1972, arguably lays out the epistemological framework we would want to understand the managerialism issue raised by recent scholars. We should expect Habermas to anticipate the problems raised by capitalism in the 21st century because his points are based on a meticulously constructed, historically informed, universalist, transcendental form of analysis. This sort of analysis is not popular in the U.S.; I have my theories about why. But I digress.

A key point from Habermas (who is summing up and reiterating a lot of other work originating, if it’s possible to say any such thing meaningfully, in Max Weber) is that it’s helpful to differentiate between different kinds of knowledge based on the “human interests” that motivate them. In one formulation (the one in KHI), there are three categories:

  1. The technical interest (from techne) in controlling nature, which leads to the “empirical-analytic”, or positivist, sciences. These correspond to fields like engineering and the positivist social sciences.
  2. The pragmatic interest (from praxis), in mutual understanding which would guide collective action and the formation of norms, leads to the “hermeneutic” sciences. These correspond to fields like history and anthropology and other homes of “interpretivist” methods.
  3. The emancipatory interest, in exposing what has been falsely reified as objective fact as socially contingent. This leads to the critical sciences, which I suppose corresponds to what is today media studies.

This is a helpful breakdown, though I should say it’s not Habermas’s “mature” position, which is quite a bit more complicated. However, it is useful for the purposes of this post because it tracks the managerialist situation raised by Waldman so nicely.

I’ll need to elaborate on one more thing before applying this to the managerialist framing, which is to skip past several volumes of Habermas’s ouvre and get to Theory of Communicative Action, volume II, where he gets to the punchline. By now he’s developed the socially pragmatic interest to be the basis for “communicative rationality”, a discursive discipline in which individual interests are excluded and instead a diversely perspectival but nevertheless measured conversation about the way the social world should normatively be ordered. But where is this field in actuality? Money and power, the “steering media”, are always mussing up this conversation in the “public sphere”. So “public discourse” becomes a very poor proxy for communicative action. Rather–and this is the punchline–the actually existing field of communicative rationality, which is establishing substantive norms while nevertheless being “disinterested” with respect to the individual participants, is the law. That’s what the legal scholarship is for.

Applying the Habermasian frame to managerialism

So here’s what I think is going on. Waldman is pointing out that whereas regulations are being written with a kind of socially pragmatic interest in their impact on the imagined field of discursively rational participants as represented by legal scholarship, corporate managers are operating in the technical mode in order to, say, maximized shareholder profits as is their legally mandated fiduciary duty. And so the meaning of the regulation changes. Because words don’t contain meaning but rather take their meaning from the field in which they operate. A privacy policy that once spoke to human dignity gets misheard and speaks instead to inconvenience of compliance costs and a PR department’s assessment of the competitive benefit of user’s trust.

I suppose this is bothersome from the legal perspective because it’s a bummer when something one feels is an important accomplishment of one’s field is misused by another. But I find the professional politics here, as everywhere, a bit dull and petty.

Crucially, the managerialism problem is not dumb and petty–I wouldn’t be writing all this if I thought so. However, the frustrating aspect of this discourse is that because of the absence of philosophical grounding in this debate, it misses what’s at stake. This is unfortunately characteristic of much American legal analysis. It’s missing because when American scholars address this problem, they do so primarily in the descriptive critical mode, one that is empirical and in a sense positivist, but without the interest in control. This critical mode leads to cynicism. It rarely leads to collective action. Something is missing.

Morality

A missing piece of the puzzle, one which cannot ever be accomplished through empirical descriptive work, is the establishment of the moral consequence of managerialism which is that human beings are being treated as means and not ends, in contradiction with the Kantian categorical imperative, or something like that. Indeed, it is this flavor of moral maxim that threads its way up through Marx into the Frankfurt School literature with all of its well-trod condemnation of instrumental reason and the socially destructive overreach of private capital. This is, of course, what Habermas was going on about in the first place: the steering media, the technical interest, positivist science, etc. as the enemy of politically legitimate praxis based on the substantive recognition of the needs and rights of all by all.

It would be nice, one taking this hard line would say, if all laws were designed with this kind of morality in mind, and if everybody who followed them did so out of a rationally accepted understanding of their import. That would be a society that respected human dignity.

We don’t have that. Instead, we have managerialism. But we’ve known this for some time. All these critiques are effectively mid 20th century.

So now what?

If the “problem” of managerialism is that when regulations reach the firms that they are meant to regulate their meaning changes into an instrumentalist distortion of the original, one might be tempted to try to combat this tendency with an even more forceful use of hermeneutic discourse or an intense training in the social pragmatic stance such that employees of these companies put up some kind of resistance to the instrumental, managerial mindset. That strategy neglects the very real possibility that those employees that do not embrace the managerial mindset will be fired. Only in the most rarified contexts does discourse propel itself with its one force. We must presume that in the corporate context the dominance of managerialist discourse is in part due to a structural selection effect. Good managers lead the company, are promoted, and so on.

So the angle on this can’t be a discursive battle with the employees of regulated firms. Rather, it has to be about corporate governance. This is incidentally absolutely what bourgeois liberal law ought to be doing, in the sense that it’s law as it applies to capital owners. I wonder how long it will be before privacy scholars begin attending to this topic.

References

Benthall, S. (2015). Designing networked publics for communicative action. Interface1(1), 3.

Bohman, J., & Rehg, W. (2007). Jürgen habermas.

Cohen, J. E. (2019). Between Truth and Power: The Legal Constructions of Informational Capitalism. Oxford University Press, USA.

Habermas, J. (2015). Knowledge and human interests. John Wiley & Sons.

Waldman, A. E. (2019). Privacy Law’s False Promise. Washington University Law Review97(3).

The diverging philosophical roots of U.S. and E.U. privacy regimes

For those in the privacy scholarship community, there is an awkward truth that European data protection law is going to a different direction from U.S. Federal privacy law. A thorough realpolitical analysis of how the current U.S. regime regarding personal data has been constructed over time to advantage large technology companies can be found in Cohen’s Between Truth and Power (2019). There is, to be sure, a corresponding story to be told about EU data protection law.

Adjacent, somehow, to the operations of political power are the normative arguments leveraged both in the U.S. and in Europe for their respective regimes. Legal scholarship, however remote from actual policy change, remains as a form of moral inquiry. It is possible, still, that through professional training of lawyers and policy-makers, some form of ethical imperative can take root. Democratic interventions into the operations of power, while unlikely, are still in principle possible: but only if education stays true to principle and does not succumb to mere ideology.

This is not easy for educational institutions to accomplish. Higher education certainly is vulnerable to politics. A stark example of this was the purging of Marxist intellectuals from American academic institutions under McCarthyism. Intellectual diversity in the United States has suffered ever since. However, this was only possible because Marxism as a philosophical movement is extraneous to the legal structure of the United States. It was never embedded at a legal level in U.S. institutions.

There is a simply historical reason for this. The U.S. legal system was founded under a different set of philosophical principles; that philosophical lineage still impacts us today. The Founding Fathers were primarily influenced by John Locke. Locke rose to prominence in Britain when the Whigs, a new bourgeois class of Parliamentarian merchant leaders, rose to power, contesting the earlier monarchy. Locke’s political contributions were a treatise pointing out the absurdity of the Divine Right of Kings, the prevailing political ideology of the time, and a second treatise arguing for a natural right to property based on the appropriation of nature. This latter political philosophy was very well aligned with Britain’s new national project of colonialist expansion. With the founding of the United States, it was enshrined into the Constitution. The liberal system of rights that we enjoy in the U.S. are founded in the Lockean tradition.

Intellectual progress in Europe did not halt with Locke. Locke’s ideas were taken up by David Hume, whose introduced arguments that were so agitating that they famously woke Immanuel Kant, in Germany, from his “dogmatic slumber”, leading him to develop a new highly systematic system of morality and epistemology. Among the innovations in this work was the idea that human freedom is grounded in the dignity of being an autonomous person. The source of dignity is not based in a natural process such as the tilling of land. It is rather based in on transcendental facts about what it means to be human. The key to morality is treating people like ends, not means; in other words, not using people as tools to other aims, but as aims in themselves.

If this sound overly lofty to an American audience, it’s because this philosophical tradition has never taken hold in American education. In both the United Kingdom and Britain, Kantian philosophy has always been outside the mainstream. The tradition of Locke, through Hume, has continued on in what philosophers will call “analytic philosophy”. This philosophy has taken on the empiricist view that the only source of knowledge is individual experience. It has transformed over centuries but continues to orbit around the individual and their rights, grounded in pragmatic considerations, and learning normative rules using the case-by-case approach of Common Law.

From Kant, a different “continental philosophy” tradition produced Hegel, who produced Marx. We can trace from Kant’s original arguments about how morality is based on the transcendental dignity of the individual to the moralistic critique that Marx made against capitalism. Capitalism, Marx argued, impugns the dignity of labor because it treats it like a means, not an end. No such argument could take root in a Lockean system, because Lockean ethics has no such prescription against treating others instrumentally.

Germany lost its way at the start of the 20th century. But the post-war regime, funded by the Marshall plan, directed by U.S. constitutional scholars as well as repatriating German intellectuals, had the opportunity to rewrite their system of governance. They did so along Kantian lines: with statutory law, reflecting a priori rational inquiry, instead of empiricist Common Law. They were able to enshrine into their system the Kantian basis of ethics, with its focus on autonomy.

Many of the intellectuals influencing the creation of the new German state were “Marxist” in the loose sense that they were educated in the German continental intellectual tradition which, at that time, included Marx as one of its key figures. By the mid-20th century they had naturally surpassed this ideological view. However, as a consequence, the McCarthyist attack on Marxism had the effect of also purging some of the philosophical connection between German and U.S. legal education. Kantian notions of autonomy are still quite foreign to American jurisprudence. Legal arguments in the United States draw instead on a vast collection of other tools based on a much older and more piecemeal way of establishing rights. But are any of these tools up to the task of protecting human dignity?

The EU is very much influenced by Germany and the German legal system. The EU has the Kantian autonomy ethic at the heart of its conception of human rights. This philosophical commitment has recently expressed itself in the EU’s assertion of data protection law through the GDPR, whose transnational enforcement clauses have brought this centuries-old philosophical fight into contemporary legal debate in legal jurisdictions that predate the neo-Kantian legal innovations of Continental states.

The puzzle facing American legal scholars is this: while industrial advocates and representatives tend to disagree with the strength of the GDPR, arguing that it is unworkable and/or based on poorly defined principle, the data protections that it offer seem so far to be compelling to users, and the shifting expectations around privacy in part induced by it are having effects on democratic outcomes (such as the CCPA). American legal scholars now have to try to make sense of the GDPR’s rules and find a normative basis for them. How can these expansive ideas of data protection, which some have had the audacity to argue is a new right (Hildebrandt, 2015), be grafted onto the the Common Law, empiricist legal system in a way that gives it the legitimacy of being an authentically American project? Is there a way to explain data protection law that does not require the transcendental philosophical apparatus which, if adopted, would force the American mind to reconsider in a fundamental way the relationship between individuals and the collective, labor and capital, and other cornerstones of American ideology?

There may or may not be. Time will tell. My own view is that the corporate powers, which flourished under the Lockean judicial system because of the weaknesses in that philosophical model of the individual and her rights, will instinctively fight what is in fact a threatening conception of the person as autonomous by virtue of their transcendental similarity with other people. American corporate power will not bother to make a philosophical case at all; it will operate in the domain of realpolitic so well documented by Cohen. Even if this is so, it is notable that so much intellectual and economic energy is now being exerted in the friction around a poweful an idea.

References

Cohen, J. E. (2019). Between Truth and Power: The Legal Constructions of Informational Capitalism. Oxford University Press, USA.

Hildebrandt, M. (2015). Smart technologies and the end (s) of law: Novel entanglements of law and technology. Edward Elgar Publishing.

Neutral, Autonomous, and Pluralistic conceptions of law and technology (Hildebrandt, Smart Technologies, sections 8.1-8.2)

Continuing notes and review of Part III of Hildebrandt’s Smart Technologies and the End(s) of Law, we begin chapter 8, “Intricate entanglements of law and technology”. This chapter culminates in some very interesting claims about the relationship between law and the printing press/text, which I anticipate provide some very substantive conclusions.

But the chapter warms up by a review of philosophical/theoretical positions on law and technology more broadly. Section 8.2. is structured as a survey of these positions, and in an interesting way: Hildebrandt lays out Neutral, Autonomous, and Pluralistic conceptions of both technology and law in parallel. This approach is dialectical. The Neutral and Autonomous conceptions are, Hildebrandt argues, narrow and naive; the Pluralistic conception captures nuances necessary to understand not only what technology and law are, but how they relate to each other.

The Neutral Conception

This is the conception of law and technology as mere instruments. A particular technology is not good or bad, it all depends on how it’s used. Laws are enacted to reach policy aims.

Technologies are judged by their affordances. The goals for which they are used can be judged, separately, using deontology or some other basis for the evaluation of values. Hildebrandt has little sympathy for this view: “I believe that understanding technologies as mere means amounts to taking a naive and even dangerous position”. That’s because, for example, technology can impact the “in-between” of groups and individuals, thereby impacting privacy by its mere usage. This echoes the often cited theme of how artifacts have politics (Winner, 1980): by shaping the social environment by means of their affordances.

Law can also be thought of as neutral instrument. In this case, it is seen as a tool of social engineering, evaluated for its effects. Hildebrandt says this view of law fits “the so-called regulatory paradigm”, which “reigns in policy circles, and also in policy science, which is a social science inclined to take an exclusively external perspective on the law”. The law regulates behavior externally, rather than the actions of citizens internally.

Hildebrandt argues that when law is viewed instrumentally, it is tempting to then propose that the same instrumental effects could be achieved by technical infrastructure. “Techno-regulation is a prime example of what rule by law ends up with; replacing legal regulation with technical regulation may be more efficient and effective, and as long as the default settings are a part of the hidden complexity people simply lack the means to contest their manipulation.” This view is aligned with Lessig’s (2009), which Hildebrandt says is “deeply disturbing”; as it is aligned with “the classical law and economics approach of the Chicago School”, it falls short…somehow. This argument will be explicated in later sections.

Comment

Hildebrandt’s criticism of the neutral conception of technology is that it does not register how technology (especially infrastructure) can have a regulatory effect on social life and so have consequences that can be normatively evaluated without bracketing out the good or bad uses of it by individuals. This narrow view of technology is precisely that which has been triumphed over by scholars like Lessig.

Hildebrandt’s criticism of the neutral conception of law is different. It is that by understanding law primarily by its external effects (“rule by law”) diminishes the true normative force of a more robust legality that sees law as necessarily enacted and performed by people (“Rule of Law”). But nobody would seriously think that “rule by law” is not “neutral” in the same sense that some people think technology is neutral.

The misalignment of these two positions, which are presented as if they are equivalent, obscures a few alternative positions in the logical space of possibilities. There are actually two different views of the neutrality of technology: the naive one that Hildebrandt takes time to dismiss, and the more sophisticated view that technology should be judged by its social effects just as an externally introduced policy ought to be.

Hildebrandt shoots past this view, as developed by Lessig and others, in order to get to a more robust defense of Rule of Law. But it has to be noted that this argument for the equivalence of technology and law within the paradigm of regulation has beneficial implications if taken to its conclusion. For example, in Deirdre Mulligan’s FAT* 2019 keynote, she argued that public sector use of technology, if recognizes as a form of policy, would be subject to transparency and accountability rules under laws like the Administrative Procedure Act.

The Autonomous Conception

In the autonomous conception of technology and law, there is no agent using technology or law for particular ends. Rather, Technology and Law (capitalized) act with their own abstract agency on society.

There are both optimistic and pessimistic views of Autonomous Technology. There is hyped up Big Data Solutionism (BDS), and dystopian views of Technology as the enframing, surveilling, overpowering danger (as in, Heidegger). Hildebrandt argues that these are both naive and dangerous views that prevent us from taking seriously the differences between particular technologies. Hildebrant maintains that particular design decisions in technology matter. We just have to think about the implications of those decisions in a way that doesn’t deny the continued agency involved the continuous improvement, operation, and maintenance of the technology.

Hildebrant associates the autonomous conception of law with legal positivism, the view of law as a valid, existing rule-set that is strictly demarcated from either (a) social or moral norms, or (b) politics. The law is viewed as legal conditions for legal effects, enforced by a sovereign with a monopoly on violence. Law, in this sense, legitimizes the power of the state. It also creates a class of lawyers whose job it is to interpret, but not make, the law.

Hildebrandt’s critique of the autonomous conception of law is that it gives the law too many blind spots. If Law is autonomous, it does not need to concern itself with morality, or with politics, or with sociology, and especially not with the specific technology of Information-Communications Infrastructure (ICI). She does not come out and say this outright, but the implication is that this view of Law is fragile given the way changes in the ICI are rocking the world right now. A more robust view of law would give better tools for dealing with the funk we’re in right now.

The Pluralistic Conception

The third view of technology and law, the one that Hildebrandt endorses, is the “pluralistic” or “relational” view of law. It does not come as a surprise after the exploration of the “neutral” and “autonomous” conceptions.

The way I like to think about this, the pluralistic conception of technology/law, is: imagine that you had to think about technology and law in a realistic way, unburdened by academic argument of any kind. Imagine, for example, a room in an apartment. Somebody built the room. As a consequence of the dimensions of the room, you can fit a certain amount of furniture in it. The furniture has affordances; you can sit at chairs and eat at tables. You might rearrange the furniture sometimes if you want a different lifestyle for yourself, and so on.

In the academic environment, there are branches of scholarship that like to pretend they discovered this totally obvious view of technology for the first time in, like, the 70’s or 80’s. But that’s obviously wrong. As Winner (1980) points out, when Ancient Greeks were building ships, they obviously had to think about how people would work together to row and command the ship, and built it to be functional. Civil engineering, transportation engineering, and architecture are fields that deal with socially impactful infrastructure, and they have to deal with the ways people react, collectively, to what was built. I can say from experience doing agile development of software infrastructure that software engineers, as well, think about their users when they build products.

So, we might call this the “realistic” view–the view that engineers, who are the best situated to understand the processes of producing and maintaining technology, since that’s their life, have.

I’ve never been a lawyer, but I believe one gets to the pluralistic, or relational, view of law in pretty much the same way. You look at how law has actually evolved, historically, and how it has always been wrapped up in politics and morality and ICI’s.

So, in these sections, Hildebrandt drives home in a responsible, scholarly way the fact that neither law nor technology (especially technological infrastructure, and especially ICI) are autonomous–they are historically situated creates of society–and nor are they instrumentally neutral–they do have a form of agency in their own right.As my comment above notes, to me the most interesting part of this chapter was the gaps and misalignment in the section on the Neutral Conception section. This conception seems most aligned with an analytically clear, normative conception of what law and technology are supposed to be doing, which is what makes this perspective enduringly attractive to those who make them. The messiness or the pluralistic view, while more nuanced, does not provide a guide for design.

By sweeping away the Neutral conception of law as instrumental, Hildebrandt preempts arguments that the law might fail to attain its instrumental goals, or that the goals of law might sometimes be attained through infrastructure. In other words, Hildebrandt is trying to avoid a narrow instrumental comparison between law and technology, and highlights instead that they are relationally tied to each other in a way that prevents either from being a substitute for the other.

References

Hildebrandt, Mireille. Smart technologies and the end (s) of law: novel entanglements of law and technology. Edward Elgar Publishing, 2015.

Lessig, Lawrence. Code: And other laws of cyberspace. ReadHowYouWant. com, 2009.

Winner, Langdon. “Do artifacts have politics?.” Daedalus(1980): 121-136.

Antinomianism and purposes as reasons against computational law (Notes on Hildebrandt, Smart Technologies, Sections 7.3-7.4)

Many thanks to Jake Goldenfein for discussing this reading with me and coaching me through interpreting it in preparation for writing this post.

Following up on the discussion of sections 7.1-7.2 of Hildebrandt’s Smart Technologies an the End(s) of Law (2015), this post discusses the next two sections. The main questions left from the last section are:

  • How strong is Hildebrandt’s defense of the Rule of Law, as she explicates it, as worth preserving despite the threats to it that she acknowledges from smart technologies?
  • Is the instrumental power of smart technology (i.e, its predictive function, which for the sake of argument we will accept is more powerful than unassisted human prognostication) somehow a substitute for Law, as in its pragmatist conception?

In sections 7.3-7.4, Hildbrandt discusses the eponymous ends of law. These are not its functions as could be externally and sociologically validated, but rather its internally recognized goals or purposes. And these are not particular goals, such as environmental justice, that we might want particular laws to achieve. Rather, these are abstract goals that the law as an entire ‘regime of veridiction’ aims for. (“Veridiction” means “A statement that is true according to the worldview of a particular subject, rather than objectively true.” The idea is that the law has a coherent worldview of its own.

Hildebrandt’s description of law is robust and interesting. Law “articulates legal conditions for legal effect.” Legal personhood (a condition) entails certain rights under the law (an effect). These causes-and-effects are articulated in language, and this language does real work. In Austin’s terminology, legal language is performative–it performs things at an institutional and social level. Relatedly, the law is experienced as a lifeworld, or Welt, but not a monolithic lifeworld that encompasses all experience, but one of many worlds that we use to navigate reality, a ‘mode of existence’ that ‘affords specific roles, actors and actions while constraining others’. [She uses Latour to make this point, which in my opinion does not help.] It is interesting to compare this view of society with Nissenbaum’s ((2009) view of society differentiated into spheres, constituted by actor roles and norms.

In section 7.3.2, Hildebrandt draws on Gustav Radbruch for his theory of law. Consistent with her preceding arguments, she emphasizes that for Radbruch, law is antinomian, (a strange term) meaning that it is internally contradictory and unruly, with respect to its aims. And there are three such aims that are in tension:

  • Justice. Here, justice is used rather narrowly to mean that equal cases should be treated equally. In other words, the law must be applied justly/fairly across cases. To use her earlier framing, justice/equality implied that legal conditions cause legal effects in a consistent way. In my gloss, I would say this is equivalent to the formality of law, in the sense that the condition-effect rules must address the form of a case, and not treat particular cases differently. More substantively, Hildebrandt argues that Justice breaks down into more specific values: distributive justice, concerning the fair distribution of resources across society, and corrective justice, concerning the righting of wrongs through, e.g., torts.
  • Legal certainty. Legal rules must be binding and consistent, whether or not they achieve justice or purpose. “The certainty of the law requires its positivity; if it cannot be determined what is just, it must be decided what is lawful, and this from a position that is capable of enforcing the decision.” (Radbruch). Certainty about how the law will be applied, whether or not the application of the law is just (which may well be debated), is a good in itself. [A good example of this is law in business, which is famously one of the conditions for the rise of capitalism.]
  • Purpose. Beyond just/equal application of the law across cases and its predictable positivity, the law aims at other purposes such as social welfare, redistribution of income, guarding individual and public security, and so on. None of these purposes is inherent in the law, for Radbruch; but in his conception of law, by its nature it is directed by democratically determined purposes and is instrumental to them. These purposes may flesh out the normative detail that’s missing in a more abstract view of law.

Two moves by Hildebrandt in this section seem particularly substantial to her broader argument and corpus of work.

The first is the emphasis on the contrast between the antinomian conflict between justice, certainty, and purpose with the principle of legal certainty itself. Law, at any particular point in time, may fall short of justice or purpose, and must nevertheless be predictably applied. It also needs to be able to evolve towards its higher ends. This, for Hildebrandt, reinforces the essential ambiguous and linguistic character of law.

[Radbruch] makes it clear that a law that is only focused on legal certainty could not qualify as law. Neither can we expect the law to achieve legal certainty to the full, precisely because it must attend to justice and to purpose. If the attribution of legal effect could be automated, for instance by using a computer program capable of calculating all the relevant circumstances, legal certainty might be achieved. But this can only be done by eliminating the ambiguity that inheres in human language: it would reduce interpretation to mindless application. From Radbruch’s point of view this would fly in the face of the cultural, value-laden mode of existence of the law. It would refute the performative nature of law as an artificial construction that depends on the reiterant attribution of meaning and decision-making by mindful agents.

Hildebrandt, Smart Technologies, p. 149

The other move that seems particular to Hildebrandt is the connection she draws between purpose as one of the three primary ends of law and purpose-binding a feature of governance. The latter has particular relevance to technology law through its use in data protection, such as in the GDPR (which she addresses elsewhere in work like Hildebrandt, 2014). The idea here is that purposes do not just imply a positive direction of action; they also restrict activity to only those actions that support the purpose. This allows for separate institutions to exist in tension with each other and with a balance of power that’s necessary to support diverse and complex functions. Hildebrandt uses a very nice classical mythology reference here

The wisdom of the principle of purpose binding relates to Odysseus’s encounter with the Sirens. As the story goes, the Sirens lured passing sailors with the enchantment of their seductive voices, causing their ships to crash on the rocky coast. Odysseus wished to hear their song without causing a shipwreck; he wanted to have his cake and eat it too. While he has himself tied to the mast, his men have their ears plugged with beeswax. They are ordered to keep him tied tight, and to refuse any orders he gives to the contrary, while being under the spell of the Sirens as they pass their island. And indeed, though he is lured and would have caused death and destruction if his men had not been so instructed, the ship sails on. This is called self-binding. But it is more than that. There is a division of tasks that prevents him from untying himself. He is forced by others to live by his own rules. This is what purpose binding does for a constitutional democracy.

Hildebrandt, Smart Technologies, p. 156

I think what’s going on here is that Hildebrandt understands that actually getting the GDPR enforced over the whole digital environment is going to require a huge extension of the powers of law over business, organization, and individual practice. From some corners, there’s pessimism about the viability of the European data protection approach (Koops, 2014), arguing that it can’t really be understood or implemented well. Hildebrandt is making a big bet here, essentially saying: purpose-binding on data use is just a natural part of the power of law in general, as a socially performed practice. There’s nothing contingent about purpose-binding in the GDPR; it’s just the most recent manifestation of purpose as an end of law.

Commentary

It’s pretty clear what the agenda of this work is. Hildebrandt is defending the Rule of Law as a social practice of lawyers using admittedly ambiguous natural language over the ‘smart technologies’ that threaten it. This involves both a defense of law as being intrinsically about lawyers using ambiguous natural language, and the power of that law over businesses, etc. For the former, Hildebrandt invokes Radbruch’s view that law is antinomian. For the second point, she connects purpose-binding to purpose as an end of law.

I will continue to play the skeptic here. As is suggested in the quoted package, if one takes legal certainty seriously, then one could easily argue that software code leads to more certain outcomes than natural language based rulings. Moreover, to the extent that justice is a matter of legal formality–attention to the form of cases, and excluding from consideration irrelevant content–then that too weighs in favor of articulation of law in formal logic, which is relatively easy to translate into computer code.

Hildebrandt seems to think that there is something immutable about computer code, in a way that natural language is not. That’s wrong. Software is not built like bridges; software today is written by teams working rapidly to adapt it to many demands (Gürses and Hoboken, 2017). Recognizing this removes one of the major planks of Hildebrandt’s objection to computational law.

It could be argued that “legal certainty” implies a form of algorithmic interpretability: the key question is “certain for whom”. An algorithm that is opaque due to its operational complexity (Burrell, 2016) could, as an implementation of a legal decision, be less predictable to non-specialists than a simpler algorithm. So the tension in a lot of ‘algorithmic accountability’ literature between performance and interpretability would then play directly into the tension, within law, between purpose/instrumentality and certainty-to-citizens.

Overall, the argument here is not compelling yet as a refutation of the idea of law implemented as software code.

As for purpose-binding and the law, I think this may well be the true crux. I wonder if Hildebrandt develops it later in the book. There are not a lot of good computer science models of purpose binding. Tschantz, Datta, and Wing (2012) do a great job mapping out the problem but that research program has not resulted in robust technology for implementation. There may be deep philosophical/mathematical reasons why that is so. This is an angle I’ll be looking out for in further reading.

References

Burrell, Jenna. “How the machine ‘thinks’: Understanding opacity in machine learning algorithms.” Big Data & Society3.1 (2016): 2053951715622512.

Gürses, Seda, and Joris Van Hoboken. “Privacy after the agile turn.” The Cambridge Handbook of Consumer Privacy. Cambridge Univ. Press, 2017. 1-29.

Hildebrandt, Mireille. “Location Data, Purpose Binding and Contextual Integrity: What’s the Message?.” Protection of Information and the Right to Privacy-A New Equilibrium?. Springer, Cham, 2014. 31-62.

Hildebrandt, Mireille. Smart technologies and the end (s) of law: novel entanglements of law and technology. Edward Elgar Publishing, 2015.

Koops, Bert-Jaap. “The trouble with European data protection law.” International Data Privacy Law 4.4 (2014): 250-261.

Nissenbaum, Helen. Privacy in context: Technology, policy, and the integrity of social life. Stanford University Press, 2009.

Tschantz, Michael Carl, Anupam Datta, and Jeannette M. Wing. “Formalizing and enforcing purpose restrictions in privacy policies.” 2012 IEEE Symposium on Security and Privacy. IEEE, 2012.

Beginning to read “Smart Technologies and the End(s) of Law” (Notes on: Hildebrandt, Smart Technologies, Sections 7.1-7.2)

I’m starting to read Mireille Hildebrandt‘s Smart Technologies and the End(s) of Law (2015) at the recommendation of several friends with shared interests in privacy and the tensions between artificial intelligence and the law. As has been my habit with other substantive books, I intend to blog my notes from reading as I get to it, in sections, in a perhaps too stream-of-consciousness, opinionated, and personally inflected way.

For reasons I will get to later, Hildebrandt’s book is a must-read for me. I’ve decided to start by jumping in on Chapter 7, because (a) I’m familiar enough with technology ethics, AI, and privacy scholarship to think I can skip that and come back as needed, and (b) I’m mainly reading because I’m interested in what a scholar of Hildebrandt’s stature says when she tackles the tricky problem of law’s response to AI head on.

I expect to disagree with Hildebrant in the end. We occupy different social positions and, as I’ve argued before, people’s position on various issues of technology policy appears to have a great deal to do with their social position or habitus. However, I know I have a good deal to learn about legal theory while having enough background in philosophy and social theory to parse through what Hildebrandt has to offer. And based on what I’ve read so far, I expect the contours of the possible positions that she draws out to be totally groundbreaking.

Notes on: Hildebrandt, Smart Technologies, §7.1-7.2

“The third part of this book inquires into the implications of smart technologies and data-driven agency for the law.”

– Hildebrandt, Smart Technologies,p.133

Lots of people write about how artificial intelligence presents an existential threat. Normally, they are talking about how a superintelligence is posing an existential threat to humanity. Hildebrandt is arguing something else: she is arguing that smart technologies may pose an existential threat to the law, or the Rule of Law. That is because the law’s “mode of existence” depends on written text, which is a different technical modality, with different affordances, than smart technology.

My take is that the mode of existence of modern law is deeply dependent upon the printing press and the way it has shaped our world. Especially the binary character of legal rules, the complexity of the legal system and the finality of legal decisions are affordances of — amongst things — the ICI [information and communication infrastructure] of the printing press.

– Hildebrandt, Smart Technologies, p.133

This is just so on point, it’s hard to know what to say. I mean, this is obviously on to something. But what?

To make her argument, Hildebrandt provides a crash course in philosophy of law and legal theory, distinguishing a number of perspectives that braid together into an argument. She discusses several different positions:

  • 7.2.1 Law as an essentially contested concept (Gallie). The concept of “law” [1] denotes something valuable, [2] covers intricate complexities, that makes it [3] inherently ambiguous and [4] necessarily vague. This [5] leads interested parties into contest over conceptions. The contest is [6] anchored in past, agreed upon exemplars of the concept, and [7] the contest itself sustains and develops the concept going forward. This is the seven-point framework of an “essentially contested concept”.
  • 7.2.2 Formal legal positivism. Law as a set of legal rules dictated by a sovereign (as opposed to law as a natural moral order) (Austin). Law as a coherent set of rules, defined by its unity (Kelsen). A distinction between substantive rules and rules about rule-making (Hart).
  • 7.2.3 Hermeneutic conceptions. The practice of law is about the creative interpretation of (e.g.) texts (case law, statutes, etc.) to application of new cases. The integrity of law (Dworkin) constrains this interpretation, but the projection of legal meaning into the future is part of the activity of legal practice. Judges “do things with words”–make performative utterances through their actions. Law is not just a system of rules, but a system of meaningful activity.
  • 7.2.3 Pragmatist conceptions (Realism legal positivism). As opposed to the formal legal positivism discusses earlier that sees law as rules, realist legal positivism sees law as a sociological phenomenon. Law is “prophecies of what the courts will do in fact, and nothing more pretentious” (Holmes). Pragmatism, as an epistemology, argues that the meaning of something is its practical effect; this approach could be seen as a constrained version of the hermeneutic concept of law.

To summarize Hildebrandt’s gloss on this material so far: Gallie’s “essentially contested concept” theory is doing the work of setting the stage for Hildebrant’s self-aware intervention into the legal debate. Hildebrandt is going to propose a specific concept of the law, and of the Rule of Law. She is doing this well-aware that this act of scholarship is engaging in contest.

Punchline

I detect in Hildebrandt’s writing a sympathy or preference for hermeneutic approaches to law. Indeed, by opening with Gallie, she sets up the contest about the concept of law as something internal to the hermeneutic processes of the law. These processes, and this contest, are about texts; the proliferation of texts is due to the role of the printing press in modern law. There is a coherent “integrity” to this concept of law.

The most interesting discussion, in my view, is loaded in to what reads like an afterthought: the pragmatist conception of law. Indeed, even at the level of formatting, pragmatism is buried: hermeneutic and pragmatist conceptions of law are combined into one section (7.2.3), where as Gallie and the formal positivists each get their own section (7.2.1 and 7.2.2).

This is odd, because the resonances between pragmatism and ‘smart technology’ are, in Hildebrandt’s admission, quite deep:

Basically, Holmes argued that law is, in fact, what we expect it to be, because it is this expectation that regulates our actions. Such expectations are grounded in past decisions, but if these were entirely deterministic of future decisions we would not need the law — we could settle for logic and simply calculate the outcome of future decisions. No need for interpretation. Holmes claimed, however, that ‘the life of law has not been logic. It has been experience.’ This correlates with a specific conception of intelligence. As we have seen in Chapter 2 and 3, rule-based artificial intelligence, which tried to solve problems by means of deductive logic, has been superseded by machine learning (ML), based on experience.

– Hildebrandt, Smart Technologies, p.142

Hildebrandt considers this connection between pragmatist legal interpretation and machine learning only to reject it summarily in a single paragraph at the end of the section.

If we translate [a maxim of classical pragmatist epistemology] into statistical forecasts we arrive at judgments resulting from ML. However, neither logic nor statistics can attribute meaning. ML-based court decisions would remove the fundamental ambiguity of human language from the centre stage of the law. As noted above, this ambiguity is connected with the value-laden aspect of the concept of law. It is not a drawback of natural language, but what saves us from acting like mindless agents. My take is that an approach based on statistics would reduce judicial and legislative decisions to administration, and thus collapse the Rule of Law. This is not to say that a number of administrative decisions could not be taken by smart computing systems. It is to confirm that such decisions should be brought under the Rule of Law, notably by making them contestable in a court of law.

– Hildebrandt, Smart Technologies, p.143

This is a clear articulation of Hildebrandt’s agenda (“My take is that…”). It is also clearly an aligning the practice of law with contest, ambiguity, and interpretation as opposed to “mindless” activity. Natural language’s ambiguity is a feature, not a bug. Narrow pragmatism, which is aligned with machine learning, is a threat to the Rule of Law

Some reflections

Before diving into the argument, I have to write a bit about my urgent interest in the book. Though I only heard about it recently, my interests have tracked the subject matter for some time.

For some time I have been interested in the connection between philosophical pragmatism and the concerns about AI, which I believe can be traced back to Horkheimer. But I thought nobody was giving the positive case for pragmatism its due. At the end of 2015, totally unaware of “Smart Technologies” (my professors didn’t seem aware of it either…), I decided that I would write my doctoral dissertation thesis defending the bold thesis that yes, we should have AI replace the government. A constitution written in source code. I was going to back the argument up with, among other things, pragmatist legal theory.

I had to drop the argument because I could not find faculty willing to be on the committee for such a dissertation! I have been convinced ever since that this is a line of argument that is actually rather suppressed. I was able to articulate the perspective in a philosophy journal in 2016, but had to abandon the topic.

This was probably good in the long run, since it meant I wrote a dissertation on privacy which addressed many of the themes I was interested in, but in greater depth. In particular, working with Helen Nissenbaum I learned about Hildebrandt’s articles comparing contextual integrity with purpose binding in the GDPR (Hildebrandt, 2013; Hildebrandt, 2014), which at the time my mentors at Berkeley seemed unaware of. I am still working on puzzles having to do with algorithmic implementation or response to the law, and likely will for some time.

Recently, been working at a Law School and have reengaged the interdisciplinary research community at venues like FAT*. This has led me, seemingly unavoidably, back to what I believe to be the crux of disciplinary tension today: the rising epistemic dominance of pragmatist computational statistics–“data science”and its threat to humanistic legal authority, which is manifested in the clash of institutions that are based on each, e.g., iconically, “Silicon Valley” (or Seattle) and the European Union. Because of the explicitly normative aspects of humanistic legal authority, it asserts itself again and again as an “ethical” alternative to pragmatist technocratic power. This is the latest manifestation of a very old debate.

Hildebrandt is the first respectable scholar (a category from which I exclude myself) that I’ve encountered to articulate this point. I have to see where she takes the argument.

So far, however, I think here argument begs the question. Implicitly, the “essentially contested” character of law is due to the ambiguity of natural language and the way in which that necessitates contest over the meaning of words. And so we have a professional class of lawyers and scholars that debate the meaning of words. I believe the the regulatory power of this class is what Hildebrandt refers to as “the Rule of Law”.

While it’s true that an alternative regulatory mechanism based on statistical prediction would be quite different from this sense of “Rule of Law”, it is not clear from Hildebrandt’s argument, yet, why her version of “Rule of Law” is better. The only hint of an argument is the problem of “mindless agents”. Is she worried about the deskilling of the legal profession, or the reduced need for elite contest over meaning? What is hermeneutics offering society, outside of the bounds of its own discourse?

References

Benthall, S. (2016). Philosophy of computational social science. Cosmos and History: The Journal of Natural and Social Philosophy12(2), 13-30.

Sebastian Benthall. Context, Causality, and Information Flow: Implications for Privacy Engineering, Security, and Data Economics. Ph.D. dissertation. Advisors: John Chuang and Deirdre Mulligan. University of California, Berkeley. 2018.

Hildebrandt, Mireille. “Slaves to big data. Or are we?.” (2013).

Hildebrandt, Mireille. “Location Data, Purpose Binding and Contextual Integrity: What’s the Message?.” Protection of Information and the Right to Privacy-A New Equilibrium?. Springer, Cham, 2014. 31-62.

Hildebrandt, Mireille. Smart technologies and the end (s) of law: novel entanglements of law and technology. Edward Elgar Publishing, 2015.

Trade secrecy, “an FDA for algorithms”, a software bills of materials (SBOM) #SecretAlgos

At the Conference on Trade Secrets and Algorithmic Systems at NYU today, the target of most critiques is the use of trade secrecy by proprietary technology providers to prevent courts and the public from seeing the inner workings of algorithms that determine people’s credit scores, health care, criminal sentencing, and so on. The overarching theme is that sometimes companies will use trade secrecy to hide the ways that their software is bad, and that that is a problem.

In one panel, the question of whether an “FDA for Algorithms” is on the table–referring the Food and Drug Administration’s approval of pharmaceuticals. It was not dealt with in too much depth, which is too bad, because it is a nice example of how government oversight of potentially dangerous technology is managed in a way that respects trade secrecy.

According to this article, when filing for FDA approval, a company can declare some of their ingredients to be trade secrets. The upshot of that is that those trade secrets are not subject to FOIA requests. However, these ingredients are still considered when approval is granted by the FDA.

It so happens that in the cybersecurity policy conversation (more so than in privacy) the question of openness of “ingredients” to inspection has been coming up in a serious way. NTIA has been hosting multistakeholder meetings about standards and policy around Software Component Transparency. In particular they are encouraging standardizations of Software Bills of Materials (SBOM) like the Linux Foundation’s Software Package Data Exchange (SPDX). SPDX (and SBOM’s more generally) describe the “ingredients” in a software package at a higher level of resolution than exposing the full source code, but at a level specific enough useful for security audits.

It’s possible that a similar method could be used for algorithmic audits with fairness (i.e., nondiscrimination compliance) and privacy (i.e., information sharing to third-parties) in mind. Particular components could be audited (perhaps in a way that protects trade secrecy), and then those components could be listed as “ingredients” by other vendors.

What proportion of data protection violations are due to “dark data” flows?

“Data protection” refers to the aspect of privacy that is concerned with the use and misuse of personal data by those that process it. Though widely debated, scholars continue to converge (e.g.) on ideal data protection consisting of alignment between the purposes the data processor will use the data for and the expectations of the user, along with collection limitations that reduce exposure to misuse. Through its extraterritorial enforcement mechanism, the GDPR has threatened to make these standards global.

The implication of these trends is that there will be a global field of data flows regulated by these kinds of rules. Many of the large and important actors that process user data can be held accountable to the law. Privacy violations by these actors will be due to a failure to act within the bounds of the law that applies to them.

On the other hand, there is also cybercrime, an economy of data theft and information flows that exists “outside the law”.

I wonder what proportion of data protection violations are due to dark data flows–flows of personal data that are handled by organizations operating outside of any effective regulation.

I’m trying to draw an analogy to a global phenomenon that I know little about but which strikes me as perhaps more pressing than data protection: the interrelated problems of money laundering, off-shore finance, and dark money contributions to election campaigns. While surely oversimplifying the issue, my impression is that the network of financial flows can be divided into those that are more and less regulated by effective global law. Wealth seeks out these opportunities in the dark corners.

How much personal data flows in these dark networks? And how much is it responsible for privacy violations around the world? Versus how much is data protection effectively in the domain of accountable organizations (that may just make mistakes here and there)? Or is the dichotomy false, with truly no firm boundary between licit and illicit data flow networks?

the resilience of agonistic control centers of global trade

This post is merely notes; I’m fairly confident that I don’t know what I’m writing about. However, I want to learn more. Please recommend anything that could fill me in about this! I owe most of this to discussion with a colleague who I’m not sure would like to be acknowledged.

Following the logic of James Beniger, an increasingly integrated global economy requires more points of information integration and control.

Bourgeois (in the sense of ‘capitalist’) legal institutions exist precisely for the purpose of arbitrating between merchants.

Hence, on the one hand we would expect international trade law to be Habermasian. However, international trade need not rest on a foundation of German idealism (which increasingly strikes me as the core of European law). Rather, it is an evolved mechanism.

A key part of this mechanism, as I’ve heard, is that it is decentered. Multiple countries compete to be the sites of transnational arbitration, much like multiple nations compete to be tax havens. Sovereignty and discretion are factors of production in the economy of control.

This means, effectively, that one cannot defeat capitalism by chopping off its head. It is rather much more like a hydra: the “heads” are the creation of two-sided markets. These heads have no internalized sense of the public good. Rather, they are optimized to be attractive to the transnational corporations in bilateral negotiation. The plaintiffs and defendants in these cases are corporations and states–social forms and institutions of complexity far beyond that of any individual person. This is where, so to speak, the AI’s clash.