We seem to be in a new moment of media excitement about the implications of artificial intelligence. This time, the moment is driven by the experience of software engineers and other knowledge workers who are automating their work with ‘agents’. Clause Code etc. The latest generation of models and services is really good at doing things.
Does this change anything about my “position on AI” and superintelligence in particular?
I wrote a brief paper in 2017 about Bostrom’s Superintelligence argument. I concluded that algorithmic self-improvement at the software level would not produce superintelligence. Rather, intelligence group is limited by data and hardware.
In 2025, this conclusion still holds up, as we’ve seen that the recent impressive advances in AI has depended on tremendous capital expenditure on data centers, high-performing chips, and energy. It also depends on well-publicized efforts to collect all the text known to humankind for training data.
About 8 years ago when I was thinking about this, I wrote a bit about the connection between the Superintelligence argument and the Frankfurt School’s views on instrumental reason and capitalism. The alignment of AI with capital has born out, and has been written about by many others. What is striking about the current moment is just how on-the-nose that alignment is in the US, in terms of the full stack of energy, hardware, models, applications, and then some.
So, so far, no update.
In 2021 I published an article saying that we already had artificial systems with the capacity to outperform individual humans at many tasks. They were and still are called corporations or firms. We also had replaced markets with platform, which are similarly more performant in terms of reducing transaction costs. In that article, Jake Goldenfein and I argue that what ultimately matters are the purposes of the social system that operates the AI technology.
I believe this argument also continues to hold up. The successful models and service we are seeing are corporate accomplishments. The corporation is still the relevant unit of analysis when considering AI.
There are a number of interesting things happening now which I think are undertheorized:
What is the real economics of AI, given that the supply chains are so long and complex, consistent of both material and intellectual inputs, and the market for demand is uncertain? This is the trillion dollar question in terms of valuations, and it’s unanswered. The empirics here are not very good because things are far out of equlibrium.
Put another way: what does AI mean for the relationships between capital, corporations, labor, and consumers? Some of these relationships are mediated by rules about corporate law, intellectual property and data use, and so are determinable by law rather than technology. Information law therefor is a key point of political intervention in an economic system that is otherwise determined by laws of nature (energy, computation, etc.?
To put it another way: superintelligence has been happening and continues to happen. Some of this is due to laws of nature. But there is still a meaningful point of human intervention, which is the laws of humanity. Designing and implementing those laws well remains an important challenge.
One last thought. I’ve been inspired by Beninger’s The Control Revolution (1986) which is a historical account of the information economy in terms of cybernetics and information theory. You can ask an AI to tell you more about it, but one item comes to mind: that each new information technology first seems to threaten the jobs of people doing information work, and then leads to an expanded number of information jobs. This has to do with the way complexity is and is not managed by the technology. There’s an open question whether this generation of AI is any different. The question is truly open, but my hunch at the moment is that today’s AI systems are creating a lot more complexity than they are controlling. We will see.
One thing that’s become clear to me lately is that I now believe that ethics can be naturalized. I also believe that there is in fact a form of ‘natural law’. By this I mean that that there are rights and values that are inherent to human nature. Real legal systems can either lie up to natural law, or not.
This is not the only position that it’s possible to take on these topics.
One different position, that I do not have, is that ethics depends on the supernatural. I bring this up because religion is once again very politically salient in the United States. Abrahamic religions ground ethics and morality in a covenant between humans and a supernatural God. Divine power authorizes the ethical code. In some cases this is explicitly stated law, in others it is a set of principles. Beyond divine articulation, this position maintains that ethics are supernaturally enforced through reward and punishment. I don’t think this is how things work.
Another position I don’t have is that there is that ethics are opinion or cultural construction, full stop. Certainly there’s a wide diversity of opinions on ethics and cultural attitudes. Legal systems vary from place to place. This diversity is sometimes used as evidence that there aren’t truths about ethics or law to be had. But that is, taken alone, a silly argument. Lots of people and legal systems are simply wrong. Moreover, moral and ethical truths can take contingency and variety into account, and they probably should. It can be true that laws should be well-adapted to some otherwise arbitrary social expectations or material conditions. And so on.
There has historically been hemming and hawing about the fact/value dichotomy. If there’s no supernatural guarantor of ethics, is the natural world sufficient to produce values beyond our animal passions? This increasingly feels like an argument from a previous century. Adequate solutions to this problem have been offered by philosophers over time. They tend to involve some form of rational or reflective process, and aggregation over the needs and opinions of people in heterogeneous circumstances. Habermas comes to mind as a one of the synthesizers of a new definition of naturalized law and ethics.
For some reason, I’ve encountered so much resistance to this form of ethical or moral realism over the years. But looking back on it, I can’t recall a convincing argument for it. I can recall many claims that the idea of ethical and moral truth are somehow politically dangerous, but that this not the same thing.
There is something teleological about most viable definitions of naturalized ethics and natural law. They are would would hypothetically be decided on by interlocutors in an idealized but not yet realized circumstance. A corollary to my position is that ethical and moral facts exist, but many have not yet been discovered. A scientific process is needed to find them. This process is necessarily a social scientific process, since ethical and moral truths are truths about social systems and how they work.
It would be very fortunate, I think, if some academic department, discipline, or research institution were to take up my position. At present, we seem to have a few different political positions available to us in the United States:
A conservative rejection of the university of being insufficiently moral because of its abandonment of God
A postmodern rejection of ethical and moral truths that relativizes everything
A positivist rejection of normativity as the object of social science because of the fact/value dichotomy
Politicized disciplines that presume a political agenda and then perform research aligned with that agenda
Explicitly normative disciplines that are discursive and humanistic but not inclined towards rigorous analysis of the salient natural facts
None of these is conducive to a scientific study of what ethics and morals should be. There are exceptions, of course, and many brilliant people in many corners who make great contributions towards this goal. But they seem scattered at the margins of the various disciplines, rather than consolidated into a thriving body of intellect. At a moment where we see profound improvements (yes, improvements!) in our capacity for reasoning and scientific exploration, why hasn’t something like this emerged? It would be an improvement over the status quo.
Push has come to shove, and I have started building something new.
I’m not building it alone, thank God, but it’s a very petite open source project at the moment.
I’m convinced that it is actually something new. Some of my colleagues are excited to hear that what I’m building will exist for them shortly. That’s encouraging! I also tried to build this thing in the context of another project that’s doing something similar. I was told that I was being too ambitious and couldn’t pull it off. That wasn’t exactly encouraging, but was good evidence that I’m actually doing something new. I will now do this thing and take the credit. So much the better for me.
What is it that I’m building?
Well, you see, it’s software for modeling economic systems. Or sociotechnical systems. Or, broadly speaking, complex systems with agents in them. Also, doing statistics with those models: fitting them to data, understanding what emergent properties occur in them, exploring counterfactuals, and so on.
I will try to answer some questions I wish somebody would ask me.
Q: Isn’t that just agent-based modeling? Why aren’t you just using NetLogo or something?
A: Agent-based modeling (ABM) is great, but it’s a very expansive term that means a lot of things. Very often, ABMs consist of agents whose behavior is governed by simple rules, rather directed towards accomplishing goals. That notion of “agent” in ABM is almost entirely opposed to the notion of “agent” used in AI — propagated by Stuart Russell, for example. To AI people, goal-directedness is essential for agency. I’m not committed to rational behavior in this framework — I’m not an economist! But I think a requirement to be able to train agents’ decision rules with respect to their goals.
There are a couple other ways in which I’m not doing paradigmatic ABM with this project. One is that I’m not focused on agents moving in 2D or 3D space. Rather, I’m much more interested in the settings defined by systems of structural equations. So, more continuous state spaces. I’m basing this work on years of contributing to heterogeneous agent macroeconomics tooling, and my frustrating with that paradigm. So, no turtles on patches. I anticipate spatial and even geospatial extensions to what I’m building would be really cool and useful. But I’m not there yet.
I think what I’m working on is ABM in the extended sense that Rob Axtell and Doyne Farmer use the term, and I hope to one day show them what I’m doing and for them to think it’s cool.
Q: Wait, is this about AI agents, as in Generative AI?
A: Ahaha… mainly no, but a little yes. I’m talking about “agents” in the more general sense used before the GenAI industry tried to make the word about them. I don’t see Generative AI or LLMs to be a fundamental part of what I’m building. However, I do see what I’m building as a tool for evaluating the economic impact and trustworthiness of GenAI systems by modeling their supply chains and social consequences. And I can imagine deeper integrations with “(generative) agentic AI” down the line. I am building a tool, and an LLM might engage it through “tool use”. It’s also I suppose possible to make the agents inside the model use LLMs somehow, though I don’t see a good reason for that at the moment.
Q: Does it use AI at all?
A: Yes! I mean, perhaps you know that “AI” has meant many things and much of what it has meant is now considered quite mundane. But it does use deep learning, which is something at “AI” means now. In particular, part of the core functionality that I’m trying to build into it is a flexible version of the deep learning econometrics methods invented not-too-long-ago by Lilia and Serguei Maliar. I hope to one day show this project to them, and for them to think it’s cool. Deep learning methods have become quite popular in economics, and this is in some ways yet-another-deep-learning-economics project. I hope it has a few features that distinguish it.
Q: How is this different from somebody else’s deep learning economics analysis package?
A: Great question! There are a few key ways that it’s different. One is that it’s designed around a clean separation between model definition and solution algorithms. There will be no model-specific solution code in this project. It’s truly intended to be library, comparable to scikit-learn, but for systems of agents. In fact, I’m calling this project scikit-agent. You heard it hear first!
Separating the model definitions from the solution algorithms means that there’s a lot more flexibility in how models are composed. This framework is based on the idea that parts of a model can be “blocks” which can be composed into more complex models. The “blocks” are bundles of structural equations, which can include state, control, and reward variables.
These ‘blocks’ are symbolically defined systems or environments. “Solving” the agent strategies in the multi-agent environment will be done with deep learning, otherwise known as artificial neural networks. So I think that it will be fair to call this framework a “neurosymbolic AI system”. I hope that saying that makes it easier to find funding for it down the line :)
Q: That sounds a little like causal game theory, or multi-agent influence diagrams. Are those part of this?
A: In fact, yes, so glad you asked. I think there’s a deep equivalence between multi-agent influence diagrams and ABM/computational economics which hasn’t been well explored. There are small notational differences that keep these communities from communicating. There are also a handful of substantively difficult theoretical issues that need to be settled with respect to, say, under what conditions a dynamic structure causal game can be solved using multi-agent reinforcement learning. These are cool problems, and I hope the thing I’m building implements good solutions to them.
Q: So, this is a framework for modeling dynamic Pearlian causal models, with multiple goal-directed agents, solving those models for agent strategy, and using those model econometrically?
A: Exactly.
Q: Does the thing you are building have any practical value? Or is it just more weird academic code?
A: I have high hopes that this thing I’m building could have a lot of practical value. Rigorous analysis of complex sociotechnical and economic systems remain hard problems. In finance, for example, as well as public policy, insurance, international relations, and other fields. I do hope what I’m building interfaces well with real data to help with significant decision-making. These are problems that Generative AI is quite bad at, I believe. I’m trying to build a strong, useful foundation for working with statistical models that include agents in them. This is more difficult than regression or even ‘transformer’-based learning from media, because the agents are solving optimization problems inside the model.
Q: What are the first applications you have in mind for this tool?
A: I’m motivated to build this because I think it’s needed to address questions in technology policy and design. This is the main subject of my NSF-funded research over the past several years. Here are some problems I’m actively working on which overlap with the scope of this tool:
Integrating Differential Privacy and Contextual Integrity. I have a working paper with Rachel Cummings where we use Structural Causal Games (SCGs) to set up the parameter tuning of a differentially private system as a mechanism design problem. The tool I’m building will be great at representing structural causal games (SCG). With it, the theoretical technique can be used in practice.
Understanding the Effects of Consumer Finance Policy. I’m working with an amazing team on a project modeling consumer lending and the effects of various consumer protection regulations. We are looking at anti-usury laws, nondiscrimination rules, forgiveness of negative information, the use of alternative data by fintech companies, and so on. This policy analysis involves comparing a number of different scenarios and looking for what regulations produce which results, robustly. I’m building a tool to solve this problem.
AI Governance and Fiduciary Duties. A lot of people have pointed out that effective AI governance requires an understanding of AI supply chains. AI services rely on complex data flows through multiple actors, which are often imperfectly aligned and incompletely contracted. They also depend on physical data centers and the consumption of energy. This raises many questions around liability and quality control that wind up ultimately being about institutional design rather than neural network architectures. I’m building a tool to help reason through these institution design questions. In other words, I’m building a new way to do threat modeling on AI supply chain ecosystems.
Q: Really?
A: Yes! I sometimes have a hard time wrapping my own head around what I’m doing, which is why I’ve written out this blog post. But I do feel very good about what I’m working on at the moment. I think it has a lot of potential.
For the private sector, there is a well-developed theory of loyalty that shows up in fiduciary duties. I and others have argued that these duties of loyalty are the right tool to bring artificial intelligence systems (and the companies that produce them) into “alignment” with those that employ them.
At the time of this writing (the early days of the second Trump administration), there appears to be a movement within the federal government to replace many human bureaucratic functions with AI.
Oddly, it doesn’t seem that government workers have as well-defined (or well-enforced) a sense of who or what they are loyal to as fiduciaries in the private sector. This makes aligning these AI systems even more challenging.
Whereas a doctor or lawyer or trustee can be expected to act in the best interest of their principal, a government worker might be loyal to the state broadly speaking, or to their party, or to their boss, or to their own career. These nuances have long been chalked up to “politics”. But the fissures in the U.S. federal government currently are largely about these divisions of loyalty, and the controversies are largely about conflicts of interest that would be forbidden in a private fiduciary context.
So, while we might wish that a democratically elected government have an affirmative obligation to loyalty and care towards, perhaps, the electorate, with subsidiary duties of confidentiality, disclosure, and so on, that is not legally the case. Instead, there’s a much more complex patchwork of duties and powers, and a set of checks and balances which is increasingly resembling the anarchic state of international relations.
The “oath of office” is perhaps an expression or commitment of loyalty that federal government workers could be held to. As far as I know, this is never directly legally enforced.
Between this inherent ambiguity of loyalty, and further complications brought on by the fact that government AI will in most cases be produced by third parties and procured, not hired, make the AI alignment especially fraught.
Complex systems theory is a way of thinking about systems with many interacting parts and functions. It draws on physics and the science of modeling dynamic systems. It’s a trans-disciplinary, quantitative science of everything. It often and increasingly gets applied to social systems, often through the methods of agent-based modeling (ABM). ABM has a long history in computational sociology. More recently, it has made inroads into economics and finance (Axtell and Farmer, 2022). That’s important intellectual territory to win over because, of course, it is vitally important for both private and public interests. Its progress there is gradual but steady. ABM and complex systems methods have no dogma besides mathematical and computational essentials. Their eventual triumph is more or less assured. As I’ve argued, ABM and complex systems theory are thus an exciting frontier for legal theory (Benthall and Strandburg, 2021). For these reasons, one line of my research is involved in developing computational frameworks (i.e. software libraries, mathematical scaffolding) for computational social scientific modeling.
Contextual Integrity (CI) is an ethical theory developed by Helen Nissenbaum. It is especially applicable to questions of the ethics of information technology and computation. Central to the theory is the idea of “appropriate information flow”, or flows of (personal) information which conform with “information norms”. According to CI, information norms are legitimized by a balance of societal values, contextual purposes, and individual ends. The work of the CI ethicist is to wrestle with the alignments and contradictions between these alignments, purposes, and ends to identify the most legitimate norms for a given context,. When the legitimate norms are identified, it is then in principle possible to design and deploy technology in accordance with these norms.
CI is a philosophy grounded in social theory. It has never been robustly quantified and many people think this is impossible to do. I’m not among these people. In fact, much of my work is about trying to quantify or model CI. It should come as no surprise, then, that I now see CI in terms of complexity theory. It has struck me recently that what this amounts to, more or less, is a computational social theory of ethics! This idea is exciting to me, and one day I’ll want to write it down in detail. For now, I have some nice diagrams and notes from a recent presentation I wanted to share.
CI is a theory of ethics that is ultimately concerned with the way that values, purposes, and ends legitimize socially understood practices. The ethicist’s job, for CI, is to design legitimate institutions. A problem for the ethicist is that some institutions can be legitimate, but utopian, in that they are not stable behavioral patterns for the sociotechnical system. Complex systems theory, as a descriptive science, is well adapted to modeling systems and identifying the regular behaviors within them, under varying conditions. Borrowing a notion from physics, a system can exhibit many regular behavioral states, which we might call phases. For example, it is well known that water has many different phases, depending on the temperature: ice, the liquid water, steam, etc.
Norms have both descriptive and (ahem) normative dimensions. (This confusing jargon is part of why it’s so hard to make progress in this area.) In other words, for there to be an actually existing norm, it has to be both regular and, to be ethical according to CI, legitimate.
There are critics of CI who argue that one problem with it is that it assumes an apolitical consensus of information norms without addressing how norms might be distorted by, e.g., power in society. This is not terribly fair to Nissenbaum’s broader corpus of work, which certainly acknowledges political complexity (see for example the recent Nissenbaum, 2024). Suffice it to say here that not all individual end up being ‘legitimized’ when ethicists assess things, and that legitimization is always political. Moreover, individual ends and politics can, of course, often be the driver of system behavior away from legitimate institutions. We can’t always have nice things.
Nevertheless, it remains useful to consider how and under what conditions a system could remain legitimate despite technological change. This is what the original CI design heuristic is: a procedure for evaluating what to do when a new technology creates a disruptive change in societal information flows.
Ideally, for CI, when a new technology destabilizes the sociotechnical system’s behavior and threatens it with illegitimate practices, society reacts (through journalism, through ethics, through a political process, through private choices and actions, etc.) and returns the system to a regular behavioral pattern that is legitimate. This might not be the same behavior as the system started with. It might be even better. And that’s OK.
What’s bad, for CI, is if the system gets stuck in an illegitimate but still robust phase.
While there are some applications of CI that serve anodyne ends of parsing and implementing uncontroversial privacy rules, there are other uses of CI as a radical critique of the status quo. This is well exemplified by Ido Sivan-Sevilla et al.’s comments on the FTC ANPR on Commercial Surveillance and Lax Data Security Practices (2022), which is a succinct and to-the-point condemnation of the “notice and consent” practices in commercial surveillance. We live in a world in which standard, even ubiquitous, technology norms depend on “laughable legal fictions” such as the idea that users of web services are legitimate parties to contracts with vendors. It is well documented how these fictions have been enshrined into law by decades of pressure by the technology sector in courts and government (Cohen, 2019).
Together, CI and complex systems theory can show how society can be a winner, or loser, beyond the sum of individual outcomes. There are certainly those that have argued that, essentially, “there is no such thing as society”, and that voluntary, binary transactions between parties are all there is. An anarchic, libertarian, or laissez-faire system certainly serves the individual ends of some, and is to some extent stable until the lords of anarchy create new systems of rules that are in their interest. It is difficult to analyze the social costs of these political changes in terms of “individual harms”, because the true marginal cost is not measurable at the level of the individual, but rather at the level of the phase transition. A complex systems theory allows for this broader view of what is at stake.
This approach also, I think, helps convey the fragility of legitimate institutions. Nothing guarantees legitimacy. Legitimate institutions typically constrain the behavior of some actors in ways that they individually do not enjoy. There are social processes which can steer a system towards a more legitimate phase, but these will meet with resistance, sometimes fail, and can be coopted by bad faith actors serving their own ends.
Indeed, there are those who would say we do not live in a legitimate system and have not lived in one for a long time. “Legitimate for whom?” Even if this is so, CI invites us to have a productive dialog about what legitimacy would entail, by sorting out different motivations and looking at the options for balancing them out. This good faith search for resolutions is often thankless and unrewarded, but certainly we would be worse off without it. On the other hand, arguments about legitimate institutions that are divorced from realistic understandings of sociotechnical processes are easily deployed as propaganda and ideology to cover illegitimate behavior. Ethics requires a science of sociotechnical systems; sociotechnical systems are complex; complex systems theory is a solid foundation for such a science.
References
Axtell, R. L., & Farmer, J. D. (2022). Agent-based modeling in economics and finance: Past, present, and future. Journal of Economic Literature, 1-101.
Benthall, S., & Strandburg, K. J. (2021). Agent-based modeling as a legal theory tool. Frontiers in Physics, 9, 666386.
Cohen, J. E. (2019). Between truth and power. Oxford University Press.
Nissenbaum, H. (2024). AI Safety: A Poisoned Chalice?. IEEE Security & Privacy, 22(2), 94-96.
Some years ago I entered a PhD program with the intention to become a Scientist. I had some funny ideas about what this meant that were more informed by reading philosophy than by scientific practice. By Science, I was thinking of something more like the German Wissenschaft than what are by and large more narrowly circumscribed institutions that are dominant in the United States today. I did, and still do, believe in, the pursuit of knowledge through rigorous inquiry.
Last week I attended the Principal Investigator (PI) meetings for the “Designing Accountable Software Systems” (DASS) program of the National Science Foundation. Attending those meetings, I at last felt like I made it. I am a scientist! Also in attendance were a host of colleagues whom I respect, with a shared interest in how to make “software systems” (a good name for the ubiquitous “digital” infrastructure that pervades everything now) more accountable to law and social norms. There were a bunch of computer scientists, but also many law professors, and some social scientists as well. What we were doing there was coming up with ideas for what the next project call under this program should be about. It was an open discussion about the problems in the world, and the role of science in fixing them. There’s a real possibility that these conversations will steer the future of research funding and in a small way nudge the future forward. Glorious.
In the past year, I’ve had a few professional developments that reinforce my feeling of being on the right track. A couple more of the grants I’ve applied to have landed. This has shifted my mindset about my work from one of scarcity (“Ack! What happens if I run out of funding!”) to one of abundance (“Ack! How am I going to hire somebody who I can pay with this funding!”). People often refer to academic careers as “precarious” up until one gets a tenure-track job, or even tenure. I’ve felt that precariousness in my own career. I still feel lower on the academic totem poll because I haven’t found a tenure track job as a professor, or a well-remunerated industrial research lab job. So I don’t take my current surplus for granted, and am well aware of all the talent swirling around that is thirsty for opportunities. But what people say is that it’s hard to get your first grant, and that it gets easier after that. There are full-time “soft money” researchers in my network who are career inspirations.
Another development is that I’ve been granted Principal Investigator status at New York University School of Law, which means I can officially manage my own grants there without a professor technically signing off or supervising the work. This is a tremendous gift of independence from Dr. Katherine Strandburg, my long-time mentor and supervisor at NYU’s Information Law Institute, where I’ve been affiliated for many years. It would be impossible to overstate Dr. Strandburg’s gentle and supportive influence on my postdoctoral work. I have been so fortunate to work with such a brilliant, nimble, open-minded, and sincerely good professor for the years I have been at NYU, both in direct collaboration and at the Information Law Institute, which, in her image, is an endless source of joyful intellectual stimulation.
Law schools are a funny place to do scientific research. They are naturally interdisciplinary in terms of scholarship — law professors work in many social scientific and technical disciplines, besides their own discipline of law. They are funded primarily by law student tuition, and so they are in many ways a professional school. But law is an inherently normative field — laws are norms — and so the question of how to design systems according to ethics and justice is part of the trade. Today, with the ubiquity of “software systems” — the Internet, “Big Data” a decade ago, “AI” today — the need for a rigorous science of sociotechnical systems is ever-present. Law schools are fine places to do that work.
However, law schools are often short on technical talent. Fortunately, I am also appointed at the International Computer Science Institute (ICSI), which is based in Berkeley, California, a non-profit lab that spun out of UC Berkeley’s (UCB) Computer Science department. I also have PI status at ICSI, and am running a couple of grants out of there at the moment.
Working as a remote PI for ICSI is a very funny “return” to Berkeley. I did my doctorate at UCB’s Information School but completed the work in New York, working with Helen Nissenbaum, who at the time was leaving NYU (and the Information Law Institute she co-founded with Kathy Strandburg) to start the Digital Life Initiative at Cornell Tech. I never expected to come “back” to Berkeley, in no small part because I discovered in California that I am not, in fact, a Californian. But the remote appointment, at a place free from the university politics and bureaucracy that drive people over there crazy, fits just right. There are some computer science all-stars that did groundbreaking work still at ICSI to look up to, and a lot of researchers in my own generation who work on very related problems of technical accountability. It is a good place to do scientific work oriented towards objective evaluation and design of sociotechnical systems.
All this means that I feel that somehow, despite pursuing an aggressively — some would say foolishly — interdisciplinary path, I have a certain amount of security in my position as a scientist. This has been the product of luck and hard work and stubbornness and to some extent an inability to do anything else. How many times have I cursed my circumstances and decisions, with setback after setback and failure after failure? I’ve lost count. The hard truth facing anybody pursuing Wissenschaft in the 21st century is that knowledge is socially constructed, and that the pursuit of objective knowledge must be performed from a very carefully constructed social position. But they do exist.
What now? Well, I find that now that I am actually positioned to make progress on the research projects that I see as most dear, I am behind on all of them. Projects that were launched years ago still haven’t manifested. I am painfully aware, every day, of the gap between what I have set out to accomplish and what I can tell the world I’ve done. Collaborating with other talented, serious people can be hard work; sometimes personality, my own or others’, makes it harder. I look on my past publications and they seem like naive sketches towards something better. I present my current work to colleagues and sometimes they find it hard to understand. It is isolating. But I do understand that this is to some extent exactly how it must be, if I’m making real progress on my own agenda. I am very hopeful that my best work is ahead of me.
For most of my formative years, writing was an important activity in my life. I would try to articulate what I cared about, the questions I had, what I was excited for. Often this was done as an intellectual pursuit, something para-academic, a search for answers beyond any curriculum, and without a clear goal in mind.
I wrote both in private and “in public”. Privately, in paper journals and letters to friends, then emails. “In public”, on social media of various forms. LiveJournal was a fantastic host for pseudonymous writing, and writing together and intimately with others was a shared pastime. These were the ‘old internet’ days, when the web was imagined as a frontier and creative space.
The character of writing changed as the Web, and its users, matured. Facebook expanded from its original base of students to include, well, everybody. Twitter had its “Weird Twitter” moment, which then passed as it became an increasingly transactional platform. Every public on-line space became a LinkedIn.
Instagram is, for a writer, depressing. Extremely popular and influential, but with hardly any written ideas, by design. YouTube can accommodate writing, but only with the added production value of performance, recording, and so on. It’s a thicker medium that makes writing, per se, seem small, or shallow.
The Web made it clear that all text is also data. Writing is a human act, full of meaning, but digital text, the visible trace of that act, is numerical. Text that is published online is at best the body of a message with so much other metadata. The provenance is, when the system is working, explicit. The message is both intrinsically numerical — encoded in bits of information — and extrinsically numerical: its impact, reach, engagement, is scored; it is indexed and served as a result to queries based on its triangulated position in the vast space of all possible text.
Every day we are inundated with content, and within the silicon cage of our historical moment, people labor to produce new content, to elevate their place in the system of ranks and numbers. I remember, long ago, I would write for an imagined audience. Sometimes this audience was only myself. This was writing. I think writing is rarer now. Many people create content and the audience for that content is the system of ranks and numbers. I miss writing.
Generative Artificial Intelligence is another phase of this evolution. Text is encoded, intrinsically, into bits. Content is sorted, extrinsically, into tables of impact, engagement, conversion, relevance, and so on. But there had been a mystery, still, about the way words were put together, and what they meant when they came in this or that order.
That mystery has been solved, they say. See, my looking at all the text at once, each word or phrase can be seen as a vector in a high dimensional space, relative to all other texts. The meaning of the text is where it is in that space, as understood by a silent, mechanical observer of all texts available. Relative to a large language model, the language in one person’s mind is small, shallow.
Generative AI now excels at creating content. The silicon cage may soon start evicting its prisoners. Their labor isn’t needed any more. The numbers can take care of themselves.
I remember how to write. I now realize that what is special about writing is not that it produces text — this is now easily done by machines. It is that it is a human act that transforms the human as writer.
I wonder for how long humanity will remember how to read.
A recent Dear Colleagues Letter from the National Science Foundation Directorate for Computer and Information Science and Engineering (CISE) calls for proposals for projects to envision research priorities. It is specifically not for research itself, but for promising ways to surface and communicate new R&D directions.
Essentially, the CISE directorate is asking for people to figure out a way to identify the future of computer and information science research. Just, you know, putting it out there.
The CISE Directorate is roughly 38 years old at the time of this writing, and computing and information science have, in that time, transformed pretty much everything.
At the same time, at this present moment, there’s a sense in which computer science feels… saturated. Maybe, indeed, lacking in future vision.
Why do I feel this is so? At least two reasons:
a) In the 90’s and 00’s, so much of the potential of computer science was being discovered and unleashed by startups. Even the companies that are today Big Tech were, then, startups. Now, notoriously, a lot of startups are just weird offshoots of Big Tech companies designed to be absorbed back in when legal or market conditions are favorable. So the technical research agenda is being set by huge companies with in-house research, rather than by a loose network of innovators.
b) “Artificial Intelligence” has for a long time meant “anything that computers can’t do yet”, with the Turing Test as one of the examples of what was still an unsolved problem in computer science. Deep learning has been blasting these unsolved problems out out of the water for almost a decade now. I’d argue that the newish LLM-powered chatbots appear so ominously to be a form of “general” AI is because they command natural language so convincingly — the key challenge of the Turing Test. So, computer science is running out of unsolved problems.
c) At the same time, this so widely hyped and lauded generation of AI, which has been credited with potentially literally apocalyptic powers, has gotten over the hump of the Gartner hype cycle, and it still can’t get hands right. On the other hand, it is supposed to be making software engineering obsolete as a profession, which would in principle cut down on the demand for computer science research.
d) It is now very clear that the success of computing and information science basic research depends on its uptake in commercial and industrial settings, and that these economics depend on business, legal, and social logic that is outside the scope of computer science research per se. Computer and information science research is not successful in virtue of, but rather in spite of, its agnosticism about social context. And, increasingly, that social context is being included within the scope of computer and information science.
So, what is to be done?
One answer, which I intend seriously, is imperialism. By this I mean the expansion of computer and information science research into areas beyond its core. Another answer is that it can occupy itself by adapting to critique. I actually think a combination of both the the best answer.
By imperialism, I mean searching for unsolved problems in other sciences, and trying to crack them with computational methods. This has been done already with Go and protein folding. But most problems in the social sciences remain unsolved problems, computationally. There are indeed parts of the social sciences that are opaque to themselves and without the guiding light of computational theory.
By adapting to critique, I mean responding to the now ample critical literature, mainly produced by humanistic scholars (some legal, some STS, etc.) which aims to show the shortcomings of computer science methodology. Indeed, a lot of “information science” today operates at this critical or political level. Humanistic critique tends to stop at the level of anthropological observation.
What is not yet solved is the internalization of these critiques into computational and information theory and methods, which entail advances in the foundations of computational social science.
There are at least three research arenas that I know of which are getting at parts of these problems.
a) The Agent Foundations research agendas (e.g. PIBBSS, Causal Incentives) that have spun out of the AI Safety research communities. This work has come to understand that some foundational advances in what an agent is, in terms of computation and information, is needed to address longtermist AI safety concerns, and perhaps also more pressing problems of AI compliance in the short term. This has quite a bit of funding from Effective Altruist philanthropists.
b) Various computational institutional theory projects that can be found in the vicinity of Metagov. A lot of this is motivated by the idea of the truly self-governing digital community, a long-held Internet dream, one which got an influx of funding and interest from the blockchain boom. That blockchain/crypto flavor has left it, to some, with a funny smell. But some more academic avenues such as the Institutional Grammar Research Initiative have a more based academic stance.
c) Research into the computational foundations of agent-based modeling, such as that led by Michael Wooldridge and Anisoara Calinescu at Oxford University. Part of the interdisciplinary social science mix at the Institute for New Economic Thought, this research vein finds useful computational methods research that pushes the limits of what social systems can be modeled with computers.
The problem with social scientific problems is that they are extremely hard. They can involve multiple agents in intractable situations. Today, we have almost no social systems that are not also sociotechnical systems where the technology is creating complications, so modeling these systems is recursive and perhaps necessarily approximate. To me, these problems remain philosophically tantalizing, when so many issues seem already to be reducible to fundamentals. Maybe this is the direction of the future of computer and information science research.
Through a few new emerging projects and opportunities, I’ve had reason to circle back to the topic of Artificial Intelligence and ethics. I wanted to jot down a few notes as some recent reading and conversations have been clarifying some ideas here.
In my work with Jake Goldenfein on this topic (published 2021), we framed the ethical problem of AI in terms of its challenge to liberalism, which we characterize in terms of individual rights (namely, property and privacy rights), a theory of why the free public market makes the guarantees of these rights sufficient for many social goods, and a more recent progressive or egalitarian tendency. We then discuss how AI technologies challenge liberalism and require us to think about post-liberal configurations of society and computation.
A natural reaction to this paper, especially given the political climate in the United States, is “aren’t the alternatives to liberalism even worse?” and it’s true that we do not in that paper outline an alternative to liberalism which a world with AI might aspire to.
John Mearsheimer’s The Great Delusion: Liberal Dreams and International Realities (2018) is a clearly written treatise on political theory. Mearsheimer rose to infamy in 2022 after the Russian invasion of Ukraine because of widely circulated videos of a lecture in 2015 in which he argued that the fault for Russia’s invasion of Crimea in 2014 was due to U.S. foreign policy. It is because of that infamy that I’ve decided to read The Great Delusion, which was a Financial Times Best Book of 2018. The Financial Times editorials have since turned on Mearsheimer. We’ll see what they say about him in another four years. However politically unpopular he may be, I found his points interesting and have decided to look at his more scholarly work. I have not been disappointed, and find that he clearly articulates political philosophy I will use these articulations. I won’t analyze his international relations theory here.
Putting Mearsheimer’s international relations theories entirely aside for now, I’ve been pleased to find The Great Delusion to be a thorough treatise on political theory, and it goes to lengths in Chapter 3 to describe liberalism as a political theory (which will be its target). Mearsheimer distinguished between four different political ideologies, citing many of their key intellectual proponents.
Modus vivendi liberalism. (Locke, Smith, Hayek) A theory committed to individual negative rights, such as private property and privacy, against the impositions by the state. The state should be minimal, a “night watchman”. This can involve skepticism about the ability of reason to achieve consensus about the nature of the good life; political toleration of differences is implied by the guarantee of negative rights.
Progressive liberalism. (Rawls) A theory committed to individual rights, including both negative rights and positive rights, which can be in tension. An example positive right is equal opportunity, which requires interventions by the state in order to guarantee. So the state must play a stronger role. Progressive liberalism involves more faith in reason to achieve consensus about the good life, as progressivism is a positive moral view imposed on others.
Utilitarianism. (Bentham, Mill) A theory committed to the greatest happiness for the greatest number. Not committed to individual rights, and therefore not a liberalism per se. Utilitarian analysis can argue for tradeoffs of rights to achieve greater happiness, and is collectivist, not in individualist, in the sense that it is concerned with utility in aggregate.
Liberal idealism. (Hobson, Dewey) A theory committed to the realization of an ideal society as an organic unity of functioning subsystem. Not committed to individual rights primarily, so not a liberalism, though individual rights can be justified on ideal grounds. Influenced by Hegelian views about the unity of the state. Sometimes connected to a positive view of nationalism.
This is a highly useful breakdown of ideas, which we can bring back to discussions of AI ethics.
Jake Goldenfein and I wrote about ‘liberalism’ in a way that, I’m glad to say, is consistent with Mearsheimer. We too identity right- and left- wing strands of liberalism. I believe our argument about AI’s challenge to liberal assumptions still holds water.
Utilitarianism is the foundation of one of the most prominent versions of AI ethics today: Effective Altruism. Much has been written about Effective Altruism and its relationship to AI Safety research. I have expressed some thoughts. Suffice it to say here that there is a utilitarian argument that ‘ethics’ should be about prioritizing the prevention of existential risk to humanity, because existential catastrophe would prevent the high-utility outcome of humanity-as-joyous-galaxy-colonizers. AI is seen, for various reasons, to be a potential source of catastrophic risk, and so AI ethics is about preventing these outcomes. Not everybody agrees with this view.
For now, it’s worth mentioning that there is a connection between liberalism and utilitarianism through theories of economics. While some liberals are committed to individual rights for their own sake, or because of negative views about the possibility of rational agreement about more positive political claims, others have argued that negative rights and lack of government intervention lead to better collective outcomes. Neoclassical economics has produced theories and ‘proofs’ to this effect, which rely on mathematical utility theory, which is a successor to philosophical utilitarianism in some respects.
It is also the case that a great deal of AI technology and technical practice is oriented around the vaguely utilitarian goals of ‘utility maximization’, though this is more about the mathematical operationalization of instrumental reason and less about a social commitment to utility as a political goal. AI practice and neoclassical economics are quite aligned in this way. If I were to put the point precisely, I’d say that the reality of AI, by exposing bounded rationality and its role in society, shows that arguments that negative rights are sufficient for utility-maximizing outcomes are naive, and so are a disappointment for liberals.
I was pleased that Mearsheimer brought up what he calls ‘liberal idealism’ in his book, despite it being perhaps a digression from his broader points. I have wondered how to place my own work, which draws heavily on Helen Nissenbaum’s theory of Contextual Integrity (CI), which is heavily influenced by the work of Michael Walzer. CI is based on a view of a society composed of separable spheres, which distinct functions and internally meaningful social goods, which should not be directly exchanged or compared. Walzer has been called a communitarian. I suggest that CI might be best seen as a variation of liberal idealism, in that it orients ethics towards a view of society as an idealized organic unity.
If the present reality of AI is so disappointing, then we must try to imagine a better ideal, and work our way towards it. I’ve found myself reading more and more work, such as by Felix Adler and Alain Badiou, that advocate for the need for an ideal model of society. What we currently are missing is a good computational model of such a society which could do for idealism what neoclassical economics did for liberalism. Which is, namely, to create a blueprint for a policy and science of its realization. If we were to apply AI to the problem of ethics, it would be good to use it this way.
Last week I spoke at PyData NYC 2023 about “Computational Open Source Economics: The State of the Art”.
It was a very nice conference, packed with practical guidance on using Python in machine learning workflows, interesting people, and some talks that were further afield. Mine was the most ‘academic’ talk that I saw there: it concerns recent developments in computational economics and what that means for open source economics tooling.
The talk discussed DYNARE, a widely known toolkit for representative agent modeling in a DSGE framework, and also more recently developed packages such as QuantEcon, Dolo, and HARK. It then outline how dynamic programming solutions to high-dimensional heterogeneous agent problems have ran into computational complexity constraints. Then, excitingly, how deep learning has been used to solve these models very efficiently, which greatly expands the scope of what can be modeled! This part of the talk drew heavily on Maliar, Maliar, and Winant (2021) and Chen, Didisheim, and Scheidegger (2023).
The talk concluded with some predictions about where computational economics is going. More standardized ways of formulating problems, coupled with reliable methods for encoding these problems into deep learning training routines, is a promising path forward for exploring a wide range of new models.
Digifesto 