Digifesto

Category: business

Life update: new AI job

I started working at a new job this month. It is at an Artificial Intelligence startup. I go to an office, use GitHub and Slack, and write software, manipulate data, and manage cloud computing instances for a living. As at this point I am relatively senior as an employee, I’m also involved in meetings of a managerial nature. There are lots of questions about how we organize ourselves and how we interact with other companies that I get to weigh in on.

This is a change from being primarily a postdoctoral researcher or graduate student. That change is apparent even though during my time as a latter I was doing similar industrial work on a part-time basis. Now, at the startup, the purpose of my work is more clearly oriented towards our company’s success.

There is something very natural about this environment for me. It is normal. I am struck by this normality because I have for years been interacting with academics who claim to be studying the very thing that I’m now doing.

I have written a fair bit here about “AI Ethics”. Much of this has been written with frustration at the way the topic is “studied”. In retrospect, a great deal of “AI Ethics” literature is about how people (the authors) don’t like the direction “the conversation” is going. My somewhat glib attitude towards it is that the problem is that most people talking about “AI Ethics” don’t know what they are talking about, and don’t feel like they have to know what they are talking about to have a good point of view on the subject. “AI Ethics” is often an expression of the point of view that while those that are “doing” AI are being somehow inscrutable and maybe dangerous, they should be tamed into accountability towards those who are not doing it, and therefore don’t really know about it. In other words, AI Ethics, as a field, is a way of articulating the interest of one class of people with one relationship to capital to another class of people with a different relationship to capital.

Perhaps I am getting ahead of myself. Artificial Intelligence is capital. I mean that in an economic sense. The very conceit that it is possible to join an “AI Startup”, whose purpose is to build an AI and thereby increase the productivity of its workers and its value to shareholders, makes the conclusion–“AI is capital”–a tautological one. Somehow, this insight rarely makes it into the “AI Ethics” literature.

I have not “left academia” entirely. I have some academic projects that I’m working on. One of these, in collaboration with Bruce Haynes, is a Bourdieusian take on Contextual Integrity. I’m glad to be able to do this kind of work.

However, one source of struggle for me in maintaining an academic voice in my new role, aside from the primary and daunting one of time management, is that many of the insights I would bring to bear on the discussion are drawn from experience. The irony of a training in qualitative and “ethnographic” research into use of technology, with all of its questions of how to provide an emic account based on the testimony of informants, is that I am now acutely aware of how my ability to communicate is limited, transforming me from a “subject” of observation into, in some sense, an “object”.

I enjoy and respect my new job and role. I appreciate that, being a real company trying to accomplish something and not a straw man used to drive a scholarly conversation, “AI” means in our context a wide array of techniques–NLP, convex optimization, simulation, to name a few–smartly deployed in order to best complement the human labor that’s driving things forward. We are not just slapping a linear regression on a problem and calling it “AI”.

I also appreciate, having done work on privacy for a few years, that we are not handling personal data. We are using AI technologies to solve problems that aren’t about individuals. A whole host of “AI Ethics” issues which have grown to, in some corners, change the very meaning of “AI” into something inherently nefarious, are irrelevant to the business I’m now a part of.

Those are the “Pros”. If there were any “Cons”, I wouldn’t be able to tell you about them. I am now contractually obliged not to. I expect this will cut down on my “critical” writing some, which to be honest I don’t miss. That this is part of my contract is, I believe, totally normal, though I’ve often worked in abnormal environments without this obligation.

Joining a startup has made me think hard about what it means to be part of a private organization, as opposed to a public one. Ironically, this public/private institutional divide rarely makes its way into academic conversations about personal privacy and the public sphere. That’s because, I’ll wager, academic conversations themselves are always in a sense public. The question motivating that discourse is “How do we, as a public, deal with privacy?”.

Working at a private organization, the institutional analogue of privacy is paramount. Our company’s DNA is its intellectual property. Our company’s face is its reputation. The spectrum of individual human interests and the complexity of their ordering has its analogs in the domain of larger sociotechnical organisms: corporations and the like.

Paradoxically, there is no way to capture these organizational dynamics through “thick description”. It is also difficult to capture them through scientific modes of visualization. Indeed, one economic reason to form an AI startup is to build computational tools for understanding the nature of private ordering among institutions. These tools allow for comprehension of a phenomenon that cannot be easily reduced to the modalities of sight or speech.

I’m very pleased to be working in this new way. It is in many ways a more honest line of work than academia has been for me. I am allowed now to use my full existence as a knowing subject: to treat technology as an instrument for understanding, to communicate not just in writing but through action. It is also quieter work.

Why managerialism: it acknowledges political role of internal corporate policies

One modern difficulty with political theory in contemporary times is the confusion between government and corporate policy. This is due in no small part to the extent to which large corporations now mediate social life. Telecommunications, the Internet, mobile phones, and social media all depend on layers and layers of operating organizations. The search engine, which didn’t exist thirty years ago, now is arguably an essential cultural and political facility (Pasquale, 2011), which sharpens the concerns that have been raised about their politics (Introna and Nissenbaum, 2000; Bracha and Pasquale, 2007).

Corporate policies influence customers when those policies drive product design or are put into contractual agreements. They can also govern employees and shape corporate culture. Sometimes these two kinds of policies are not easily demarcated. For example, Uber has an internal privacy policy about who can access which users’ information, like most companies with a lot of user data. The privacy features that Uber implicitly guarantees to their customers are part of their service. But their ability to provide this service is only as good as their company culture is reliable.

Classically, there are states, which may or may not be corrupt, and there are markets, which may or may not be competitive. With competitive markets, corporate policies are part of what make firms succeed or fail. One point of success is a company’s ability to attract and maintain customers. This should in principle drive companies to improve their policies.

An interesting point made recently by Robert Post is that in some cases, corporate policies can adopt positions that would be endorsed by some legal scholars even if the actual laws state otherwise. His particular example was a case enforcing the right to be forgotten in Spain against Google.

Since European law is statute driven, the judgments of its courts are not amenable to creative legal reasoning as they are in the United States. Post’s criticism of the EU’s judgment in this case is because of their rigid interpetation of data protection directives. Post argues a different legal perspective on privacy is better at balancing other social interests. But putting aside the particulars of the law, Post makes the point that Google’s internal policy matches his own legal and philosophical framework (which prefers dignitary privacy over data privacy) more than EU statutes do.

One could argue that we should not trust the market to make Google’s policies just. But we could also argue that Google’s market share, which is significant, depends so much on its reputation and users trust that in fact it is under great pressure to adjucate disputes with its users wisely. It is a company that must set its own policies, which do have political significance. It has the benefits of more direct control over the way these policies get interpreted and enforced in the state, faster feedback on whether the policies are successful, and a less chaotic legislative process for establishing policy in the first place.

Political liberals would dismiss this kind of corporate control as just one commercial service among many, or else wring their hands with concern over a company coming to have such power over the public sphere. But managerialists would see the emergence of search engines as an organization among others, comparable to other private entities that have been part of the public sphere, such as newspapers.

But a sound analysis of the politics of search engines need not depend on analogies with past technologies. This is a function of legal reasoning. Managerialism, which is perhaps more a descendent of business reasoning, would ask how, in fact, search engines make policy decisions and how does this affect political outcomes. It does not prima facie assume that a powerful or important corporate policy is wrong. It does ask what the best corporate policy is, given a particular sector.

References

Bracha, Oren, and Frank Pasquale. “Federal Search Commission-Access, Fairness, and Accountability in the Law of Search.” Cornell L. Rev. 93 (2007): 1149.

Introna, Lucas D., and Helen Nissenbaum. “Shaping the Web: Why the politics of search engines matters.” The information society 16.3 (2000): 169-185.

Pasquale, Frank A. “Dominant search engines: an essential cultural & political facility.” (2011).

industrial technology development and academic research

I now split my time between industrial technology (software) development and academic research.

There is a sense in which both activities are “scientific”. They both require the consistent use of reason and investigation to arrive at reliable forms of knowledge. My industrial and academic specializations are closely enough aligned that both aim to create some form of computational product. These activities are constantly informing one another.

What is the difference between these two activities?

One difference is that industrial work pays a lot better than academic work. This is probably the most salient difference in my experience.

Another difference is that academic work is more “basic” and less “applied”, allowing it to address more speculative questions.

You might think that the latter kind of work is more “fun”. But really, I find both kinds of work fun. Fun-factor is not an important difference for me.

What are other differences?

Here’s one: I find myself emotionally moved and engaged by my academic work in certain ways. I suppose that since my academic work straddles technology research and ethics research (I’m studying privacy-by-design), one thing I’m doing when I do this work is engaging and refining my moral intuitions. This is rewarding.

I do sometimes also feel that it is self-indulgent, because one thing that thinking about ethics isn’t is taking responsibility for real change in the world. And here I’ll express an opinion that is unpopular in academia, which is that being in industry is about taking responsibility for real change in the world. This change can benefit other people, and it’s good when people in industry get paid well because they are doing hard work that entails real risks. Part of the risk is the responsibility that comes with action in an uncertain world.

Another critically important difference between industrial technology development and academic research is that while the knowledge created by the former is designed foremost to be deployed and used, the knowledge created by the latter is designed to be taught. As I get older and more advanced as a researcher, I see that this difference is actually an essential one. Knowledge that is designed to be taught needs to be teachable to students, and students are generally coming from both a shallower and more narrow background than adult professionals. Knowledge that is designed to by deployed and used need only be truly shared by a small number of experienced practitioners. Most of the people affected by the knowledge will be affected by it indirectly, via artifacts. It can be opaque to them.

Industrial technology production changes the way the world works and makes the world more opaque. Academic research changes the way people work, and reveals things about the world that had been hidden or unknown.

When straddling both worlds, it becomes quite clear that while students are taught that academic scientists are at the frontier of knowledge, ahead of everybody else, they are actually far behind what’s being done in industry. The constraint that academic research must be taught actually drags its form of science far behind what’s being done regularly in industry.

This is humbling for academic science. But it doesn’t make it any less important. Rather, in makes it even more important, but not because of the heroic status of academic researchers being at the top of the pyramid of human knowledge. It’s because the health of the social system depends on its renewal through the education system. If most knowledge is held in secret and deployed but not passed on, we will find ourselves in a society that is increasingly mysterious and out of our control. Academic research is about advancing the knowledge that is available for education. It’s effects can take half a generation or longer to come to fruition. Against this long-term signal, the oscillations that happen within industrial knowledge, which are very real, do fade into the background. Though not before having real and often lasting effects.

Bay Area Rationalists

There is an interesting thing happening. Let me just try to lay down some facts.

There are a number of organizations in the Bay Area right now up to related things.

  • Machine Intelligence Research Institute (MIRI). Researches the implications of machine intelligence on the world, especially the possibility of super-human general intelligences. Recently changed their name from the Singularity Institute due to the meaninglessness of the term Singularity. I interviewed their Executive Director (CEO?), Luke Meuhlhauser, a while back. (I followed up on some of the reasoning there with him here).
  • Center for Applied Rationality (CFAR). Runs workshops training people in rationality, applying cognitive science to life choices. Trying to transition from appearing to pitch a “world-view” to teaching a “martial art” (I’ve sat in on a couple of their meetings). They aim to grow out a large network of people practicing these skills, because they think it will make the world a better place.
  • Leverage Research. A think-tank with an elaborate plan to save the world. Their research puts a lot of emphasis on how to design and market ideologies. I’ve been told that they recently moved to the Bay Area to be closer to CFAR.

Some things seem to connect these groups. First, socially, they all seem to know each other (I just went to a party where a lot of members of each group were represented.) Second, the organizations seem to get the majority of their funding from roughly the same people–Peter Thiel, Luke Nosek, and Jaan Tallinn, all successful tech entrepreneurs turned investors with interest in stuff like transhumanism, the Singularity, and advancing rationality in society. They seem to be employing a considerable number of people to perform research on topics normally ignored in academia and spread an ideology and/or set of epistemic practices. Third, there seems to be a general social affiliation with LessWrong.com; I gather a lot of the members of this community originally networked on that site.

There’s a lot that’s interesting about what’s going on here. A network of startups, research institutions, and training/networking organizations is forming around a cluster of ideas: the psychological and technical advancement of humanity, being smarter, making machines smarter, being rational or making machines to be rational for us. It is as far as I can tell largely off the radar of “mainstream” academic thinking. As a network, it seems concerned with growing to gather into itself effective and connected people. But it’s not drawing from many established bases of effective and connected people (the academic establishment, the government establishment, the finance establishment, “old boys networks” per se, etc.) but rather is growing its own base of enthusiasts.

I’ve had a lot of conversations with people in this community now. Some, but not all, would compare what they are doing to the starting of a religion. I think that’s pretty accurate based on what I’ve seen so far. Where I’m from, we’ve always talked about Singularitarianism as “eschatology for nerds”. But here we have all these ideas–the Singularity, “catastrophic risk”, the intellectual and ethical demands of “science”, the potential of immortality through transhumanist medicine, etc.–really motivating people to get together, form a community, advance certain practices and investigations, and proselytize.

I guess what I’m saying is: I don’t think it’s just a joke any more. There is actually a religion starting up around this. Granted, I’m in California now and as far as I can tell there are like sixty religions out here I’ve never heard of (I chalk it up to the lack of population density and suburban sprawl). But this one has some monetary and intellectual umph behind it.

Personally, I find this whole gestalt both attractive and concerning. As you might imagine, diversity is not this group’s strong suit. And its intellectual milieu reflects its isolation from the academic mainstream in that it lacks the kind of checks and balances afforded by multidisciplinary politics. Rather, it appears to have more or less declared the superiority of its methodological and ideological assumptions to its satisfaction and convinced itself that it’s ahead of the game. Maybe that’s true, but in my own experience, that’s not how it really works. (I used to share most of the tenets of this rationalist ideology, but have deliberately exposed myself to a lot of other perspectives since then [I think that taking the Bayesian perspective seriously necessitates taking the search for new information very seriously]. Turns out I used to be wrong about a lot of things.)

So if I were to make a prediction, it would go like this. One of these things is going to happen:

  • This group is going to grow to become a powerful but insulated elite with an expanded network and increasingly esoteric practices. An orthodox cabal seizes power where they are able, and isolates itself into certain functional roles within society with a very high standard of living.
  • In order to remain consistent with its own extraordinarily high epistemic standards, this network starts to assimilate other perspectives and points of view in an inclusive way. In the process, it discovers humility, starts to adapt proactively and in a decentralized way, losing its coherence but perhaps becomes a general influence on the preexisting societal institutions rather than a new one.
  • Hybrid models. Priesthood/lay practitioners. Or denominational schism.

There is a good story here, somewhere. If I were a journalist, I would get in on this and publish something about it, just because there is such a great opportunity for sensationalist exploitation.

Don’t use Venn diagrams like this

Today I saw this whitepaper by Esri about their use of open source software. It’s old, but still kept my attention.

There’s several reasons why this paper is interesting. One reason is that it reflects the trend of companies that once used FUD tactics around open source software to singing a soothing song of compatibilism. It makes an admirable effort to explain the differences between open source, proprietary software, and open standards to its enterprise client audience. That is the good news.

The bad news is that since this new compatibilism is just bending to market pressure after the rise of successful open source software complements, it lacks an understanding of why the open source development process has caused those market successes. Of course, proprietary companies have good reason to blur these lines, because otherwise they would need to acknowledge the existence of open source substitutes. In Esri’s case, that would mean products like the OpenGeo Suite.

I probably wouldn’t have written this post if it were not for this Venn diagram, which is presented with the caption A hybrid relationship:

I don’t think there is a way to interpret this diagram in a way that makes sense. It correctly identifies that Closed Source, Open Source, and Open Standards are different. But what do the overlapping regions represent? Presumabely they are meant to indicate that a system may both be open source and use open standards, or have open standards and be closed, or…be both open and closed?

It’s a subtle point but the semantics of set containment implied by the Venn diagram really don’t apply here. A system that’s a ‘hybrid’ between a closed and open software is not “both” closed and open the same way closed software that uses open standards is “both” closed and open. Rather, the hybrid system is just that, a hybrid, which means that its architecture is going to suffer tradeoffs as different components have different properties.

I don’t think that the author of this whitepaper was trying to deliberately obscure this idea. But I think that they didn’t know or care about it. That’s a problem, because it’s marketing material like this that clouds the picture about the value of open source. At a pointy-haired managerial level, one can answer the question “why aren’t you using more open source software” with a glib, “oh, we’re using a hybrid model, tailored to our needs.” But unless you actually understand what you’re talking about, your technical stack may still be full of buggy and unaccountable software, without you even knowing it.

The open source acqui-hire

There’s some interesting commentary around Twitter’s recent acquisition, Whisper Systems:

Twitter has begun to open source the software built by Whisper Systems, the enterprise mobile security startup it acquired just three weeks ago. …This move confirms the, well, whipsers that the Whisper Systems deal was mostly made for acqui-hire purposes.

Another acquisition like this that comes to mind is Etherpad, which Google bought (presumably to get the Etherpad team working on Wave) then open sourced. The logic of these acquisitions is that the talent is what matters, the IP is incidental or perhaps better served by an open community.

When I talk to actual or aspiring entrepreneurs, they often make the assumption that it would spoil their business to start building out their product open source. For one thing, they argue, there will be competitors who launch their own startups off of the open innovation. Then, they will miss their chance at a big exit because there will be no IP to tempt Facebook or whoever else to buy them out.

These open source acqui-hires defy these concerns. Demonstrating talent is part of what makes one acquirable. Logically, then, starting a competing company based on technology in which you don’t have talent makes you less competitive, from the perspective of a market exit. It’s hard to see what kind of competitive advantage the copycat company would have, really, since it doesn’t have the expertise in technology that comes from building it. If they do find some competitive advantage (perhaps they speak a foreign language and so are able to target a different market), then they are natural partners, not natural competitors.

One can take this argument further. Making open and available software is one of the best ways for a developer to make others aware of their talents and increase the demand (and value) for their own labor. So the talent in an open source company should be on average more valuable in case of an acqui-hire.

This doesn’t seem like a bad way out for a talented entrepreneur. Why, then, is this not a more well-known model for startups?

One reason is that the real winners in the startup scene are not the entrepreneurs. It’s the funders, and to the funders it is more worthwhile to invest in several different technologies with the small chance of selling one off big than to invest in the market value of their entrepreneurs. Because, after all, venture capitalists are in the same war for engineering talent as Google, Facebook, etc.. This should become less of an issue, however, as crowdfunding becomes more viable.