Digifesto

January 10, 2017

Loving Tetlock’s Superforecasting: The Art and Science of Prediction

I was a big fan of Philip Tetlock’s Expert Political Judgment (EPJ). I read it thoroughly; in fact a book review of it was my first academic publication. It was very influential on me.

EPJ is a book that is troubling to many political experts because it basically says that most so-called political expertise is bogus and that what isn’t bogus is fairly limited. It makes this argument with far more meticulous data collection and argumentation than I am able to do justice to here. I found it completely persuasive and inspiring. It wasn’t until I got to Berkeley that I met people who had vivid negative emotional reactions to this work. They seem to mainly have been political experts who do not having their expertise assessed in terms of its predictive power.

Superforecasting: The Art and Science of Prediction (2016) is a much more accessible book that summarizes the main points from EPJ and then discusses the results of Tetlock’s Good Judgment Project, which was his answer to an IARPA challenge in forecasting political events.

Much of the book is an interesting history of the United States Intelligence Community (IC) and the way its attitudes towards political forecasting have evolved. In particular, the shock of the failure of the predictions around Weapons of Mass Destruction that lead to the Iraq War were a direct cause of IARPA’s interest in forecasting and their funding of the Good Judgment Project despite the possibility that the project’s results would be politically challenging. IARPA comes out looking like a very interesting and intellectually honest organization solving real problems for the people of the United States.

Reading this has been timely for me because: (a) I’m now doing what could be broadly construed as “cybersecurity” work, professionally, (b) my funding is coming from U.S. military and intelligence organizations, and (c) the relationship between U.S. intelligence organizations and cybersecurity has been in the news a lot lately in a very politicized way because of the DNC hacking aftermath.

Since so much of Tetlock’s work is really just about applying mathematical statistics to the psychological and sociological problem of developing teams of forecasters, I see the root of it as the same mathematical theory one would use for any scientific inference. Cybersecurity research, to the extent that it uses sound scientific principles (which it must, since it’s all about the interaction between society, scientifically designed technology, and risk), is grounded in these same principles. And at its best the U.S. intelligence community lives up to this logic in its public service.

The needs of the intelligence community with respect to cybersecurity can be summed up in one word: rationality. Tetlock’s work is a wonderful empirical study in rationality that’s a must-read for anybody interested in cybersecurity policy today.

Leave a comment

December 23, 2016

notes about natural gas and energy policy

I’m interested in energy (in the sense of the economy and ecology of energy as it powers society) but know nothing about it.

I feel like the last time I really paid attention to energy, it was still a question of oil (and its industrial analog, Big Oil) and alternative, renewable energy.

But now energy production in the U.S. has given way from oil to natural gas. I asked a friend about why, and I’ve filled in a big gap in my understanding of What’s Going On. What I filled it in with might be wrong, but here’s what it is so far:

At some point natural gas became a viable alternative to oil because the energy companies discovered it was cheaper to collect natural gas than to drill for oil.
The use of natural gas for energy has less of a carbon footprint than oil does. That makes it environmentally friendly relative to the current regulatory environment.
The problem (there must be a problem) is that the natural gas collection process has lots of downsides. These downsides are mainly because the process is very messy, involving smashing into some pocket of natural gas under lots of rock and trying to collect the good stuff. Lots of weird gases go everywhere. That has downsides, including:
- Making the areas where this is happening unlivable. Because it’s harder to breathe? Because the water can be set on fire? It’s terrible.
- It releases a lot of methane into the environment, which may be as bad if not worse for climate change than carbon. Who knows how bad it really is? Unclear.
Here’s the point (totally unconfirmed): The shift from oil to natural gas as an energy source has been partly due to a public awareness and regulatory gap about the side effects. There’s now lots of political pressure and science around carbon. But methane? I thought that was an energy source (because of Mad Max Beyond Thunderdome). I guess I was wrong.
Meanwhile, OPEC and non-OPEC have teamed up to restrict oil sales to hike up oil prices. Sucks for energy consumers, but that’s actually good for the environment.
Also, in response to the apparent reversal of U.S. federal interest in renewable energy, philanthropy-plus-market has stepped in with Breakthrough Energy Ventures. Since venture capital investors with technical backgrounds, unlike the U.S. government, tend to be long on science, this is just great.
So what: The critical focus for those interested in the environment now should be on the environmental and social impact of natural gas production, as oil has been taken care of and heavy hitters are backing sustainable energy in a way that will fix the problem if it can truly be fixed. We just have to not boil the oceans and poison all the children before they can get to it.

If that doesn’t work, I guess at the end of the day, there’s always pigs.

Leave a comment

December 14, 2016

Protected: What’s going on?

Enter your password to view comments.

December 11, 2016

energy, not technology

I’m still trying to understand what’s happening in the world and specifically in the U.S. with the 2016 election. I was so wrong about it that I think I need to take seriously the prospect that I’ve been way off in my thinking about what’s important.

In my last post, I argued that the media isn’t as politically relevant we’ve been told. If underlying demographic and economic variables were essentially as predictive as anything of voter behavior, then media mishandling of polling data or biased coverage just isn’t what’s accounting for the recent political shift.

Part of the problem with media determinist accounts of the election is that because they deal with the minutia of reporting within the United States, they don’t explain how Brexit foreshadowed Trump’s election, as anybody paying attention has been pointing out for months.

So what happens if we take seriously explanation that really what’s happening is a reaction against globalization. That’s globalization in the form of a centralized EU government, or globalization in the form of U.S. foreign policy and multiculturalism. If the United States under Obama was trying to make itself out to be a welcoming place for global intellectual talent to come and contribute to the economy through Silicon Valley jobs, then arguably the election was the backfire.

An insulated focus on “the tech industry” and its political relevance has been a theme in my media bubble for the past couple of years. Arguably, that’s just because people thought the tech industry was where the power and the money was. So of course the media should scrutinize that, because everyone trying to get to the top of that pile wants to know what’s going on there.

Now it’s not clear who is in power any more (I’ll admit I’m just thinking about power as a sloppy aggregate of political and economic power. Let’s assume that political power backing an industry leads to a favorable regulatory environment for that industry’s growth, and it’s not a bad model). It doesn’t seem like it’s Silicon Valley any more. Probably it’s the energy industry.

There’s a lot going on in the energy industry! I know basically diddly about it but I’ve started doing some research.

One interesting thing that’s happening is that Russia and OPEC are teaming up to cut oil production. This is unprecedented. It also, to me, creates a confusing narrative. I thought Obama’s Clean Power Plan, focusing on renewable energy, and efforts to build international consensus around climate change were the best bets for saving the world from high emissions. But since cutting oil production leads to cutting oil production, what if the thing that really can cut carbon dioxide emissions is an oligopolistic price hike on oil?

That said, oil prices may not necessarily dictate energy prices because the U.S. because a lot of energy used is natural gas. Shale gas, in particular, is apparently a growing percentage of natural gas used in the U.S. It’s apparently better than oil in terms of CO2 emissions. Though it’s mined through fracking, which disgusts a lot of people!

Related: I was pretty pissed when I heard about Rex Tillerson, CEO of Exxon Mobil, being tapped for Secretary of State. Because that’s the same old oil companies that have messed things up so much before, right? Maybe not. Apparently Exxon Mobil also invests heavily in natural gas. As their website will tell you, that gas industry uses a lot of human labor. Which is obviously a plus in this political climate.

What’s interesting to me about all this is that it all seems very important but it has absolutely nothing to do with social media or even on-line marketplaces. It’s all about stuff way, way upstream on the supply chain.

It is certainly humbling to feel like your area of expertise doesn’t really matter. But I’m not sure what to even do as a citizen now that I realize how little I understand. I think there’s been something very broken about my theory about society and the world.

The next few posts may continue to have this tone of “huh”. I expect I’ll be stating what’s obvious to a lot of people. But whatever. I just need to sort some things out.

3 Comments

December 7, 2016

post-election updates

Like a lot of people, I was completely surprised by the results of the 2016 election.

Rationally, one has to take these surprises as an opportunity to update ones point of view. As it’s been almost a month, there’s been lots of opportunity to process what’s going on.

For my own sake, more than for any reader, I’d like to note my updates here.

The first point has been best articulated by Jon Stewart:

Stewart rejected the idea that better news coverage would have changed the outcome of the election. “The idea that if [the media] had done a better job this country would have made another choice is fake,” he said. He cited Brexit as an example of an unfortunate outcome that occurred despite its lead-up being appropriately covered by outlets like the BBC, which offered a much more balanced view than CNN, for example. “Trump didn’t happen because CNN sucks—CNN just sucks,” he said.

Satire and comedy also couldn’t have stood in the way of Trump winning, Stewart said. If this election has taught us anything, he said, its that “controlling the culture does not equate to holding the power.”

I once cared a lot about “money in politics” at the level of campaign donations. After a little critical thinking, this leads naturally to a concern about the role of the media more generally in elections. Centralized media in particular will never put themselves behind a serious bid for campaign finance reform because those media institutions cash out every election. This is what it means for a problem to be “systemic”: it is caused by a tightly reinforcing feedback loop that makes it into a kind of social structural knot.

But with the 2016 presidential election, we’ve learned that Because of the Internet, media are so fragmented that even controlled media are not in control. People will read what they want to read, one way or another. Whatever narrative suits a person best, they will be able to find it on the Internet.

A perhaps unhelpful way to say this is that the Internet has set the Bourdieusian habitus free from media control.

But if the media doesn’t determine habitus, what does?

While there is a lot of consternation about the failure of polling (which is interesting), and while that could have negatively impacted Democratic campaign strategy (didn’t it?), the more insightful sounding commentary has recognized that the demographic fundamentals were in favor of Trump all along because of what he stood for economically and socially. Michael Moore predicted the election result; logically, because he was right, we should update towards his perspective; he makes essentially this point about Midwestern voters, angry men, depressed progressives, and the appeal of oddball voting all working against Hilary. But none of these conditions have as much to do with media as they do to the preexisting population conditions.

There’s a tremendous bias among those who “study the Internet” to assign tremendous political importance to the things we have expertise on: the media, algorithms, etc. My biggest update this election was that I now think that these are eclipsed in political relevance compared to macro-economic issues like globalization. At best changes to, say, the design of social media platforms are going to change things for a few people at the margins. But larger structural forces are both more effective and more consequential in politics. I bet that a prediction of the 2016 election based primarily on the demographic distribution of winners and losers according to each candidate’s energy policy, for example, would have been more valuable than all the rest of the polling and punditry combined. I suppose I was leaning this way throughout 2016, but the election sealed the deal for me.

This is a relief for me because it has revealed to me just how much of my internalization and anxieties about politics have been irrelevant. There is something very freeing in discovering that many things that you once thought were the most important issues in the world really just aren’t. If all those anxieties were proven to just be in my head, then it’s easier to let them go. Now I can start wondering about what really matters.

Leave a comment

November 26, 2016

reflexive control

A theory I wish I had more time to study in depth these days is the Soviet field of reflexive control (see for example this paper by Timothy Thomas on the subject).

Reflexive control is defined as a means of conveying to a partner or an opponent specially prepared information to incline him to voluntarily make the predetermined decision desired by the initiator of the action. Even though the theory was developed long ago in Russia, it is still undergoing further refinement. Recent proof of this is the development in February 2001, of a new Russian journal known as Reflexive Processes and Control. The journal is not simply the product of a group of scientists but, as the editorial council suggests, the product of some of Russia’s leading national security institutes, and boasts a few foreign members as well.

While the paper describes the theory in broad strokes, I’m interested in how one would formalize and operationalize reflexive control. My intuitions thus far are like this: traditional control theory assumes that the controlled system is inanimate or at least not autonomous. The controlled system is steered, often dynamically, to some optimal state. But in reflexive control, the assumption is that the controlled system is autonomous and has a decision-making process or intelligence. Therefore reflexive control is a theory of influence, perhaps deception. Going beyond mere propaganda, it seems like reflexive control can be highly reactive, taking into account the reaction time of other agents in the field.

There are many examples, from a Russian perspective, of the use of reflexive control theory during conflicts. One of the most recent and memorable was the bombing of the market square in Sarejevo in 1995. Within minutes of the bombing, CNN and other news outlets were reporting that a Serbian mortar attack had killed many innocent people in the square. Later, crater analysis of the shells that impacted in the square, along with other supporting evidence, indicated that the incident did not happen as originally reported. This evidence also threw into doubt the identities of the perpetrators of the attack. One individual close to the investigation, Russian Colonel Andrei Demurenko, Chief of Staff of Sector Sarejevo at the time, stated, “I am not saying the Serbs didn’t commit this atrocity. I am saying that it didn’t happen the way it was originally reported.” A US and Canadian officer soon backed this position. Demurenko believed that the incident was an excellent example of reflexive control, in that the incident was made to look like it had happened in a certain way to confuse decision-makers.

Thomas’s article points out that the notable expert in reflexive control in the United States is V. A. Lefebvre, a Soviet ex-pat and mathematical psychologist at UC Irvine. He is listed on a faculty listing but doesn’t seem to have a personal home page. His wikipedia page says that reflexive theory is like the Soviet alternative to game theory. That makes sense. Reflexive theory has been used by Lefebvre to articulate a mathematical ethics, which is surely relevant to questions of machine ethics today.

Beyond its fascinating relevance to many open research questions in my field, it is interesting to see in Thomas’s article how “reflexive control” seems to capture so much of what is considered “cybersecurity” today.

One of the most complex ways to influence a state’s information resources is by use of reflexive control measures against the state’s decision-making processes. This aim is best accomplished by formulating certain information or disinformation designed to affect a specific information resource best. In this context an information resource is defined as:

information and transmitters of information, to include the method or technology of obtaining, conveying, gathering, accumulating, processing, storing, and exploiting that information;
infrastructure, including information centers, means for automating information processes, switchboard communications, and data
transfer networks;
programming and mathematical means for managing information;
administrative and organizational bodies that manage information processes, scientific personnel, creators of data bases and knowledge, as well as personnel who service the means of informatizatsiya [informatization].

Unlike many people, I don’t think “cybersecurity” is very hard to define at all. The prefix “cyber-” clearly refers to the information-based control structures of a system, and “security” is just the assurance of something against threats. So we might consider “reflexive control” to be essentially equivalent to “cybersecurity”, except with an emphasis on the offensive rather than defensive aspects of cybernetic control.

I have yet to find something describing the mathematical specifics of the theory. I’d love to find something and see how it compares to other research in similar fields. It would be fascinating to see where Soviet and Anglophone research on these topics is convergent, and where it diverges.

4 Comments

November 21, 2016

For “Comments on Haraway”, see my “Philosophy of Computational Social Science”

One of my most frequently visited blog posts is titled “Comments on Haraway: Situated knowledge, bias, and code”. I have decided to password protect it.

If you are looking for a reference with the most important ideas from that blog post, I refer you to my paper, “Philosophy of Computational Social Science”. In particular, its section on “situated epistemology” discusses how I think computational social scientists should think about feminist epistemology.

I have decided to hide the original post for a number of reasons.

I wrote it pointedly. I think all the points have now been made better elsewhere, either by me or by the greater political zeitgeist.
Because it was written pointedly (even a little trollishly), I am worried that it may be easy to misread my intention in writing it. I’m trying to clean up my act :)
I don’t know who keeps reading it, though it seems to consistently get around thirty or more hits a week. Who are these people? They won’t tell me! I think it matters who is reading it.

I’m willing to share the password with anybody who contacts me about it.

Leave a comment

November 18, 2016

Protected: I study privacy now

Enter your password to view comments.

October 7, 2016

moved BigBang core repository to DATACTIVE organization

I made a small change this evening which I feel really, really good about.

I transferred the BigBang project from my personal GitHub account to the datactive organization.

I’m very grateful for DATACTIVE‘s interest in BigBang and am excited to turn over the project infrastructure to their stewardship.

Leave a comment

August 22, 2016

becoming a #seriousacademic

I’ve decided to make a small change to my on-line identity.

For some time now, my Twitter account has been listed under a pseudonym, “Gnaeus Rafinesque”, and has had a picture of a cat. Today I’m changing it to my full name (“Sebastian Benthall”) and a picture of my face.

I chose to use a pseudonym on Twitter for a number of reasons. One reason was that I was interested in participant observation in an Internet subculture, Weird Twitter, that generally didn’t use real names because most of their activity on Twitter was very silly.

But another reason was because I was afraid of being taken seriously myself. As a student, even a graduate student, I felt it was my job to experiment, fail, be silly, and test the limits of the media I was working (and playing) within. I learned a lot from this process.

Because I often would not intend to be taken seriously on Twitter, I was reluctant to have my tweets associated with my real name. I deliberately did not try to sever all ties between my Twitter account and my “real” identity, which is reflected elsewhere on the Internet (LinkedIn, GitHub, etc.) because…well, it would have been a lot of futile work. But I think using a pseudonym and a cat picture succeeded in signalling that I wasn’t putting the full weight of my identity, with the accountability entailed by that, into my tweets.

I’m now entering a different phase of my career. Probably the most significant marker of that phase change is that I am now working as a cybersecurity professional in addition to being a graduate student. I’m back in the working world and so in a sense back to reality.

Another marker is that I’ve realized that I’ve got serious things worth saying and paying attention to, and that projecting an inconsequential, silly attitude on Twitter was undermining my ability to say those things.

It’s a little scary shifting to my real name and face on Twitter. I’m likely to censor myself much more now. Perhaps that’s as it should be.

I wonder what other platforms are out there in which I could be more ridiculous.

Leave a comment