Suggested reading: Psychohistory is now real

Your portable (or desktop) personality test

Isaac Asimov.
Source: New York World-Telegram and the Sun Newspaper Photograph Collection, Library of Congress

In Isaac Asimov's Foundation novels, published shortly after psychological testing became widespread during World War II, a secretive group applies mathematical formulas—or as we would say today, algorithms—to psychological data in order to predict and shape the future course of human history. Asimov called this amalgam of prediction and manipulation "psychohistory."

The 2016 presidential election has shown that psychohistory is now real. Today using psychological profiling to manipulate people is called "psychometrics" or "psychographics," but it's essentially the system that Asimov foresaw 65 years ago.

In their article "The Data That Turned the World Upside Down," journalists Hannes Grassegger and Mikael Krogerus describe how two Cambridge graduate students, David Stillwell and Michael Kosinski, correlated results from online personality tests and Facebook profiles. (I've posted about Stillwell and his research group before.) They realized that, with a dataset of millions of subjects, they could link a particular set of "likes" with specific personal tendencies and attributes: gender, sexual orientation, ethnic heritage, political affiliation, intelligence, religion, the use or abuse of alcohol and drugs, whether your parents divorced before you were 21, and the Big Five personality traits: openness, conscientiousness, extroversion, agreeableness, and neuroticism (OCEAN). As Grassegger and Krogerus write,

". . .before long, [Kosinski] was able to evaluate a person better than the average work colleague, merely on the basis of ten Facebook 'likes.' Seventy 'likes' were enough to outdo what a person's friends knew, 150 what their parents knew, and 300 'likes' what their partner knew. More 'likes' could even surpass what a person thought they knew about themselves."

Presumably, "likes" are voluntarily shared. But often without our explicit knowledge or consent, our computers and smartphones are constantly transmitting data on our behavior and interests. Companies are collecting, aggregating, sharing, and reselling that data. "Our smartphone, Kosinski concluded, is a vast psychological questionnaire that we are constantly filling out, both consciously and unconsciously."

Ingrid Bergman, Michael Chekov and Gregory Peck in Alfred Hitchcock's Spellbound (1945)

Enter Cambridge Analytica, a "predictive analytics" company that combines data from many sources to create detailed profiles of specific individuals. Cambridge Analytica claims that they have "profiled the personality of every adult in the United States of America—220 million people." By name.

All sorts of companies—employers, insurance companies, and marketers, to name three—might be interested in this information. But there's another class of client for whom detailed personal profiles are highly desirable: political campaigns. Cambridge Analytica worked for Brexit and the presidential campaign of Donald Trump. Both achieved stunning upsets and defied conventional political wisdom. Cambridge Analytica helped them both to win against the odds.

The vote for (orange) and against (blue) the Brexit referendum, from 50% (light) to 80% (dark).
Source: New York Times.

Two key strategies were used by both campaigns:

• Micro-targeting messages to specific individuals to motivate them to vote, often against their own interests and beliefs. In one day, Grassegger and Krogerus report, the Trump campaign tested thousand of different ads for effectiveness with various types of voters.

• Micro-targeting messages to specific individuals to convince them not to vote. A series of "dark posts"—nonpublic posts that appear in the Facebook timelines of users with specific profiles (in the US election, Sanders supporters, African Americans, and young women, among others)—promoted heavily negative views of the opposing side. The object was not, as in traditional political advertising, to gain these voters' support, but rather to discourage them from going to the polls at all. The Trump campaign boasted of its voter suppression efforts to Bloomberg Businessweek.

The Trump campaign also automated the amplification of its messages on social media platforms. As Sue Halpern writes in the NYR Daily, at one point nearly 40% of Trump's Twitter followers were bots masquerading as humans and retweeting his messages.

How effective was this approach? As reported by Andrew Cockburn in Harper's, in the final days before the election, the models used by Clinton's campaign predicted that she would win Michigan by 5 points, or about a quarter of a million votes. Instead she lost by 11,000 votes.

Why facts don't change our minds

The spectacle of working-class people voting for a developer of luxury resorts, women voting for a self-confessed sexual predator, and Affordable Care Act beneficiaries voting for a man who has vowed to repeal the ACA may be dismaying. But do you think that you respond to well-reasoned positions cogently argued, and that you're immune from manipulation?

Think again. As Elizabeth Kolbert writes in The New Yorker in her survey of three recent books on cognitive psychology, all of us repeatedly exhibit an immunity to information that contradicts our beliefs.

• We are highly suggestible. Students who were told they were especially good or especially bad at a judgment exercise assessed themselves as, respectively, better or worse than average—even after they were told that their assignments to the original groups were random. The researchers who carried out this study were surprised that "even after the initial evidential basis for their beliefs has been totally refuted, people fail to make appropriate revisions in those beliefs."

• We are subject to confirmation bias, "the tendency people have to embrace information that supports their beliefs and reject information that contradicts them." Students presented with statistics on capital punishment that supported their already-held opinions, either pro or con, became more hardened in their positions even after they were told that the statistics were falsified.

• We fall victim to the "illusion of explanatory depth," which is our tendency to believe that we are far more knowledgeable than we actually are. Shortly after the Russian invasion and annexation of Crimea, the farther off US survey respondents were when asked to identify Ukraine on a map the more likely they were to favor military intervention.

You should take the evolutionary rationales that are presented for these tendencies in Kolbert's article with a grain of salt, since they're not testable. But the implications in a world of fake news, Twitter bots, and other tools of deliberate misinformation are incontrovertible. The Big Data that we ourselves supply makes it ever more possible to identify, exploit and manipulate our biases.

This year three European countries in which right-wing parties have made recent gains are holding elections: Holland, France, and Germany. Brace yourself for more surprises.

Update 7 March 2016: In a New York Times article, Nicholas Confessore and Danny Hakim write that "a dozen Republican consultants and former Trump campaign aides, along with current and former Cambridge employees, say the company’s ability to exploit personality profiles — 'our secret sauce,' Mr. Nix once called it — is exaggerated. Cambridge executives now concede that the company never used psychographics in the Trump campaign."

Hmmm. Cambridge Analytica is a company that was founded to exploit personality profiling. It is largely funded by Robert Mercer, "a Trump backer and investor in Breitbart." It had Steve Bannon on its board of directors until he was officially hired as the Trump campaign's manager (although, as The Daily Beast reports, "Breitbart's ties to Trump were long suspected before Bannon was brought aboard the campaign following the ouster of campaign chairman Paul Manafort in August 2016" and that Bannon wrote in an August 30, 2015 e-mail, "'I'm Trump's campaign manager.'") The company was paid a reported $15 million by the Trump campaign for its services.

And yet the campaign operatives and company executives quoted in the article now claim that Cambridge Analytica was not heavily involved in the Trump campaign, or that it was involved but did not employ its proprietary methodology, or that it was involved and used its methodology and but that it wasn't effective.  You'll have to pardon me, but this denial simply does not seem credible. And the denial is contradicted by an article by McKenzie Funk published right after the election in November, which appeared in The New York Times.