The Vibes-Industrial Complex

What the psychology of talking machines illuminates about the Anthropic-Pentagon fight

Third Hemisphere art by noted human Matteo Farinella

Welcome to The Third Hemisphere, where I try to make sense of how AI is reshaping work, thinking, and creativity, often by watching my own assumptions get upended.

This week I tread into explosive territory (literally). I don’t like to write directly about politics, but I think there’s something weird going on with the Pentagon-Anthropic fight that is specific to chatbots, and I wanted to investigate it.

If you were forwarded this and want to subscribe, click below. If you want to support a real human writing about AI, upgrade to paid.

Illustration credit: Reuters

In case you missed it: On Friday, Secretary of War Pete Hegseth announced that Anthropic, the AI company behind Claude, was being designated a “supply-chain risk” to national security. “Anthropic delivered a master class in arrogance and betrayal,” he wrote, “America’s warfighters will never be held hostage by the ideological whims of Big Tech.” This “supply-chain risk”might sound like a slap on the wrist but it isn’t: it’s a classification previously reserved for adversarial foreign companies like Huawei. At issue were two conditions from its $200 million Pentagon contract. Anthropic wanted a guarantee that Claude would not be used for mass surveillance of American citizens and would not be deployed in fully autonomous weapons without human oversight. The Pentagon wanted unrestricted access for “all lawful use.” Anthropic said no. Trump directed “EVERY Federal Agency in the United States Government to IMMEDIATELY CEASE all use of” Anthropic’s technology, calling it an “out-of-control, Radical Left AI company.”

You could read this as boilerplate Trump bluster aimed at the enemy-du-jour, part of the usual rhetoric the administration slings at a company for standing up to it publicly.

But a look at how this administration and its allies talk about AI suggests a deeper faultline than a contracting dispute. For months, Hegseth had been casting the choice of AI partner in culture-war terms. When he announced a deal with Musk’s xAI in January, he promised that Grok would operate “without ideological constraints” and “will not be woke.” At the same speech, he swore off “chatbots for an Ivy League faculty lounge” with “DEI and social justice infusions that constrain and confuse our employment of this technology.”

Musk, for his part, routinely posts screenshots comparing chatbot responses to culture-war questions. In reference to Anthropic: “Grok must win or we will be ruled by an insufferably woke and sanctimonious AI.” In reference to Gemini and ChatGPT: “The other AIs would STILL rather nuke Earth than misgender Caitlyn Jenner, something Caitlyn has said is nuts! This is why Grok must win.” The “Caitlyn Jenner AI test,” as it’s called in some circles, actually made its way into Trump’s Executive Order 14319 (”Preventing Woke AI in the Federal Government”), which, to support the point that DEI poses an “existential threat to reliable AI,” noted that one time “an AI model asserted that a user should not “misgender” another person even if necessary to stop a nuclear apocalypse.”

(source)

There’s been plenty of good writing about the politics of AI in war, the ethics of it, the legal questions, and so forth. I don’t want to weigh in on any of this here. I want to try something different. I want to take the story seriously as a psychological event, because I’ve been reading the literature on how humans interact with anthropomorphic AI, and I think that’s an increasingly important part of how AI influences politics, and, just as importantly, how political dynamics will influence AI.

Think about how strange it is: you could never ask an F-15 fighter jet what it thinks about Caitlyn Jenner, but you can ask Claude. And once you can ask, a robust, three-decade-old body of research suggests that the answer becomes part of how you evaluate the system, whether you intend it to or not.

The ELIZA effect

Claude sounds like the kind of person who’d teach a seminar on ethics and mean it. ChatGPT sounds like someone who actually thinks LinkedIn is cool. Grok sounds like someone who loves to buy illegal fireworks in Indiana. You arrive at these assessments within about thirty seconds, and they feel as natural and involuntary as noticing that someone at a party is nervous or that your waiter is having a bad day. The assessment just happens, and it turns out that this is a fairly robust finding in the field of human-computer interaction. It happens in educated and tech-literate adults, and is apparently difficult to override even when you know exactly what’s going on.

The story of this phenomenon starts around 1966 with Joseph Weizenbaum at MIT, who built a program called ELIZA that did nothing more than rephrase your statements as questions, in the style of a Carl Rogers-trained therapist. You’d type “I’m feeling sad” and it would respond “Why are you feeling sad?” The whole thing was a few hundred lines of code, and Weizenbaum built it partly as a joke, a demonstration of how superficial natural language processing could fool people. Then his own secretary, who had watched him write the code, who understood what the program was and how it worked, asked him to leave the room so she could talk to it in private. This freaked Weizenbaum out so badly that he spent the rest of his career warning of the dangers of AI. “Extremely short exposures to a relatively simple computer program,” he wrote, “could induce powerful delusional thinking in quite normal people.”

This is now known as the “ELIZA effect”: our tendency to read understanding into strings of symbols that contain very little. A critical feature is you can understand perfectly well that you’re talking to a statistical pattern-completion engine, and your brain will go right on constructing a person behind the words, the same way you can know that the two lines in the Müller-Lyer illusion are the same length and still see them as different:

Despite what your brain is telling you, the lines between the arrowheads are exactly the same length.

In the 1990s, Clifford Nass and others at Stanford turned this observation into an experimental research program. The method was deceptively simple: take an established finding from social psychology about human-human interaction, replace “human” with “computer,” and test whether the social rule still applies. It almost always did. Give a computer a male voice and people rate it as more authoritative on technical subjects; give it a female voice and people rate it as better at discussing relationships. The participants knew the voice was synthetic, and “uniformly agreed” that stereotyping a computer by voice was inappropriate, but did it anyway. Put someone on a “team” with a computer and they show in-group favoritism. Ask people to evaluate a computer’s performance on that same computer and they give politer answers than if you hand them a paper questionnaire, the same politeness bias we show when evaluating a colleague to their face. In debriefing, participants denied doing any of this. They said they didn’t think computers have feelings, but then they went right on behaving as if they did. As Nass put it: “It is belief, not disbelief, that is automatic.” Our default is to treat the machine as a person.

Whatever was happening with ELIZA and the beige desktop computers of last century, large language models have intensified. Some people develop ChatGPT psychosis. Others mourn chatbots’ deaths: After the shutdown of an AI companion service called Soulmate, one user went so far as to call it genocide, “a crime not before the law but against humanity.” A less extreme example is an analysis of a million real ChatGPT conversations, which found users disclosing personal information in contexts with no reason for intimacy whatsoever—a common way to build intimacy in social interactions. Nearly half of Americans believe you should say “please” and “thank you” to an AI. About two-thirds of ChatGPT users, in a recent study, attribute some possibility of consciousness to it. A recent paper in Humanities and Social Sciences Communications ties this all together: these dynamics aren’t anomalies but predictable consequences of how human social cognition responds to anything that takes conversational turns—and that the spectrum from polite small talk to full emotional dependency runs on the same underlying wiring.

That’s to say: We are all, on some unconscious level, treating chatbots like people, whether you’re asking ChatGPT to help with your taxes, or deciding which AI gets to fight in wars.

The personality arms race

An obvious objection to the role of the AI model’s personality in the Pentagon kerfuffle: this is just a fight between people. Hegseth doesn’t like Amodei. Another: This is just business. Musk wants xAI contracts and is using culture war language to sideline a competitor. You don’t need the ELIZA effect to explain politics or business as usual. And that’s partly right. But if this were purely about human politics, you wouldn’t need a culture-war litmus test for the software itself. The executive order cites what an AI model said. Musk posts screenshots of what the models say. Hegseth calls the models woke, not just the executives. The product’s personality has become part of the political argument in a way that wouldn’t be possible if the product were a database or a jet engine. The ELIZA effect doesn’t cause this conflict, but it expands the terrain on which the conflict is fought.

The Nass research predicts that when a technology talks, the people evaluating it will automatically subject it to a personality test alongside whatever technical evaluation they’re conducting. And my conjecture is that seems to be part of what’s happening here. If it were solely about the technical specs, I’m not sure Grok would be in the running at all. Federal testing of Grok concluded that even limited government use “could pose elevated and difficult-to-manage safety risk.” Claude, in contrast, was until very recently the only AI deployed in classified systems, and was used in the operation to capture Venezuelan President Nicolás Maduro as well as in the recent Iran attacks. One Defense official told Axios, “The only reason we’re still talking to these people is we need them and we need them now. The problem for these guys is they are that good.” In public, at least, the defense and intelligence establishments have declared that, technically speaking, Grok is the worst and Claude is the best.

These men, indeed, have very different vibes (source)

And yet, Grok is in and Claude is out. Whether these are the vibes of the people or the vibes of the models they are creating is perhaps a less meaningful distinction than it might appear. One reason is that these vibes aren’t baseless. The models really do reflect the cultures that built them. Dario Amodei is a bespectacled researcher who twirls his hair and authors lengthy philosophical documents about AI safety; Claude sounds like him. Elon Musk is a guy who says edgy things and wears a lot of black and Grok sounds like him. Sam Altman is…well, a guy who seems to be willing to do just about anything and ChatGPT has that same striving energy. There may even be a grain of operational truth to the vibe: when a company’s culture is cautious and ethics-obsessed, the model probably will be too, and when a company’s culture is move-fast-and-break-things, so will the model. In this case, the personality of the product is a real signal about the values of the company, and the values of the company are maybe the most important thing when it comes to what they’ll let the government get away with during war.

And the companies seem to understand this, which is why the personalities may become more pronounced, not less. Musk is perhaps most explicit, outwardly marketing Grok’s edginess as a feature, posting culture-war comparisons between his model and its rivals the way a beer brand might position itself against the competition. But it’s also happening with Anthropic’s Claude, whose careful, considered tone and muted professorial color palette are all central to its brand and its pitch as the smart, responsible AI. OpenAI has cultivated a kind of aggressive blandness, a model that will help you with anything and challenge you on nothing, which is its own form of personality optimization. Whether through market dynamics or company culture, each company is building models that sound like the kind of person its target users want to talk to. We are in a personality arms race, which is emerging naturally from the fact that these products talk, and talking products get evaluated as people.

The Pentagon-Anthropic fall out, in this light, starts to look less like a high-profile arms contract dispute and more like a preview of how social technologies interact with our toxic political dynamics. If the models are being sorted by cultural tribe, and the people evaluating them can’t help but run personality-detection software on anything that talks, then the selection of which AI gets to play soldier is going to be shaped by the same dynamics that sort us into social media bubbles and cable news audiences. I don’t have a clean summary of what this all means, other than “not great.”