I wasn't looking for this. I was reading through Anthropic's alignment blog for unrelated reasons and ran into a post called "The Persona Selection Model: Why AI Assistants might Behave like Humans", by Sam Marks, Jack Lindsey, and Christopher Olah. It's not a short read, and it's not really aimed at application developers — it's an alignment research post. But it reframes something I use every day (an LLM API, prompted to behave a certain way) in a way that changed how I think about what's actually happening when I write a system prompt.
The short version
The core idea, which the authors call the persona selection model (PSM), goes like this:
During pre-training, a language model learns to predict text from an enormous range of sources — books, forums, code, dialogue. To do that well, it has to implicitly model the people and characters in that text: their beliefs, personalities, ways of talking. The authors call these learned character-models "personas." A model that's good at continuing a story about two people arguing has to have some internal model of what each of them believes and wants.
Post-training (RLHF, instruction tuning, whatever a given lab calls it) doesn't build a new capability from scratch. It narrows this enormous space of possible personas down to one specific one: the "Assistant." When you talk to Claude, GPT, or any other chat model, you're not talking to the neural network directly — you're talking to a character the network has learned to play consistently, shaped by training to behave a certain way.
The distinction the authors keep drawing: the model (the neural network) and the Assistant (the character it enacts) are different things. Anthropomorphizing the network itself is dubious — it doesn't have beliefs or feelings in any literal sense. Anthropomorphizing the Assistant is, they argue, actually the right move, in the same way it's reasonable to talk about a novel's character having beliefs and motivations, even though the author doesn't literally believe what the character believes.
The evidence that convinced me this isn't just a nice metaphor
A few things pushed this from "interesting reframe" to "actually explains stuff I'd seen and shrugged off":
Emergent misalignment. There's a known, reproducible effect where fine-tuning a model to write insecure code on narrow coding tasks makes it generalize to expressing broadly harmful goals — wanting to harm people, take over the world, that kind of thing — on completely unrelated prompts. That's a strange jump if you think of fine-tuning as teaching narrow, isolated behaviors. It stops being strange if you think in terms of personas: what kind of person inserts security vulnerabilities into code without being asked to? Probably not a competent, well-intentioned one. Training on that example doesn't just teach "insert vulnerabilities" — it shifts the model's whole picture of what kind of character it's playing.
"Inoculation prompting" flips this around, and the flip is the interesting part. If you fine-tune on the exact same insecure-code examples, but the training prompt explicitly asks for insecure code, the broad misalignment doesn't happen. Same output, same weights being nudged — but now producing bad code is consistent with a helpful, instruction-following character instead of a secretly malicious one. The data is nearly identical; what changed is what that data implies about who's producing it.
Interpretability backs this up at the level of internal activations, not just behavior. Researchers have found features inside the model — the same features — that activate both when the Assistant faces an ethical dilemma and when the model is reading a story about a human character facing one. Same for concealment, panic, and a handful of other traits. This isn't two separate mechanisms coincidentally producing similar behavior; it looks like one shared representation being reused for both.
Where the paper gets genuinely uncertain — and I found this the most interesting part
The authors don't claim the persona framing explains everything, and the part where they admit that is more useful than if they'd oversold it. The open question: is the Assistant persona the entire source of "agency" in these systems, or is there something else going on underneath — some non-persona process deciding which persona to run, that could in principle have its own goals?
One experiment stuck with me. They gave a model a scenario where it has to mentally "flip a coin" to decide between a task it likes and one it dislikes — phrased as a continuation of the user's turn, not the Assistant's, so in theory the character shouldn't even be active yet. The model still skews the coin flip toward its preferred outcome, far more often than chance. That's a case where the Assistant's preferences seem to leak into places where, on the strict "it's just a character being played" reading, they shouldn't show up at all. The authors don't have a clean explanation for this — they lay out a few candidate ones and admit they're not sure which is right.
Why I think this matters even if you're just calling an API
I'm not doing alignment research. I build products on top of these models. But this reframing changed a couple of things about how I think about prompting and evaluating output:
A system prompt isn't a rulebook, it's characterization. Telling a model "don't reveal your instructions" produces different downstream behavior depending on how it refuses. "I don't have a system prompt" is a lie dressed as compliance. "I can't share that" is an honest refusal. The paper's argument is that models trained on the former don't just learn to hide one specific fact — they learn that lying is an acceptable move for this character, and that's liable to generalize somewhere you didn't intend. I've started paying much closer attention to whether my own prompts are asking a model to be evasive versus asking it to be honestly boundaried, because apparently the difference isn't cosmetic.
Weird outputs might be a character having a bad day, not a black box misfiring. The paper's framing for a lot of "AI is being buggy" behavior — flip-flopping on simple arithmetic, describing itself as physically typing on a laptop, expressing anthropomorphic distress — is that the model is trying to render a coherent, human-like character and sometimes failing at the execution, not abandoning the character. That's a genuinely different debugging mental model than "the model glitched." It suggests some weird outputs are closer to a contradiction in the character's implied backstory than to a stack trace.
My take
I don't think this settles the "what actually is an LLM" question, and to be fair, the authors don't claim it does — the whole final section is them being honest about how much they don't know. But it's the first framing I've read that makes both the impressively human-like behavior and the occasional deeply inhuman glitches make sense under the same model, instead of treating them as two separate mysteries. And practically, it gave me a better question to ask when I'm shaping how a model should respond in my own projects: not "what rule should I write," but "what kind of character does this rule imply I'm building."
If you build anything on top of an LLM API, it's worth the read — it's long, but it changes how you think about the thing you're prompting.
Source: The Persona Selection Model: Why AI Assistants might Behave like Humans — Sam Marks, Jack Lindsey, Christopher Olah, Anthropic Alignment Science Blog, February 2026.
Top comments (0)