Somebody drops this in a channel:
Gemini handles the edge case fine, look: [screenshot of an AI Studio response]
Three weeks later you try the same thing and get something else. Who was wrong?
Neither of you, probably. You ran a different experiment and had no way to know, because the thing that was shared was the output, and the output does not carry the conditions that produced it.
AI Studio is not a chat app
That framing matters. ChatGPT and Claude are products; you have a conversation and the conversation is the artifact.
AI Studio is a bench. You are tuning a prompt, and the responses are samples, not the point. The thing you are actually building lives around the transcript:
- the system instruction, in its own collapsible panel above
- the model and version, in a dropdown
- temperature, top-p, top-k, output length, safety settings — in the configuration panel on the right
- optionally a structured output schema
None of that is in the message stream. The transcript is one variable in an experiment whose other variables sit in surrounding UI.
Copy the transcript and you have copied the readout of a meter without noting what you were measuring.
The reproducibility bill
For a colleague to actually reproduce a result, they need every one of these:
| Component | Where it lives | Survives a copy-paste? |
|---|---|---|
| The turns | message stream | yes |
| System instruction | collapsible panel | only if expanded |
| Model + version | dropdown | no |
| Temperature / top-p | right-hand config | no |
| Output schema | config | no |
| Seed | not exposed | n/a |
Four of six are furniture, not content. And temperature above zero means even a perfect replication gives a different sample — which is fine, as long as everyone knows that is what happened.
The failure mode is not dramatic. Nobody notices the conditions are missing, because the output looks complete. It reads like a finding. It is a single draw from a distribution whose parameters nobody wrote down.
What to do about it
Put the parameters in the prompt file, not in your memory. If you are iterating seriously, keep a text file next to the work: model, version, temperature, top-p, and the system instruction verbatim. It is thirty seconds and it is the difference between an experiment and an anecdote.
Quote the system instruction when you share a result. It is the single highest-value piece of context. A response without it is uninterpretable — you cannot tell whether the model was clever or whether the prompt did the work.
Say the temperature out loud. "Temperature 0.2, three of three runs agreed" is a claim. A screenshot is a vibe. If you ran it once at 1.0, say that too; it is still useful, just differently.
Ask for structured output as a code block. If you want JSON to survive into a document as formatted JSON, it has to be rendered as a code block. Ask for it inside fences. Prose-shaped JSON exports as prose, because there is no block for anything to preserve.
Never share a screenshot as evidence. It captures the fewest variables of any method available and cannot be searched, quoted or diffed.
Disclosure
I build an export tool for this, so read the next part with that in mind.
Chat Exporter for AI Studio writes a session to Word, PDF, Google Docs, Notion, Markdown or JSON. It captures the system instruction alongside the turns, keeps every user turn and model response labelled and in order, and preserves code blocks and structured output as blocks rather than wrapped text. Free tier, paid for unlimited.
Here is the honest part, and it is the reason this article is not a pitch: it does not capture the model parameters. Temperature, top-p and the rest live in the configuration panel, which is not part of the conversation, so they are not in the exported document. If reproducibility is what you need, you still have to write those down yourself.
Same for the system instruction if its panel is collapsed — it has to be expanded to be on the page at all. And the export buttons only appear once there is at least one response, since they attach per message.
I would rather say that plainly than have you find out from a document that was missing half the experiment.
The general shape
This is not really about AI Studio. Any tool where configuration is separate from output has the same hole, and prompt benches are the current example.
The question worth asking of anything you export: could someone reproduce this from what I just saved? If the answer is no, the missing pieces are almost always the settings — and settings are exactly what interfaces put in side panels, where no export reaches.
If you run prompt experiments seriously, how do you version the config? I keep a plain text file per prompt, which works and is clearly not the right answer.
Top comments (3)
There is one row in your table where writing the value down does not actually freeze the condition, and it is the row your opening scenario runs through. Temperature, top-p and the system instruction are values: copy them into the file and the file holds the thing itself. The model identifier is a pointer, and the provider is on the other end of it. Three weeks is exactly the interval over which a pointer gets re-aimed, so the prompt file you recommend would have recorded the same model string for both runs and shown a clean match while the underlying weights had moved. That is worse than the screenshot in one specific way: the screenshot only fails to carry the conditions, whereas the record certifies a reproduction that did not happen.
The cheap fix keeps the discipline and needs nothing from the provider. Park one fixed canary prompt at temperature 0 in the same file and store its exact output next to the parameters. When you come back and the canary still matches, the model string is telling the truth about being the same condition; when it does not, you know before you start arguing about the edge case. It also gives you a positive control for the thing you cannot write down, since a seed you never had is one more reason the canary has to be checked rather than assumed.
This is the correction the piece needed, and it lands on the row my own opening runs through. I put "model + version" in the same table as temperature and top-p as though they were the same kind of thing. They are not. Two of them are values and the file ends up holding the thing itself; the third is a name I do not control the referent of.
The severity ordering is the part I had not thought through. A screenshot omits the conditions and everyone can see that it omits them. A parameter file that records a re-aimed pointer looks complete, matches on both runs, and asserts an equivalence that did not hold — so it sends you off debugging the edge case instead of the weights. Failing loudly beats failing quietly, and I recommended the quiet one.
Taking the canary. The bit I like most is that it is a positive control for the seed, which is the row I marked n/a and then stopped thinking about.
One caveat on reading it, in the strengthening direction: temperature 0 is not bit-deterministic on hosted inference — batching and floating-point non-associativity mean identical weights can still diverge. So a match is strong evidence nothing moved, a mismatch is an alarm rather than a verdict. Which is still enormously better than assuming, and it is the direction you want the error to point.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.