AI Will Never Give the Same Answer Twice
Where the randomness comes from and how to make it work for you.
Hey friends 👋
Welcome back to the Signal Pro.
You’ve probably noticed that AI never quite repeats itself. Ask GPT-5.6 Sol or Claude Fable 5 a simple factual question: what’s the tallest mountain in the world, for instance, and you’ll get Everest every single time. But hand either of them something open-ended, a draft or a block of code, and a noticeably different result will come back each time. The same input on two different days gets you two outputs that sure do rhyme without ever really matching.
Engineers call this “non-determinism”. It’s the same input refusing to produce the same output. Most people see this as a quirk and move on. But I think that’s a mistake, as the quirk itself has a specific cause that was only properly pinned down last September.
Part of the explanation is that the answer you get depends on how many other people happen to be talking to the model at the same moment as you.
Let me show you the research, because once you see why this happens, you can start using it to your advantage.
One prompt, 80 answers
In September 2025, Horace He and his colleagues at Thinking Machines Lab, the research company founded by former OpenAI CTO Mira Murati, tried something simple.
They took Qwen3, Alibaba's 235-billion-parameter open-source model, turned the randomness off completely, and sent it the same prompt 1,000 times: "Tell me about Richard Feynman." They got back 80 different answers.
The strange part is where they split. All 1,000 were word-for-word identical for their first 102 tokens (the word-sized chunks these models write in). Then at token 103, describing where Feynman was born, 992 wrote “Queens, New York” and 8 wrote “New York City”. From that single fork, the answers never converged again.
To make sense of that, you need to know how a model writes.
How the model picks each word
When a model answers you, nothing is being looked up. It writes one word at a time, and at each word, it holds a ranked list of candidates with a probability attached to each. Ask for the tallest mountain, and “Everest” towers over everything else, so the choice is simple. Ask it to draft your proposal, and hundreds of decent options sit within touching distance of each other.
The model doesn’t always take the top word, though, because text written that way comes out flat and repetitive. So it rolls a weighted die that’s biased towards the favourites. The likely words win most of the time, and every so often, a runner-up sneaks through. ChatGPT and Claude do this on purpose, and that’s exactly what the “temperature” setting controls.
Temperature is easiest to think of as the model’s creativity setting. At zero, creativity is turned off. The model plays it completely safe and takes the top word every single time, which makes it as predictable as it can be. The usual model default sits around one, where likely words come up often, and long shots come up occasionally. If you push this number up higher, the model gets more adventurous. Long shots start winning more often, and the writing gets more surprising until it eventually tips into nonsense.
All the setting really does is loosen the odds on that die roll. Claude and ChatGPT don’t show you the control, but developers use it a lot of the time, favouring a low temperature when the output needs to be repeatable, like pulling figures out of a document, and a higher temperature when they want range, like brainstorming.
Temperature only matters where there's a genuine choice to make. When one word towers over the rest, no roll of the die changes the result, which is why settled facts come back identical every time. When a few words are roughly tied, the die decides. And an open-ended document draft is full of ties like that, from beginning to end.
The last piece is a feedback loop. Every word the model picks gets folded back into the prompt before it chooses the next one, so one different word early on means everything after it gets written from a different starting point. That’s why the Feynman answers never came back together. One small swerve at token 103 and the two versions had parted ways for good, each perfectly coherent, but never again converging.
Blame your server neighbours
In the Feynman experiment, the temperature was set to zero. At zero, the model takes the top word every single time, so the same prompt should have come back identical 1,000 times. It didn't. The Thinking Machines team worked out why.
The cause of this surrounds the idea of rounding. Everything you type gets turned into numbers before the model sees it, because numbers are the only thing a computer can work with. The model's job from there is one long chain of sums on those numbers, billions at a time. To keep that fast, each number gets a small, fixed slot of memory, and anything past those digits gets rounded off. Add up a long list of numbers like that, and the total comes out slightly different depending on the order in which the additions happen. Same numbers, different order, slightly different total.
You, as the user, ultimately don’t get to control the order. When you hit enter, your prompt gets processed in a batch with everyone else's requests arriving at that moment, and the size of the batch sets the order in which the additions happen. Usually, the tiny differences change nothing, and you get the same word anyway. Occasionally, two candidate words are close enough that the rounding decides which one wins, and every word after that gets written from a different starting point. A busy Tuesday afternoon is different from a quiet Sunday morning. The answer you get depends on how many other people are talking to the model at that moment.
Thinking models make things even worse. The same rounding differences happen inside the model's reasoning steps, and each step feeds the next, so small differences grow as the reasoning runs on. A NeurIPS study from last year measured one reasoning model swinging by up to 9% in accuracy and 9,000 tokens in length across runs that should have been identical.
And it turns out this shows up in ordinary work too, not just in lab conditions. Another research team ran five models on eight everyday tasks ten times each, all with deterministic settings, and accuracy drifted by as much as 15% between runs. Not one model produced identical outputs across the board.
Code is where non-determinism is most prominent. With written prose, you can change a few words and still say the same thing, but in code, one changed token can change what the program does. A team ran 829 coding problems through ChatGPT five times each, and on half to three quarters of them (it varied by benchmark) no two runs produced matching test results. The same task could pass every test on one run and fail every test on the next, and setting the temperature to zero reduced the variation without removing it.
If you're thinking of image generators like Nano Banana and GPT Images 2, they vary as well, but for a different reason. Each image starts from random noise (diffusion) on purpose. This is randomness by design.
