Why AI Gets Numbers Wrong
Handling calculations, statistics, and financial data
① AI doesn't calculate numbers — like every other output, it just picks the next plausible token. The more digits involved, especially in operations with carrying (multiplication, division, growth rates), the higher the odds of a wrong answer.
② Code execution fixes the arithmetic itself. A real interpreter does the math. But if the model misremembers the input numbers it feeds into that code, a "correct calculation" produces a "precisely wrong answer."
③ Financial data stacks percentages, ratios, and currency conversion on top of each other — tripling the places an error can creep in. The only safe move is verifying numbers against the primary source, not the AI.
① Numbers aren't computed — the limits of token prediction
As covered in how LLMs work, a model does exactly one thing: pick the highest-probability next token given everything so far. That's true whether it's writing prose or handling numbers — there is no separate calculator running inside the model.
Numbers are text, so they get tokenized like everything else. The problem is that tokenization doesn't split numbers the way humans think about them. Depending on the tokenizer, "1400" might become a single token, or split into "14" and "00" — the boundaries rarely line up with how people reason about place value.
Two-digit addition and multiplication show up so often in training data that they're almost memorized as patterns. But as digit count grows, the frequency of any specific combination drops exponentially, and operations with multiple carries produce combinations that look plausible but are subtly off. Even three-digit multiplication tends to see a noticeably higher error rate. What makes this dangerous is that the digit count is usually right — it's one digit in the middle that's wrong, so at a glance the answer looks correct.
② Why percentages and growth rates are especially risky
More common than raw arithmetic in practice are growth-rate questions. "500K last month, 650K this month — what's the percent increase?" chains subtraction, division, and a percentage conversion. Every additional step is another place for error to creep in.
In financial writing, this shows up in a more deceptive form. "Grew 18% year-over-year" is grammatically and structurally flawless whether the real number is 18% or 13%. The sentence looks equally natural either way. A single swapped digit reads as completely normal.
| Operation type | Why the error compounds | Practical example |
|---|---|---|
| Simple growth rate | Subtraction + division + percentage, 3 chained steps | "What's the month-over-month increase?" |
| Reverse calculation | Must back out the original value from the result | "This year is X, growth is Y% — what was last year?" |
| Compound metrics | Multiple calculations stack, errors accumulate | Ratios like P/E, P/B between two separate numbers |
| Currency conversion | Adds a third variable — the exchange rate | USD earnings converted to KRW |
During earnings season, asking AI to "summarize this quarter's results" stacks all four of the above at once — revenue growth, the beat/miss versus consensus, EPS, and currency-converted figures are all numbers. Trusting an AI summary for investment decisions without cross-checking it against filings-based material like the Big Tech earnings guide is risky.
③ What changes when code execution is available
Recent AI tools can actually run code mid-answer (ChatGPT's Code Interpreter, Claude's code execution, and similar features). When that happens, the arithmetic itself is no longer token prediction — it's a real calculation. A Python interpreter adds, multiplies, and divides exactly the way a calculator would, so if the code genuinely ran, the result is as accurate as punching it in yourself.
Even so, the final answer can still be wrong, for three reasons.
1. Wrong inputs produce precisely wrong results
Code accurately computes whatever numbers it's given — it doesn't recall the correct numbers on its own. If the model misremembers revenue or a growth rate and feeds that into the code, the computation is flawless but the result is wrong. Garbage in, garbage out applies to AI just as much as to any other system.
2. The model decides on its own whether to use code
Models don't reach for code on every calculation. Anything that looks simple enough often gets answered from memory (i.e., token prediction) instead. Since the model's sense of "simple enough" doesn't always match a human's, it can skip code precisely on the calculations most prone to error.
3. Summarizing the result back into a sentence reintroduces risk
Even when the code returns an accurate number, turning that into "so revenue grew 18% year-over-year" is, once again, token prediction. It's not rare for the number to drift slightly during that final summarization step.
Code execution makes the "calculation" step safe. But the "input" and "summary" steps are still token prediction, and still vulnerable. Having code execution available doesn't mean every number can be trusted blindly.
④ Patterns to watch for in financial data
Most numbers used in investment decisions carry two or three of the risk factors above at once. Watch for these patterns in particular.
- Beat/miss versus consensus — requires knowing both the actual result and the market estimate; get either one wrong and the whole surprise percentage is off.
- YoY / QoQ growth rates — mixing up the comparison period is common: comparing to the prior quarter while labeling it "year-over-year."
- Currency-converted amounts — if it's unclear which date's exchange rate was used, the converted figure can't be verified at all.
- Ratios (P/E, P/B, etc.) — the numerator or denominator may be a stale, training-time value rather than the current one.
⑤ Where AI is still genuinely useful with numbers
Read this far and it's tempting to conclude "never let AI touch numbers" — that goes too far. What AI is actually bad at is remembering and computing exact figures, not working with numbers in general.
- Deciding which metrics matter is a strength. "What should I compare to judge if this company is cheap?" gets a genuinely useful answer.
- Setting up the right formula is a strength too. Explaining how to compute operating margin, for instance, is reliable.
- Explicitly requesting code execution makes the arithmetic itself trustworthy — as long as you supply the input numbers yourself, pulled directly from the source.
The line to draw isn't "use AI or don't" — it's "is this task a calculation or a judgment call?" Delegate the judgment. Verify the calculation.
⑥ Closing — a practical checklist
□ Multiplication or division with 3+ digits? → Explicitly request code execution
□ Growth rate or beat/miss percentage? → Verify both original numbers against the filing
□ Year-over-year or quarter-over-quarter comparison? → Double-check the comparison period
□ Currency-converted amount? → Ask which date's exchange rate was used
□ Input numbers going into a calculation? → Paste them in yourself, not from AI memory
The bottom line: it's fine to ask AI what to calculate. But the numbers that go into the calculation should always come from the primary source. That one rule alone prevents most AI-driven mistakes in financial data.
※ This article reflects information as of August 2026 and is not investment advice. If you use AI-derived figures for investment decisions, verify them against primary sources such as company IR pages, SEC filings, or exchange data. Investment decisions and their outcomes are the investor's own responsibility.
※ This guide is provided for general educational purposes and simplifies technical details for readability.
New guides, when they land
We publish AI literacy guides twice a week. Subscribe and the next one comes to you — free, unsubscribe anytime.