Every calibration comes down to a number. A tolerance on a procedure, a guard band on a pass/fail decision, an uncertainty on a certificate: each one decides whether an instrument goes back into service, and each one reaches well beyond the bench, into every product and measurement that depends on that instrument. AI is arriving in calibration laboratories, and it’s remarkably good with language. The question every metrologist should be asking is what it should be allowed to do with numbers. Building MetTutor forced its development team to answer that question, more than once.

During development, the MetTutor team asked a language model to write calibration procedures from values entered in a form. One test was a torque wrench procedure based on ISO 6789:2003, the standard for hand torque tools, with readings in foot-pounds.

The procedures that came back were well organized and clearly written. They also didn’t say what the team had entered. Across three rounds of testing, the reference standard the team specified was swapped for a different manufacturer’s. Foot-pounds became newton-metres. Three repeat readings became five. A two-source uncertainty budget grew to four or more sources, and the model supplied organization and author names nobody had typed.

None of it looked like an error. Every substitution was a reasonable choice a knowledgeable person might make. That was the problem: the model wasn’t copying the team’s values. It was improving them.

A word on where this comes from. This article draws on the experience of the team that builds MetTutor, an AI-powered metrology education platform, so it has a stake in the argument. The team uses large language models every day and believes they have a real place in metrology education. What follows is what development and testing taught the team about where that place ends: a model can explain a measurement, but it should never be the source of one.

Fluent is not faithful
A language model produces the most plausible continuation of the text in front of it. Most of the time, plausible and correct coincide. In metrology, the gap between them is where the damage happens. NIST’s Generative AI Profile names this risk confabulation: a system confidently producing false information.

The team’s first response was the obvious one: write better instructions. Three revisions of the prompt told the model, with increasing force, to use the values exactly as entered. None of them held. The team’s conclusion, recorded in its development notes, was blunt: prompting alone cannot stop a model from overriding inputs when it has strong domain knowledge. The architecture was wrong, not the prompt.

So the team rebuilt the procedure generator as a deterministic tool, with no model in the path. Entered values now go straight into a fixed template, and every value reaches the procedure exactly as typed.

Then it happened again
Two months later, the same lesson arrived from a different direction. MetTutor’s calculators had been built the conventional way: the model read the inputs, did the arithmetic, and reported the result.

Testing caught a Decision Rules calculation where the guard band the model reported didn’t match its own worked arithmetic a few paragraphs below. The worked value was the right one. This wasn’t a rounding difference; it was a different number. A guard band is a pass/fail boundary, and one that’s too small widens the acceptance zone, so more marginal instruments pass.

Testing found the same kind of disagreement in a Welch-Satterthwaite result. The team tried two prompt-level fixes. Both still produced wrong numbers.

The second time, the team didn’t need three rounds to reach the answer. Instead of another prompt revision, it took the arithmetic away from the model entirely and replaced it with deterministic calculation tools: ordinary code whose output is fixed by its input. For Welch-Satterthwaite and Decision Rules, the deterministic tool reads the values from the submitted form, checks that every required value is present, applies the formulas, and looks up coverage factors and guard band fractions in tables built into the software. The result card, the headline, and the exported report are all built from those computed values.

The model still runs, but its job changed. It receives the finished numbers with an explicit instruction to explain them, not recompute them. The disagreement testing caught can’t happen anymore, because there is only one calculation, and a deterministic tool performs it the same way every time.

That second fix turned a one-time repair into a design rule the team now applies across the platform: the model never gets to be the source of a number.

Three jobs, three sources of trust
MetTutor now divides its work among three components: a controlled reference library and two kinds of deterministic tools, each trusted only with what it can be trusted with. A deterministic tool is one whose output is fixed entirely by its input: give it the same values and it returns the same result, every time. It’s the same property a calibration lab already demands of a validated method.

Knowing: MetLibrary. MetLibrary wasn’t added later to patch a problem. It has been a central part of MetTutor’s design from the start, because the team recognized early that a metrology platform needs its own controlled reference library, not whatever a model absorbed in training. It now holds more than 950 standards and reference documents. When a user asks what a standard requires, the system retrieves the relevant passages and answers from them, citing the document and clause. This approach, retrieval-augmented generation, was introduced in a 2020 paper by Lewis and colleagues, who found that grounding a model in retrieved documents produced more specific and more factual answers than relying on what it had memorized.

Calculating: deterministic code. Welch-Satterthwaite and Decision Rules now work the way described above: code calculates, and the model explains. MetTutor’s other calculators are being moved to code the same way. Some of them agreed with their own arithmetic in testing after a prompt change, but agreement in testing is not a guarantee, and that’s the point of this article.
Published research supports the design. In Program-Aided Language Models, handing the calculation to code instead of letting the model do the math produced more accurate results than much larger models working alone. It also avoids a problem that hasn’t been solved inside the models themselves: one 2024 study found that language models given identical inputs, with randomness turned down to zero, still produced varying output.

Transcribing: deterministic templates. When values move into a document, as in the procedure generator, a fixed template places each entered value in a defined position. The template decides where a value goes. It never decides what the value is. One option goes further: the procedure references the manufacturer’s datasheet as the controlled source instead of retyping its specifications at all.

The model does everything else, and it’s very good at it: understanding a question asked in plain language and explaining a result so a technician actually understands it.

What deterministic tools give MetTutor
Moving the numbers out of the model changed more than the error rate. It changed what MetTutor can promise.

Values arrive exactly as entered. A template can only place what the user typed. The substitutions that marked the first procedure generator can’t happen.
The arithmetic is right every time. Formulas and lookup tables, such as the GUM’s table of t-distribution coverage factors, live in code, not in a model’s memory.
Results can be validated. The same inputs always give the same result, which is what ISO/IEC 17025 expects before laboratory software is put into service (7.11.2). Behavior changes only when the team changes the code, never because a vendor updated a model.
One number has one source. The result card, the headline, and the exported document are all built from the same computed value, so they can’t disagree with each other.
Missing inputs stop the calculation. If a required value is missing or isn’t a number, the tool says what’s missing instead of quietly assuming something, such as a default coverage factor of k = 2.
Results can be checked. A deterministic calculation can be compared against hand calculations and published worked examples. A model’s answer can only be sampled.
The model teaches better. Freed from arithmetic, it does what it’s good at: explaining why a guard band matters, or why the same reading can pass under one decision rule and fail under another.

Deterministic tools don’t make MetTutor smarter. They make it accountable: every number has one origin, and that origin can be inspected.

A result you can check
Here’s how MetTutor handles a Decision Rules question now:
Code reads the measured value, tolerance, and uncertainty from the form.
Code looks up the guard band fraction in a fixed table and applies it to the tolerance.
Code sets the acceptance limit, decides the result, and applies all four decision rules to the same measurement for comparison.
The model explains why the same reading can pass under one rule and fail under another.

Every part of that answer has a known origin. The number comes from a reproducible calculation, and the explanation comes from the model. Nothing important rests on the model’s memory.

Faithful is not the same as correct
None of this makes MetTutor infallible, and it’s worth being precise about what it doesn’t do.

Retrieval narrows the problem; it doesn’t eliminate it. A Stanford and Yale study of commercial AI legal research tools, a field that leans on citations as heavily as ours, found that retrieval reduced errors compared with general-purpose chatbots, yet the tools still produced false information 17% to 33% of the time. Retrieval can still pull the wrong passage or a superseded edition. Code faithfully computes whatever it’s given, so a mistyped input produces a precisely wrong answer. A template carries a wrong tolerance into every copy, perfectly.

What the design changes isn’t whether errors can happen, but whether they can be found. Every value has a source you can open and a method you can reproduce. That’s also why the final check stays with people. ISO/IEC 17025 expects a laboratory to verify a method before putting it into use (7.2.1.5) and to approve documents before they’re issued (8.3.2). A generated procedure arrives with its approval signatures blank. It’s a draft until someone qualified checks it against its sources and signs it.

Three questions for any AI tool in your lab
Where does each number come from? If the answer is “the model,” it isn’t ready to become a record.
Can I reproduce the calculation? If you can’t see the formula and get the same answer by hand, you can’t validate it.
Can I check the source? A claim about a standard should come with the document and clause.

The lesson under the lesson
Look at that architecture again: a cited source, a documented calculation, and a result whose path back to its origin is unbroken and open to inspection. Metrology already has a word for that logic. We call it traceability.

That was the real lesson, and testing had to teach it twice. Making AI trustworthy in our field doesn’t require inventing a new kind of trust. It requires making the AI earn the kind we already have.

AI is going to change how we train technicians, write procedures, and run laboratories. The people best placed to make sure that change is done well are the ones who already understand why a value must stay unbroken from its source to its use. That’s us. So ask where the numbers come from, of every tool your lab adopts, MetTutor included.