When the people defining the correct answer don't always agree, one accuracy number can hide more than it reveals.
If I told you an AI triage system agreed with doctors more than 90% of the time, that would probably sound like a strong result.
It was one of the strongest findings from our pilot study at the University of Abuja Teaching Hospital.
Across 61 triage sessions, MedGemma agreed with one independent clinician on 90.2% of urgency classifications and with the other on 93.4%. Weighted agreement was also substantial.
Then you look at the clinicians.
They agreed with each other on 90.2% of those same cases.
Suddenly, the obvious question, How accurate is the AI?, becomes more complicated.
There is another question that has to come first:
Accurate compared with what?
The problem with a clean ground truth
Machine learning likes ground truth.
You have an input, a correct label and a model prediction. Compare both and calculate performance.
Clinical decisions are not always that clean.
In our study, every AI triage session was independently reviewed by two clinicians. Neither clinician could see MedGemma's classification or the other clinician's decision. Each classified the case as routine, urgent or emergency.
The two clinicians disagreed on six of the 61 sessions. In five of those cases, one clinician chose a more cautious urgency level than the other.
Consider a simple case:
AI: routine
Clinician A: urgent
Clinician B: routine
Is the AI wrong?
Against Clinician A, yes.
Against Clinician B, no.
Nothing about the patient changed. Nothing about the model changed. The evaluator changed.
This does not mean AI errors should be excused because doctors sometimes disagree. In healthcare, disagreement is exactly the kind of thing we need to understand properly.
It means a human label should not automatically become perfect ground truth simply because a human produced it.
One clinician gives you a reference.
Multiple independent clinicians start telling you something about the reliability of that reference.
That difference matters.
Not every disagreement means the same thing
The actual disagreement patterns made this clearer.
There were cases where MedGemma disagreed with one clinician while matching the other. In session S064, for example, the AI classified the case as emergency. Clinician A also chose emergency, while Clinician B chose urgent.
But there was another pattern.
In session S013, MedGemma classified the case as routine while both clinicians independently classified it as urgent. S065 showed the same pattern. These should not be treated as the same type of disagreement.
Compare them:
Case A
AI: routine
Clinician A: urgent
Clinician B: routine
Case B
AI: routine
Clinician A: urgent
Clinician B: urgent
If all we report is that the AI "disagreed with a clinician," both cases look similar.
They aren't.
In Case A, the available clinical reference is itself divided.
In Case B, the clinicians converge and the AI is the outlier.
That second case naturally deserves more scrutiny.
This is where a single accuracy percentage starts to hide useful information. It compresses different types of disagreement into one number.
And in clinical AI, the structure of the disagreement matters.
The evaluator also needs evaluating
In our aggregate results, MedGemma's weighted agreement with the two clinicians was 0.659 and 0.719. The clinicians' agreement with each other was 0.675.
I would not interpret that as "AI is as good as doctors." Our study was too small for a conclusion like that, and agreement on an urgency classification is not the same as demonstrating clinical safety.
But it tells us something important.
Human judgement itself contains variation.
If we had validated the system against only one clinician, that variation would have disappeared completely. We would have had a cleaner result, but not necessarily a more truthful one.
Sometimes another evaluator makes the result messier.
Good.
Reality is allowed to be messy.
Evaluation should help us understand the system, not produce the neatest possible performance number.
More information changed the picture
There was another finding that made me rethink what we call model performance.
Our triage system was conversational. It could gather additional information over multiple turns and use the patient's recorded history as context.
Agreement was stronger in conversations containing four or more patient turns.
MedGemma's weighted agreement with Clinician A increased from 0.465 in shorter conversations to 0.705 in longer ones. Against Clinician B, it increased from 0.646 to 0.731.
But the clinicians also agreed more with each other.
Their weighted agreement increased from 0.330 in shorter conversations to 0.781 in the longer ones.
The sample here was small, particularly for the shorter conversations, and the confidence intervals were wide. So I would not claim that longer conversations caused better decisions. The paper itself warns against that interpretation.
But the pattern raises an interesting possibility:
Part of what we call model performance may actually be information availability.
If the AI and clinicians both agree more when more information is available, then some difficult cases may not simply involve a bad classifier.
The case itself may still be ambiguous.
A patient might begin with:
"My stomach hurts."
That isn't much to work with.
Where exactly? For how long? How severe? Any vomiting? Fever? Medication? Previous episodes? Other symptoms?
The first statement is not necessarily enough to make a sensible triage decision. It is the beginning of an information-gathering process.
For conversational clinical AI, the ability to reduce uncertainty through useful follow-up questions may therefore be part of the capability we need to evaluate.
Accuracy is still important
None of this makes accuracy irrelevant.
Neither does clinician disagreement lower the standard we should hold AI to.
It means the number needs context.
When evaluating AI triage, I would want to know three things beyond the headline performance.
First, how reliable is the reference standard?
If one clinician provides every label, we know how well the AI matches that clinician. We don't necessarily know how stable that judgement is.
Second, what kind of disagreement occurred?
AI disagreeing with one clinician while another clinician agrees with the AI is different from AI disagreeing with two independent clinicians who agree with each other.
Third, under what conditions does agreement change?
Does additional patient information help? Do longer interactions introduce useful context or cause the model to drift? Does the same behaviour hold across different languages and patient populations?
Our pilot could not answer all of those questions. It was conducted at one institution, the concordance sample was small, and MedGemma was not tested in Nigerian languages or code-switched English. A larger study with additional independent clinical validation is still needed.
But those limitations also tell us where better evaluation should go next.
There is something attractive about saying:
The model is 92% accurate.
It is clean. Easy to compare. Easy to put in a table.
Healthcare is not obligated to be equally clean.
Sometimes the AI is wrong.
Sometimes two clinicians interpret the same information differently.
Sometimes everyone simply needs more information.
A useful evaluation framework should be able to tell those situations apart.
If these systems are going to influence who stays home, who needs a doctor and who may need emergency care, matching one answer is only the beginning.
We also need to understand how stable that answer is, how much information produced it and what happens when reasonable clinical judgement itself is divided.
If two clinicians can look at the same case and arrive at different urgency classifications, the solution is not to lower the standard for AI.
It is to build a better standard for evaluating it.
This article draws from my research, “Artificial Intelligence and Mobile Health Technologies for Improved Healthcare Access: Design and Pilot Evaluation of an Integrated Digital Health Platform for a Nigerian Teaching Hospital,” published as a medRxiv preprint in September 2026. The study was designed as a feasibility pilot and has been submitted for peer review. DOI: 10.64898/2026.09.02.26361014.