The promise of AI-assisted diagnostics hinges on a critical assumption: that making AI more transparent improves outcomes for everyone who uses it. New findings from Nature Medicine challenge that assumption directly, revealing that the benefits of explainable AI in dermatology are not evenly distributed — and that greater transparency may actually backfire depending on who is reading the explanation.
The study examined large language models equipped with explainability features — mechanisms designed to show users why an AI reached a particular dermatological diagnosis. When primary care physicians used these explainable AI tools, diagnostic performance improved, suggesting that clinical training enables practitioners to critically evaluate AI reasoning, integrate it with their own knowledge, and course-correct when the model errs. For lay users, however, the effect diverged markedly: explainability features did not confer the same benefit and may have introduced overreliance or miscalibrated confidence, as users without medical training lacked the framework to critically assess AI-generated rationales.
This finding sits at a significant intersection of human-computer interaction and clinical safety research. A growing body of work on algorithm aversion and automation bias suggests that non-expert users often either over-trust or under-trust AI outputs in unpredictable ways, and that adding explanation layers does not reliably solve this problem. The dermatology context is particularly consequential: skin cancer misclassification carries serious downstream risk, and consumer-facing AI skin checkers are already widely accessible. The study's central limitation is that the precise mechanisms driving the lay-user divergence — whether overconfidence, anchoring on plausible-sounding rationale, or inability to detect model errors — remain incompletely characterized. This is an incrementally confirmatory result for experts in AI transparency, but a potentially paradigm-shifting caution for developers deploying explainable AI directly to the public without clinical intermediaries.