diff --git a/foundation-frontiers/datasciencebites/posts/2025/11/21/uncertainty.qmd b/foundation-frontiers/datasciencebites/posts/2025/11/21/uncertainty.qmd index 86997018..cdefc678 100644 --- a/foundation-frontiers/datasciencebites/posts/2025/11/21/uncertainty.qmd +++ b/foundation-frontiers/datasciencebites/posts/2025/11/21/uncertainty.qmd @@ -1,7 +1,7 @@ --- title: "Beyond Quantification: Navigating Uncertainty in Professional AI Systems" description: | - If AI is to genuinely assist professionals, it must go beyond quantification. More than simplu accurate, it should be appropriately uncertain. + If AI is to genuinely assist professionals, it must go beyond quantification. More than simply accurate, it should be appropriately uncertain. author: Annie Flynn date: 11/21/2025 @@ -14,7 +14,7 @@ toc: true **Title:** Beyond Quantification: Navigating Uncertainty in Professional AI Systems -**Author(s) and year:** Sylvie Delacroix1, Diana Robinson, Umang Bhatt, Jacopo Domenicucci1, Jessica Montgomery, Gaël Varoquaux, Carl Henrik Ek, Vincent Fortuin, Yulan He, Tom Diethe, Neill Campbell, Mennatallah El-Assady, Søren Hauberg, Ivana Dusparic12 and Neil D. Lawrenceg (2025) +**Author(s) and year:** Sylvie Delacroix, Diana Robinson, Umang Bhatt, Jacopo Domenicucci, Jessica Montgomery, Gaël Varoquaux, Carl Henrik Ek, Vincent Fortuin, Yulan He, Tom Diethe, Neill Campbell, Mennatallah El-Assady, Søren Hauberg, Ivana Dusparic12 and Neil D. Lawrence (2025) **Status:** Published in *RSS: Data Science and Artificial Intelligence*, open access: [HTML](https://academic.oup.com/rssdat/article/1/1/udaf002/8317136) ::: @@ -30,7 +30,9 @@ This paper asks a simple but transformative question: **What if uncertainty isn The authors highlight a fundamental mismatch between the way today’s AI systems handle uncertainty and the way real professionals experience it. They distinguish between: + * Epistemic uncertainty – when we simply don’t know enough yet (e.g., missing data, incomplete measurements). *This can often be quantified.* + * Hermeneutic uncertainty – when a situation allows multiple legitimate interpretations, often shaped by culture, ethics, or context. *This cannot meaningfully be reduced to a percentage.* ::: {.column-page} @@ -39,10 +41,12 @@ They distinguish between: Professional judgment often depends on this second kind. Teachers, doctors, and lawyers rely on tacit skills: subtle perceptions, ethical intuitions, and context-sensitive interpretation. AI systems trained on statistical patterns struggle to reflect this nuance. -When an AI model gives a probability score—“I’m 70% sure this infection is bacterial”—it communicates something useful. But if the real uncertainty stems from ethical or contextual complexity (e.g., whether asking a patient certain questions might put them at risk), probability scores offer a false sense of clarity. +When an AI model gives a probability score — “I’m 70% sure this infection is bacterial” — it communicates something useful. But if the real uncertainty stems from ethical or contextual complexity (e.g., whether asking a patient certain questions might put them at risk), probability scores offer a false sense of clarity. The paper gives practical examples: + * A medical AI might be highly confident about symptoms but blind to the social dynamics suggesting abuse. + * An educational AI may accurately flag grammar issues but miss culturally sensitive interpretations in a student essay. In both cases, the most important uncertainties are precisely the ones that cannot be captured by numbers. @@ -53,14 +57,18 @@ The authors warn that the problem becomes even more serious as we move toward ag To address this, the authors propose shifting away from trying to algorithmically “solve” uncertainty, and instead enabling professionals themselves to shape how AI expresses it. -##Takeaways and implications +## Takeaways and implications **1. Uncertainty expression is part of professional expertise, not just a technical feature** AI should not simply output probabilities. It should help preserve and enhance the ways professionals reason through complex, ambiguous situations. That means: + * highlighting when interpretation is required + * surfacing multiple plausible perspectives + * signalling when ethical judgment is involved + * encouraging expanded inquiry rather than false certainty For example, instead of producing a diagnosis score, an AI assistant might say: “This pattern warrants attention to social context. Consider asking open-ended questions to understand the patient’s circumstances.” @@ -70,8 +78,11 @@ This kind of prompting respects and supports professional judgment. **2. Professionals—not engineers—must define how uncertainty is communicated** The authors propose participatory refinement, a process where communities of practitioners (teachers, doctors, judges, etc.) collectively shape: + * the categories of uncertainty that matter in their field + * the language and formats AI systems should use + * how these systems should behave in ethically sensitive scenario This differs from typical user feedback loops. Instead of individuals clicking “thumbs down,” whole professions deliberate on what kinds of uncertainty an AI system should express and how. @@ -84,8 +95,11 @@ This differs from typical user feedback loops. Instead of individuals clicking To make participatory refinement possible, future AI systems need: + * architectures that can incorporate community-defined uncertainty frameworks + * interfaces designed for collective sense-making, not just individual use + * institutional support (e.g., workshops, governance processes, professional committees) While this takes more time than simply deploying an AI system “out of the box,” the authors argue that in fields like healthcare or law, these deliberative processes are essential, not optional. @@ -99,15 +113,22 @@ Well-designed AI should help maintain that reflective space, not close it down. **Further reading** For readers interested in exploring more: + * David Spiegelhalter – The Art of Uncertainty (accessible introduction to uncertainty in science) + * Iris Murdoch – The Sovereignty of Good (on moral perception) + * Participatory AI frameworks such as STELA (Bergman et al., 2024) + * Visual analytics research on human-in-the-loop data interpretation + * Discussions of agentic AI systems and coordinated AI in healthcare + * Delacroix’s work on LLMs in ethical and legal decision-making -In summary +## In summary + This paper argues that if AI is to genuinely assist professionals, it must go beyond quantification. Numbers alone cannot capture the ethical, interpretive, and contextual uncertainties that define professional practice. Instead, AI should help preserve and enrich human judgment by communicating uncertainty in ways co-designed with the communities who rely on it. AI should not just be *accurate* —it should be *appropriately uncertain*.