The Clinical Data Challenge
Clinical AI inherits the processes that produced its data: who received care, what clinicians measured, which outcomes were recorded, and how information entered the record. EHR copying, informative missingness, representation gaps, and differences across hospitals can alter performance even when the software is unchanged, as Zech and colleagues demonstrated across chest-radiograph datasets from different institutions (Zech et al., 2018). Dataset provenance and validation context are part of the clinical evidence, not implementation details.
After reading this chapter, clinicians should be able to:
- Understand why clinical data differs fundamentally from other AI application domains
- Recognize data quality issues that undermine medical AI performance
- Identify the EHR data problems affecting AI deployment
- Evaluate dataset representativeness and potential biases
- Understand the importance of external validation
- Assess data requirements for different AI applications
- Recognize when data limitations preclude safe AI deployment
Introduction
Every medical AI system inherits the limitations of its training data. An algorithm trained on chest X-rays from one hospital may fail at another where imaging protocols differ. A sepsis prediction model validated on academic medical center patients may underperform in community hospitals with different patient populations. These data challenges determine whether AI systems work in practice.
Public documentation often omits important training and testing details. In a 2025 review of FDA-reviewed AI/ML devices, only 1.8% of summaries reported an exact training-dataset source and 3.6% reported an exact testing-dataset source (Mehta et al., 2025). The core appraisal question is whether the model’s development and evaluation conditions reflect the intended patients, equipment, documentation, threshold, and workflow.
Why Clinical Data is Uniquely Challenging
1. Missingness is Non-Random
- Lab tests ordered based on clinical suspicion (sicker patients have more data)
- Absence of documentation does not mean absence of finding
- Missing data patterns themselves carry clinical information
- Some analytic methods assume particular missing-data mechanisms; those assumptions must be justified for the task and estimand
Example: Patient with normal vital signs may have fewer documented vitals than deteriorating patient. AI must distinguish “stable so checked infrequently” from “actually normal values.”
2. EHR Data is Messy and Inconsistent
- Different documentation practices across providers, specialties, institutions
- Copy-paste errors propagate outdated information
- Free-text notes contain critical information not in structured fields
- Billing codes do not always reflect actual clinical diagnoses
- Temporal relationships matter (sequence of events, timing of interventions)
Example: Blood pressure documented as 120/80 could be actual measurement, could be copy-pasted from previous visit, could be manually entered days after visit.
3. Heterogeneity Across Healthcare Systems
- Different EHR platforms (Epic, Oracle Health, Allscripts) structure data differently
- Laboratory reference ranges vary by institution
- Clinical workflows differ substantially
- Disease prevalence varies by geography, institution type
- Documentation quality varies dramatically
Result: Performance can change across institutions even when the model is unchanged (Zech et al., 2018). The direction is not predetermined: transport can worsen, remain similar, or occasionally improve depending on case mix and task difficulty.
4. Temporal Complexity
- Clinical decisions depend on trajectories, not just snapshots
- Time-varying confounding (sicker patients get more interventions)
- Informative censoring (patients transfer, get discharged, die)
- Treatment effects take time to manifest
Example: Sepsis prediction must account for recent antibiotic administration, fluid resuscitation, ICU transfers, not just current vital signs.
Tokenising the patient journey
Mahmood and Topol frame a shift from isolating single variables toward temporal foundation models that tokenise the recorded patient journey: heterogeneous time-stamped observations (codes, measurements, text, images, and intervals) become model-readable units whose sequence can be learned into an evolving, queryable representation (Mahmood & Topol, 2026). Their Digital medicine essay ladders five recent scale examples without treating them as cleared products: Delphi-2M (~2 million people; UK Biobank + Danish registries; >1000 diseases), Curiosity (>100 billion structured events from >100 million patients), Apollo (structured events + text + images across three decades), ALADYNOULLI (longitudinal modelling + germline genetics across three biobanks; >683,000 people), and AURORA (seven modalities; >425,000 people; ageing and metabolic health). Named uses stay in the support / hypothesis-generation lane (rare-disease diagnosis support, trial recruitment, external comparators, intervention-point ideas). Observational journey representations do not establish causality; prospective validation, auditability of whose journeys are represented, and governance are required. Essay, not a trial; industry advisory/equity COIs disclosed (including Danaher, AstraZeneca, Microsoft, and others named on the published article).
Population-scale journey tokenisation framing is in the Public Health AI Handbook.
5. Measurement Variability
- Inter-observer variability (different clinicians measure differently)
- Intra-observer variability (same clinician varies over time)
- Equipment differences (different BP cuffs, imaging protocols)
- Biological variability (diurnal rhythms, stress responses)
6. Selection Bias is Pervasive
- Who gets tested? Who gets admitted? Who gets specific treatments?
- Referral patterns concentrate certain patients at certain institutions
- Clinical trials systematically exclude many real-world patients
- Academic medical centers see sicker, more complex patients
Key Insight: A model developed from tertiary-care data should not be assumed to transport to primary care without relevant evaluation.
Critical Data Quality Issues
Label Noise: Training labels (diagnoses) may be inaccurate - Billing codes are administrative observations, not automatically valid clinical reference standards - Diagnostic uncertainty not captured in structured data - Rare diseases frequently misclassified initially
Immortal Time Bias: Patients must survive long enough to receive certain treatments - Makes treatments appear more effective than they are - AI learns spurious protective associations
Confounding by Indication: Sickest patients get most aggressive treatments - Makes effective treatments appear harmful - Requires a causal design and analysis when the question concerns treatment effects; a predictive model alone does not solve confounding
Distribution Shift Over Time: Medical practice evolves - New treatments become standard - Diagnostic criteria change - Disease epidemiology shifts - Performance can change as the data-generating process changes (Finlayson et al., 2021)
Labels, Targets, and Proxies
A label may come from expert review, pathology, imaging interpretation, medication orders, billing codes, utilization, or a later outcome. Those sources are not interchangeable. Evaluation should document who created the label, when it became available, inter-rater agreement, adjudication, and how uncertainty was handled.
The target itself may be wrong for the decision. Obermeyer and colleagues found that a population-health algorithm used healthcare cost as a proxy for health need. At the same score, Black patients were sicker than White patients. Replacing cost with a direct measure of need increased the proportion of Black patients selected for additional care from 17.7% to 46.5% (Obermeyer et al., 2019). Accurate prediction of an inequitable or clinically misaligned target remains a target-design failure.
Dataset Representativeness Problems
Demographic Bias: Training data over-represents certain populations - Academic medical centers: disproportionately insured, urban, referred patients - Clinical trials: systematically exclude elderly, pregnant, children, complex comorbidities - Many widely used datasets have limited demographic, geographic, device, or acquisition diversity; the relevant gap must be measured for each dataset
Example: Dermatology benchmarks have documented representation and performance gaps across skin tones and disease categories (Daneshjou et al., 2022). This is evidence for the evaluated models and datasets, not a universal effect size for every dermatology system.
Geographic Bias: Disease patterns vary globally - Infectious disease prevalence differs by region - Genetic disease prevalence varies by ancestry - Environmental exposures differ - Healthcare access patterns differ
Specialty Bias: Dataset characteristics reflect specialty focus - Pathology datasets: only biopsied lesions (selection bias, most suspicious lesions) - Radiology datasets: only imaged patients (healthier patients may not get imaging) - ICU datasets: only critically ill (predictions may not generalize to wards)
Data Requirements by AI Application Type:
Supervised Learning: - A labeled sample adequate for prevalence, label reliability, model complexity, operating point, intended subgroups, and target precision - High-quality expert labels (pathologist-confirmed diagnoses, radiologist annotations) - Representative of target deployment population - Balanced classes (or appropriate handling of imbalance)
Deep Learning for Imaging: - No universal image count guarantees useful performance; required information depends on task complexity, transfer learning, label quality, prevalence, and evaluation precision - Diverse imaging equipment, protocols, patient populations - Expert annotations (bounding boxes, segmentations, diagnoses) - External validation on different equipment/populations
Clinical Prediction Models: - Cohorts sized for event prevalence, model complexity, the intended operating point, and confidence intervals in relevant subgroups - Complete follow-up (outcomes verified) - Temporal validation (train on older data, test on newer) - External validation at different institutions
Natural Language Processing: - Pretraining and task-specific sample needs vary by model, task, annotation reliability, language, and deployment distribution - Representation of documentation variability
Validation Designs and the Claims They Support
Internal validation can estimate performance in held-out data from the development source, but it does not establish transportability. Random splitting can also preserve site-specific shortcuts, duplicate patients, acquisition artifacts, and documentation patterns across development and test sets.
| Evaluation | Question answered |
|---|---|
| Internal validation | Does performance persist in held-out data from the development source? |
| Temporal validation | Does performance persist in a later period? |
| Geographic or site validation | Does performance persist in a different institution or system? |
| Device or acquisition validation | Does performance persist with different equipment and protocols? |
| Prospective workflow evaluation | Does the configured system function when used in practice? |
| Comparative outcome evaluation | Does using the system improve a prespecified patient or care outcome relative to a relevant comparator? |
External validation is not one design and does not itself establish clinical benefit. The source distinction, model version, threshold, interface, comparator, and endpoint determine what the study supports.
A 2025 retrospective mammography study demonstrated why calibration data must resemble the target population. When case-control timing, age, breast density, or equipment vendor distributions were deliberately mismatched, cancer-detection and false-positive rates changed materially after the resulting thresholds were applied to the representative target data (Schurz et al., 2025). The study simulated calibration errors; it did not test patient outcomes.
What physicians should demand: External validation: Source conditions materially distinct from development and relevant to intended use Temporal validation: Evaluation after the development period, with changes in care documented Prospective workflow evaluation: Real-world use, with the configured interface, version, threshold, and response pathway Comparative outcome evaluation: A relevant comparator and prespecified outcome when patient or workflow benefit is claimed Subgroup analysis: Adequately precise performance across clinically relevant groups and contexts
Famous Failures Due to Poor External Validation:
- At Michigan Medicine, an external validation of the Epic Sepsis Model reported 33% sensitivity and 12% positive predictive value at the evaluated alert threshold; the study evaluated prediction performance and alert burden, not whether using the model improved patient outcomes (Wong et al., 2021).
- Chest-radiograph models showed variable transport across hospital systems, partly reflecting differences in disease prevalence and institutional cues (Zech et al., 2018).
- Dermatology evaluations have documented representation and performance gaps, but the magnitude and direction depend on the model, dataset, task, and subgroup definition (Daneshjou et al., 2022).
Data Privacy and Governance
Governance considerations: - Whether HIPAA, another law, a research rule, a contract, or institutional policy applies to the specific data flow and purpose - De-identification method and residual linkage or inference risk in high-dimensional data - Role allocation, Business Associate Agreements where required, subcontractors, retention, deletion, and secondary use - Whether consent, authorization, notice, or another lawful basis is required for the specific activity and jurisdiction
Federated Learning: Keep raw records at participating sites while exchanging model information - Can reduce central pooling and expand multi-site participation - Does not by itself establish privacy, security, representativeness, regulatory compliance, or clinical validity; updates can still leak information without appropriate safeguards
Detailed legal and operational requirements are addressed in Privacy, HIPAA, and Patient Data Security.
Data Use Agreements: - Who controls the source records, derived data, model updates, and resulting intellectual property? - May a vendor use institutional data to improve another model or service? - What retention, deletion, portability, and model-decommissioning obligations apply when the relationship ends?
- What data was used for training?
- How many patients? From how many institutions?
- What time period? (Older data may be obsolete)
- What geographic regions and demographics?
- How representative is training data?
- Does it match the intended patient population and workflow?
- Are all relevant subgroups represented?
- What exclusion criteria were applied?
- How were labels obtained?
- Expert review? Billing codes? Chart review?
- Inter-rater reliability measured?
- What’s the label error rate?
- Was external validation performed?
- At how many independent institutions?
- In patient populations similar to mine?
- Prospectively or only retrospectively?
- How does the model handle missing data?
- Simple imputation? Advanced methods?
- Does performance degrade with missingness?
- What happens when my data differs from training data?
- Does the model detect distribution shift?
- Are there alerts for out-of-distribution inputs?
- How often will recalibration be needed?
- How will the model be updated?
- Can it learn from my institution’s data?
- Who controls updates and versioning?
- Will performance be monitored post-deployment?
Data Preprocessing Pipeline
Understanding how raw clinical data becomes AI input helps identify failure points:
- Data Extraction: Pull from EHR databases
- Timing matters (when was value entered vs. when was it measured?)
- Version control (which data definition was used?)
- Cleaning: Handle errors, outliers, unit mismatches, and implausible values
- Prespecify how implausible values are flagged, corrected, excluded, or retained
- Preserve an audit trail because an unusual value can be a true extreme, a unit error, or a device artifact
- Transformation: Convert to standard formats
- Units standardization (mg/dL vs. mmol/L)
- Code mapping (ICD-9 to ICD-10)
- Text normalization
- Feature Engineering: Create meaningful variables
- Trends (increasing/decreasing)
- Ratios (BUN/Cr)
- Time-since-last-measurement
- Handling Missing Data:
- Carry-forward (use last known value)
- Imputation (fill with mean, median, model-predicted)
- Missingness indicators (flag when data is missing)
Each step introduces assumptions that may not hold in deployment.
Monitoring Distribution Shift
Monitoring should compare current inputs, missingness, outcome prevalence, calibration, operating-point performance, subgroup performance, and workflow consequences with an accepted baseline. A drift alert is a trigger for investigation, not proof that retraining is required. The response may be continued observation, recalibration, workflow repair, restricted use, model updating, or withdrawal, depending on the source and consequence of change.
Subasri and colleagues retrospectively studied 143,049 adult inpatients across seven Toronto hospitals and simulated time-ordered deployment of a 14-day in-hospital mortality model. Their label-agnostic pipeline detected shifts associated with hospital type, admission source, demographic composition, laboratory assays, and the COVID-19 period; transfer and continual-learning strategies improved selected prediction metrics in that dataset (Subasri et al., 2025). The study supports a monitoring method in one retrospective prognostic setting, not a universal drift threshold or proof that automated updating improves patient outcomes.
Data Modalities and Their Failure Modes
Imaging Data: - Equipment variability (different manufacturers, models, protocols) - Technical factors (image quality, positioning, artifacts) - DICOM metadata (may contain hidden information AI learns spuriously) - Annotation consistency (radiologist variability in labeling)
Laboratory Data: - Reference range differences across labs - Measurement method changes over time - Result reporting variability (e.g., “undetectable” vs. “< 0.01”) - Timing of collection vs. reporting
Clinical Notes: - Documentation style varies by specialty, provider - Copy-paste creates duplicate information with different timestamps - Negation and uncertainty poorly captured - Critical information often in unstructured text
Physiologic Waveforms (ICU monitoring): - Massive data volume - Artifacts and alarms - Missing data when sensors disconnected - Quality varies with patient movement, equipment
Genomic Data: - Population stratification (ancestry-associated variants) - Batch effects (different sequencing runs) - Rare variants (limited training examples) - Ethical considerations (genetic discrimination)
Data Scale, Access, and Governance Tension
More relevant information can improve estimation and subgroup precision, but volume alone does not repair unreliable labels, selection bias, leakage, or a misaligned target. Pooling data across institutions can also raise privacy, regulatory, contractual, governance, and competitive concerns.
Solutions: - Federated learning (train without data sharing) - Synthetic data (generate artificial records whose utility and disclosure risk must be evaluated for the intended use) - Transfer learning (pre-train on large datasets, fine-tune on local data) - Multi-institutional collaborations with data use agreements
Clinical Questions and Answers
Performance can change across hospitals because patient mix, prevalence, equipment, acquisition, documentation, treatment, and workflow differ. A model may also learn site-specific shortcuts. External and local validation estimate transportability but do not guarantee clinical benefit.
What is selection bias in medical AI training data?
Who gets tested, admitted, or treated shapes training data. AI trained on academic medical center patients may fail in primary care because referral patterns concentrate certain patients.
Why do billing codes make poor AI training labels?
Billing codes are created for administrative and reimbursement purposes and may not represent the clinical construct required by a model. Their fitness as labels depends on the condition, coding process, timing, validation, and intended task.
What is distribution shift in medical AI?
Distribution shift is a change between development and use conditions, including patient mix, prevalence, equipment, documentation, treatment, or workflow. It can change discrimination, calibration, error rates, or workload, but input drift alone does not prove performance degradation.
Why does dermatology AI perform worse on dark skin?
Dermatology datasets and models can contain representation and performance gaps across skin tones and disease categories. Under-representation requires subgroup evaluation, but it does not by itself establish the direction or magnitude of error for every model.
What validation should physicians demand from AI vendors?
Evidence should match the claim: external and temporal validation assess transportability, prospective workflow studies assess use in practice, and comparative studies with prespecified patient-relevant endpoints are needed for outcome-benefit claims.
The Clinical Bottom Line:
Clinical data encode care processes as well as biology: Missingness, measurement, documentation, selection, and treatment can all become model inputs
Training data constrain the claim: Dataset size cannot repair an unreliable label, leakage, biased selection, or a misaligned target
Validation designs answer different questions: Internal, temporal, external, prospective workflow, and comparative outcome studies are not interchangeable
Distribution shift requires measurement: A new site or time period can change performance, but the direction and clinical consequence must be evaluated
Bias requires an end-to-end analysis: Examine representation, labels, target, model, threshold, workflow, access, and the intervention produced by the output
Data quality questions are essential: Ask vendors detailed questions about training data, validation, and handling of data differences
Local data matter: Acceptance testing and post-deployment monitoring are needed because the local population and data-generating process may differ from development conditions
Transparency is necessary for appraisal: Missing provenance, version, threshold, or validation detail limits a defensible adoption decision
Moving Forward:
Understanding data challenges prepares you to evaluate specialty-specific AI applications (Part II) and implementation considerations (Part III) with appropriate skepticism and rigor.
Next: Specialty chapters examine how these data constraints affect particular clinical tasks, beginning with diagnostic imaging and radiology.