Appendix N: Teaching with the Physician AI Handbook
Residency and fellowship programs, medical schools, and CME planners can teach clinical AI from the Physician AI Handbook without asking permission or paying a fee. The materials below are built for a program director with an existing didactic calendar: short session tracks, maps to the competency frameworks programs already report against, journal club questions for every chapter, case discussions, and assessment items with answer points. The full 12-week seminar, with a grading rubric and a learner AI-use policy, is the Course Syllabus Template.
These materials support local teaching. They are not an accredited course, do not award CME credit, and do not replace a program’s own curriculum review.
Who the Materials Serve
No programming background is needed. Each session assumes clinical training and about 45 minutes of reading.
| Setting | Suggested starting point | Typical slot |
|---|---|---|
| Residency didactics, any specialty | Four-session track | Noon conference or academic half-day |
| Fellowship programs | Eight-session track with the fellowship’s specialty chapter in session 7 | Fellow conference series |
| Medical school electives and longitudinal threads | Eight-session track, or the 12-week syllabus | Elective block or small groups |
| Quality, patient safety, and informatics tracks | Sessions on evaluation, safety, workflow, and governance | Track curriculum |
| Journal club | Chapter questions in the journal club guide | Monthly |
| Faculty development and CME planning | Four-session track | Grand rounds series or workshop |
Session Tracks
Each session runs 60 to 90 minutes: learners read the assigned chapter sections beforehand, a faculty lead frames the question for ten minutes, the group works through one case or one primary study cited in the chapter, and the session closes with one or two assessment items. Competency columns name where a session gives learners practice; the program decides whether that practice counts toward its own assessment.
Four-Session Track
| Session | Question for the session | Read before | Case for discussion | ACGME competency | AAMC AI domain |
|---|---|---|---|---|---|
| 1 | What does a performance claim show, and what does it leave open? | AI Fundamentals for Clinicians; Evaluating AI Clinical Decision Support Systems | Epic Sepsis Model External Validation | Practice-Based Learning and Improvement (4.7.f) | I. AI Knowledge for Practice |
| 2 | Where do harm and inequity enter a clinical AI system? | Clinical AI Safety and Risk Management; Medical Ethics, Bias, and Health Equity | COVID-19 Imaging Models and Methodological Failure | Systems-Based Practice (4.9.d); Professionalism (4.3.e) | IV. Professionalism and Ethics; VI. Systems-Based AI Practice |
| 3 | What does the physician owe the patient when a language model drafts, summarizes, or advises? | Large Language Models in Clinical Practice; AI-Assisted Clinical Documentation | ChatGPT Health Emergency Triage in Simulation | Interpersonal and Communication Skills (4.8.f); Practice-Based Learning and Improvement (4.7.f) | II. AI-Assisted Patient Care and Clinical Reasoning; III. Communicating About AI |
| 4 | Who decides whether a tool is used here, and who answers for it? | Integration into Clinical Workflow; Physician AI Liability and Regulatory Compliance; Clinical AI Policy and Governance | Viz.ai Stroke Triage and Notification | Systems-Based Practice (4.9.a, 4.9.e) | VI. Systems-Based AI Practice |
Eight-Session Track
Twelve Sessions or a Semester
The Course Syllabus Template sets out a 12-week seminar with weekly readings, discussion questions, assignments, a grading rubric, a learner AI-use policy, and a 6-week intensive version. Programs that start with the four- or eight-session track can move to it without changing readings.
Competency Maps
ACGME Common Program Requirements
The ACGME Common Program Requirements (Residency), effective July 1, 2026, require every program to integrate all six ACGME Competencies into its curriculum, and state that each specialty’s Milestones describe the developmental path within them (ACGME, 2026). Clinical AI fits most naturally in four of the six. The numbers in the table are the subcompetency numbers in that document; fellowship programs use the parallel fellowship version.
| ACGME competency and subcompetency | Where the handbook gives learners practice |
|---|---|
| Practice-Based Learning and Improvement, 4.7.f: locating, appraising, and assimilating evidence from scientific studies | Evaluation chapter; journal club guide; every specialty chapter’s evidence tables |
| Practice-Based Learning and Improvement, 4.7.d: quality improvement, including activities aimed at reducing health care disparities | Safety chapter (monitoring and incident review); ethics chapter (subgroup performance) |
| Systems-Based Practice, 4.9.d: identifying system errors and implementing potential systems solutions | Safety and workflow chapters; case discussions |
| Systems-Based Practice, 4.9.e: value, cost awareness, delivery and payment, and risk-benefit analysis | Policy and governance chapter; Clinical AI Vendor Evaluation Toolkit |
| Systems-Based Practice, 4.9.g: tools and techniques that promote patient safety and disclosure of patient safety events | Safety chapter; liability chapter |
| Interpersonal and Communication Skills, 4.8.f: comprehensive, timely, and legible health care records | Documentation chapter |
| Interpersonal and Communication Skills, 4.8.d: educating patients, families, students, and other professionals | Misinformation chapter; language model chapter |
| Professionalism, 4.3.d and 4.3.h: respect for patient privacy and autonomy; disclosing and addressing conflict or duality of interest | Privacy chapter; ethics chapter; Industry Reports & Benchmarks |
Specialty Milestones set the levels a program assesses. The handbook supplies the material; the Clinical Competency Committee decides what a learner has shown.
AAMC AI Competencies
The AAMC published AI Competencies Across the Learning Continuum on October 27, 2026. It sets out six domains, each with competencies at three developmental tiers (Novice, Proficient, and Expert), and the AAMC states that the tiers do not correspond to a specific role such as student, resident, or faculty member (AAMC, 2026).
| AAMC domain | Chapters that address it |
|---|---|
| I. AI Knowledge for Practice | AI Fundamentals for Clinicians; The Clinical Data Challenge; Evaluating AI Clinical Decision Support Systems |
| II. AI-Assisted Patient Care and Clinical Reasoning | Large Language Models in Clinical Practice; the specialty chapters in Part II |
| III. Communicating About AI and Interprofessional Collaboration | AI-Assisted Clinical Documentation; Medical Misinformation and AI; Medical Ethics, Bias, and Health Equity (disclosure and consent) |
| IV. Professionalism and Ethics | Medical Ethics, Bias, and Health Equity; Clinical Research with AI |
| V. Practice-Based AI Learning and Improvement | AI in Medical Education: Curriculum, Competency, and the Future Physician; The Physician-AI Partnership: Future Perspectives |
| VI. Systems-Based AI Practice | Clinical AI Safety and Risk Management; Privacy, HIPAA, and Patient Data Security; Integration into Clinical Workflow; Clinical AI Policy and Governance |
Most sessions in the four- and eight-session tracks fit the Novice and Proficient tiers. Sessions where learners teach a peer, lead a journal club, or draft a departmental AI policy give practice at the Expert tier.
Journal Club Guide
The chapter is the briefing; the paper is the evidence. Choose one primary study cited in the chapter, assign it with the chapter, and give the group 60 minutes in three parts: what the study claims (10 minutes), whether its design supports that claim (30 minutes), and whether the result would hold in the group’s own hospital and patients (20 minutes). The chapter questions further down are written for that last part and can be used with any study the chapter cites.
Starter Papers
Six studies the handbook discusses in detail make good first sessions. Each is read with its chapter, and the question gives the middle 30 minutes a focus.
| Study | Read with | Design question |
|---|---|---|
| Wong et al., 2021, JAMA Internal Medicine: external validation of a proprietary sepsis prediction model | Critical Care and Pulmonary Medicine | What does an independent external validation measure that a developer’s own validation cannot? What alert volume would the reported performance produce on your units? |
| Obermeyer et al., 2019, Science: racial bias in a widely used population-health algorithm | Medical Ethics, Bias, and Health Equity | The algorithm predicted spending rather than need. Which outcome labels in your institution’s models stand in for clinical need, and how would you test them for the same failure? |
| Goh et al., 2024, JAMA Network Open: randomized trial of physician diagnostic reasoning with and without GPT-4 access | Large Language Models in Clinical Practice | Model access did not significantly improve diagnostic-reasoning scores. Which features of the trial (cases, training, interface, outcome measure) could explain that, and what would a trial measuring patient outcomes need? |
| Lång et al., 2023, Lancet Oncology: the MASAI randomized trial of AI-supported mammography screening | Diagnostic Imaging, Radiology, and Nuclear Medicine | Which outcomes did the trial report, and why do interval cancers matter for judging screening benefit? Why does the software version matter when a program adopts the result? |
| Abràmoff et al., 2018, npj Digital Medicine: pivotal study of autonomous diabetic retinopathy detection in primary care | Primary Care, Family Medicine, and Preventive Medicine | The study was a prospective pivotal diagnostic study, not a randomized trial. What did it establish for authorization, and what does it leave open about patient outcomes in your clinic? |
| Lukac et al., 2025, NEJM AI: randomized trial of two ambient documentation products in outpatient practice | AI-Assisted Clinical Documentation | The two products differed in their effect on time in notes. What does a product-level difference mean for a health system choosing a vendor, and which outcomes beyond time should it measure? |
Questions by Chapter
| Chapter | Discussion questions |
|---|---|
| History of AI in Medicine: From MYCIN to Foundation Models | Which barriers that kept MYCIN out of routine care still apply to current tools? What evidence would show that a new model is different from earlier waves rather than a repeat? |
| AI Fundamentals for Clinicians | For a model in the study, what was the label, and who produced it? Which evidence-based medicine concept (sensitivity, prevalence, spectrum) changes most when the model leaves its training setting? |
| The Clinical Data Challenge | Which variables in the dataset reflect clinician behavior (ordering, documenting, admitting) rather than patient physiology? How would the model behave if those practices changed at your site? |
| Industry Reports & Benchmarks | What exactly did the report count as adoption? Who funded it, and how would that shape the question it asked? |
| Diagnostic Imaging, Radiology, and Nuclear Medicine | Was performance measured on curated retrospective images or in live reading? What would change about the radiologist’s work if the tool were wrong one time in twenty? |
| Internal Medicine and Hospital Medicine | Who receives the alert, and can that person act on it at that moment? How would alert volume on a typical ward change the result? |
| Surgery, Anesthesiology, and Perioperative Care | Did the study measure a decision that changed, or only a prediction that was accurate? What evidence would justify using the model to alter an operative plan? |
| Pediatrics and Neonatology | Was the model developed in children, or transferred from adults? Is the tool machine learning at all, or a conventional risk model or automated control? |
| Obstetrics and Gynecology | Did the study measure maternal and neonatal outcomes, or only prediction accuracy? How were patient choice and subgroup performance reported? |
| Emergency Medicine | Did faster process times translate into better functional outcomes? How does time pressure change what verification is feasible? |
| Critical Care and Pulmonary Medicine | What happens to the patient after a false negative and after a false positive in this unit? Did the trial’s primary endpoint matter to patients? |
| Hematology-Oncology and Precision Medicine | Is a detection gain being presented as a survival benefit? Which cancer, stage, and population does the result actually cover? |
| Cardiology and Cardiothoracic Surgery | Was the model externally validated, and in which patients? How would the output change management for one patient on your service? |
| Neurology and Neurological Surgery | Is the task an acute imaging pattern or a complex diagnosis that depends on the examination? What would the neurologist still have to do that the model cannot? |
| Psychiatry and Behavioral Health | What did the model predict, for whom, over what period, and what action followed? Is there evidence that using it prevents harm, or only that it ranks risk? |
| Primary Care, Family Medicine, and Preventive Medicine | Which task does the tool remove from the visit, and which does it add? How would the result differ in a clinic with a different patient mix? |
| Pathology and Laboratory Medicine | How was ground truth established, and by how many pathologists? Would the result survive a different scanner or staining protocol? |
| Dermatology | How was skin tone measured and reported, and is Fitzpatrick type an adequate proxy? What happens to a patient whose app result delays an examination of a concerning lesion? |
| Ophthalmology | What does the FDA authorization specify about camera, population, and workflow? What must be true in your clinic for the pivotal study’s performance to apply? |
| Orthopedic Surgery and Physical Medicine | Does the evidence come from a reader study, prospective workflow data, or vendor reports? Which outcome would a patient care about that the study did not measure? |
| Infectious Diseases and Antimicrobial Stewardship | Is the tool machine learning, or established laboratory automation or genomics? Which stewardship measure improved, and is it a process measure or a patient outcome? |
| Allergy, Immunology, and Medical Genetics | How do rare conditions and small datasets limit what the study can show? For which ancestries does the evidence apply? |
| Surgical Subspecialties | Did an improvement in detection rate translate into fewer cancers or deaths? Why might a trial in community practice reach a different result from pooled trial data? |
| Evaluating AI Clinical Decision Support Systems | Which level of evidence (technical, transportability, live performance, clinical utility, lifecycle safety) does the study reach? What local test would the group require before go-live? |
| Medical Ethics, Bias, and Health Equity | Which proxy, threshold, or access pattern could produce unequal harm? What would a reasonable patient want disclosed about the tool’s role in their care? |
| Privacy, HIPAA, and Patient Data Security | Draw the data flow: who receives patient data, under what agreement, and for how long? Would de-identification as described prevent re-identification in your setting? |
| Clinical AI Safety and Risk Management | How could the configured human review step fail? Who would detect performance drift after a software update, and how quickly? |
| Integration into Clinical Workflow | Does the output reach the right person at the right time in a usable format? Which step does the tool add, and who absorbs the extra work? |
| Physician AI Liability and Regulatory Compliance | In your jurisdiction, what standard would apply to a physician who followed, or overrode, the output? Which facts about product version and intended use would matter in a review? |
| AI Tools Every Physician Should Know | Does the published evidence match the product version, population, comparator, and endpoint being sold? Which selection criterion would the group weigh most, and why? |
| Large Language Models in Clinical Practice | Which output in the study would a clinician have been unable to catch without checking a primary source? How should a team decide when a language model may be used at all? |
| AI-Assisted Clinical Documentation | Did the trial measure documentation time, burnout, note quality, or patient outcomes? What must a physician verify before signing an ambient-scribe draft? |
| Clinical Research with AI | Where in the review or analysis did AI make or suggest a decision, and was its error rate measured? What disclosure would a journal require? |
| Clinical Trials and AI | Is AI the intervention under test or a tool for running the trial? Were the endpoints and estimands specified before the trial began? |
| Emerging AI Technologies in Healthcare | Is the result a benchmark score or a clinical outcome? What evidence would move the technology from demonstrated in bounded settings to ready for routine use? |
| AI and Global Health Equity | What would the service need beyond the model (power, connectivity, referral capacity, payment)? Who bears the cost of errors, and who decides when the pilot ends? |
| Clinical AI Policy and Governance | Which body in your hospital would approve, monitor, and retire the tool? How do the regulatory pathway and the hospital’s own review differ in what they check? |
| Medical Misinformation and AI | How would you respond to a patient who brings a chatbot answer that conflicts with your advice? Which source checks would you model for the patient? |
| AI in Medical Education: Curriculum, Competency, and the Future Physician | How could AI use in training weaken the skills it is meant to support? What would the group assess to detect deskilling or never-skilling? |
| The Physician-AI Partnership: Future Perspectives | Which clinical tasks should remain with physicians regardless of model performance, and why? What evidence would change that list? |
Case Discussions
Case Studies in Medical AI holds full cases with background, implementation, outcomes, and lessons. A role-based format works well: assign learners the roles of attending physician, chief medical information officer, nurse manager, patient safety officer, and patient advocate, and have each argue from that role before the group reaches a decision.
| Case | Prompts |
|---|---|
| IDx-DR and Diabetic Retinopathy | What made autonomous use acceptable for this task when it is not acceptable for most tasks? Which conditions of the authorization would a clinic most easily break? |
| Viz.ai Stroke Triage and Notification | What did the evidence show about process times, and what did it leave open about patient outcomes? Who in your hospital would own the notification pathway? |
| Epic Sepsis Model External Validation | Why did a widely deployed model perform differently on external evaluation? What local testing would have revealed the gap before go-live? |
| Watson for Oncology and Variable Concordance | What does concordance with an oncologist measure, and what does it miss? How should a hospital weigh a vendor’s reputation against published evidence? |
| COVID-19 Imaging Models and Methodological Failure | Which methodological problems would a reviewer catch from the methods section alone? What does the case suggest about evaluating models built quickly during an emergency? |
| Utah AI Prescription-Renewal Pilot | Which renewals should never be automated, and who decides? What monitoring would the group require, and what result would stop the pilot? |
Assessment Questions
The items below suit short-answer quizzes, oral examination prompts, or the closing minutes of a session. Answer points are collapsed so the page can be projected.
- A vendor states that its sepsis tool is FDA-cleared. What does that establish, and what does it leave unanswered for your hospital?
- A deterioration model reports good discrimination at the developer’s site. Name three reasons its performance could change at your site.
- Explain why a model can rank patients well and still give misleading risk estimates at the threshold used for action.
- A risk score was trained to predict health care spending as a stand-in for clinical need. How can that choice produce biased results even with a representative dataset?
- A clinician reviews every output of an AI system before it reaches the patient. Why is that review not an automatic safety control?
- An ambient scribe produces a draft note that includes an examination finding the physician did not elicit. Who is responsible for the note, and what should happen next?
- A language model gives a fluent, referenced answer to a clinical question. What steps should precede acting on it?
- After a software update, a tool’s alert rate doubles. Which monitoring and governance steps should already be in place?
- List the facts that determine how liability would be analyzed after a patient is harmed while an AI tool was in use.
- A tool performs well in a pilot but is rarely used after rollout. Name the workflow failures most likely to explain it.
- Authorization defines a regulatory status and an intended use. It does not show local safety, performance in your population and workflow, or patient-outcome benefit (Evaluating AI Clinical Decision Support Systems).
- Different patient mix and prevalence, different equipment and data pipelines, different documentation and ordering practices, and changes over time in practice or software (The Clinical Data Challenge).
- Discrimination measures ranking; calibration measures whether predicted risks match observed risks. A well-ranking model can be miscalibrated, so decisions made at a fixed threshold can be wrong (Evaluating AI Clinical Decision Support Systems).
- The label itself encodes unequal access: patients who receive less care generate less spending at the same level of illness, so the model learns to rate them as healthier (Medical Ethics, Bias, and Health Equity).
- The human-AI workflow, including automation bias, time pressure, and how the output is displayed, determines whether errors are caught; the configured workflow itself needs evaluation (Clinical AI Safety and Risk Management).
- The signing physician is accountable for the record. The finding is removed or corrected before signing, and the error is reported through the institution’s process so the tool can be monitored (AI-Assisted Clinical Documentation).
- Check that each cited source exists and says what the answer claims, compare the answer with current guidance, and weigh the patient’s context; fluency is not evidence (Large Language Models in Clinical Practice).
- Version and change control, defined performance and alert-rate thresholds, an owner who reviews monitoring data, an incident-response route, and authority to pause the tool (Clinical AI Safety and Risk Management; Clinical AI Policy and Governance).
- Jurisdiction, the parties involved, the product and its version, the intended use, the workflow as configured, the evidence available at the time, and the harm alleged (Physician AI Liability and Regulatory Compliance).
- Output arriving too late, reaching someone who cannot act on it, appearing in a format that is easy to miss, adding steps to an already full workflow, or raising alert burden (Integration into Clinical Workflow).
Citing the Version Used in a Course
The handbook is revised continuously, so a syllabus should record the date on which learners read it. The Changelog lists what changed after that date.
Tegomoh B. The Physician AI Handbook: Peer-Reviewed Evidence for Every Specialty. 2025. Accessed [date]. doi:10.5281/zenodo.18251405
Chapter-level citations and other formats (APA, Vancouver, Chicago, BibTeX) are on the How to Cite page.
Reuse and Adaptation
The handbook text is licensed under the Creative Commons Attribution 4.0 International License (Creative Commons, CC BY 4.0). Programs may copy, excerpt, translate, and adapt it into slides, course packs, learning management systems, and assessments, including for courses that charge tuition, provided they give credit, link to the license, and note any changes. An attribution line can read:
Adapted from The Physician AI Handbook by Bryan Tegomoh, MD, MPH (physicianaihandbook.com), licensed under CC BY 4.0. Changes: [describe].
Studies, figures, and quotations from other sources keep their own licenses; cite them from the original.
Contact
Using this handbook in a course or program? Tell us. A line on the program, the sessions used, and what did not work helps decide what to revise next.