Privacy, HIPAA, and Patient Data Security

A vendor’s willingness to sign a business associate agreement does not establish that its AI use is lawful, secure, or limited to the intended purpose. Governance must connect legal role, permitted use, data flow, technical safeguards, retention, model-training terms, subcontractors, and incident response. Clinical AI can expose data through prompts, recordings, embeddings, vector stores, model-training corpora, telemetry, human-review queues, cached output, support tickets, and downstream integrations.

Learning Objectives

After completing this chapter, clinicians should be able to:

  • Determine when an AI vendor may be a HIPAA business associate
  • Distinguish a BAA from authorization for every proposed data use
  • Evaluate de-identification, re-identification risk, and privacy-enhancing technologies
  • Map clinical AI data flows and vendor subprocessors
  • Identify relevant GDPR, FTC, state-law, and ONC requirements
  • Establish risk-based security, retention, and incident-response controls

Medical AI Creates New Privacy Challenges:

Some medical AI development uses large clinical datasets, while other tools use smaller, synthetic, federated, public, or nonclinical data. The privacy analysis should follow the actual system: - Data scale: The volume and richness of records can increase exposure, but no universal “millions” requirement applies - Data aggregation: Combining data from multiple sources increases re-identification risk - Data persistence: Models can memorize training examples under some conditions, but extraction risk depends on the model, data, training, access, and defenses - Third-party access: Cloud-based processing can extend the data flow to vendors and subprocessors without eliminating institutional responsibility

HIPAA Compliance is Necessary But Insufficient:

HIPAA establishes baseline requirements: - BAAs are required for qualifying business-associate relationships, not for every vendor or every health-data use - De-identification standards apply (Safe Harbor or Expert Determination) - Minimum necessary principle: limit data access to what’s needed - Patient rights to access, amend, and restrict use of their data

But HIPAA has limitations: - Written before modern AI: doesn’t address many AI-specific risks - De-identification isn’t foolproof (re-identification possible with external data) - Does not cover every health-data setting; applicability follows the parties, data status, and activity rather than physical location alone - Enforcement gaps: violations often go undetected (Price & Cohen, 2019)

Key Privacy Risks in Medical AI:

  1. Re-identification Attacks:
    • Properly de-identified datasets retain some identification risk, and linkage risk depends on context
    • Geographic, demographic, and clinical data combinations are often unique
    • Risk increases as more “anonymized” datasets become available (Sweeney, 2015)
  2. Model Inversion Attacks:
    • Tested attacks can sometimes infer or reconstruct information from model access
    • Particularly concerning for rare diseases or unique patient characteristics
    • Vulnerability depends on architecture, overfitting, access, data, and defenses rather than a universal model-family rule
  3. Membership Inference Attacks:
    • Determine whether specific individual’s data was used in model training
    • Can reveal sensitive information (e.g., person was in HIV study dataset)
    • Documented in medical AI models (Shokri et al., 2017)
  4. Data Breaches:
    • Clinical data and AI infrastructure are high-value security targets
    • AI vendors may have inadequate security practices
    • Cloud infrastructure creates additional attack surface
  5. Secondary Use Without Consent:
    • Data collected for care may be proposed for model development or product improvement under a different purpose
    • Commercial reuse must be evaluated under the actual legal roles, contracts, patient expectations, and applicable law
    • Lack of transparency about data use

Best Practices for Privacy Protection:

Before Deploying AI: - Conduct privacy impact assessment - Verify legal roles, BAA requirements, contracts, deployment configuration, security evidence, and data flows - Review data governance and use agreements - Determine which notice, consent, authorization, disclosure, or research requirements apply

During Use: - Minimize data collection (only necessary elements) - Implement access controls and audit logging - Encrypt data in transit and at rest - Monitor for unauthorized access or breaches

Emerging Privacy-Preserving Techniques: - Federated learning: Train AI without centralizing patient data - Differential privacy: Add mathematical noise to protect individuals while preserving population-level patterns - Synthetic data: Generate artificial datasets that preserve statistical properties without containing real patient data - Homomorphic encryption: Perform computations on encrypted data

Clinical Bottom Line:

Privacy is both a regulatory and professional obligation. Medical AI governance should use lawful purposes, proportionate controls, accurate transparency, and applicable patient rights rather than promise control that the governing law or clinical record system does not provide (Price and Cohen, 2019; Vayena and Blasimme, 2018).

Do not send protected health information to an AI service until its legal role, contract, data flow, and safeguards are verified.

Introduction

Medical encounters generate diagnoses, medications, laboratory results, images, recordings, and notes. These data can reveal health, genetics, behavior, identity, relationships, and vulnerability.

Clinical information has never been confined to one room or database: insurers, laboratories, pharmacies, consultants, public-health authorities, researchers, and vendors may all participate under different legal roles. AI adds prompts, embeddings, model training, telemetry, and human-review queues to that existing network.

AI systems differ materially. Some are trained on large multi-institutional datasets; others use fixed models, local data, federated training, retrieval, or no patient-specific training at all. Cloud processing may extend control to a vendor and its subprocessors. Model memorization and extraction are possible under tested conditions, but are not inevitable properties of every model.

These are not merely theoretical categories. Re-identification has been demonstrated under particular datasets and linkage conditions (Sweeney, 2015). HHS and FTC enforcement records document health-data breaches and unauthorized disclosures, while the magnitude and cause must be taken from each primary record. Privacy-invasive practices can also impose unequal burdens on populations with less power to contest collection, inference, or reuse (Price and Cohen, 2019).

HIPAA: The Regulatory Foundation

The HIPAA Rules apply to covered entities and business associates, not to every organization that handles health-related data. HHS explains that an entity outside those categories is not subject to HIPAA merely because its data concern health (HHS, Covered Entities and Business Associates).

What HIPAA Requires

1. Privacy Rule: - Governs use and disclosure of PHI - Permits and conditions specified uses and disclosures, including treatment, payment, and health care operations - Provides access, amendment, accounting, restriction, confidential-communication, and other rights with important scope and exceptions

2. Security Rule: - Requires administrative, physical, and technical safeguards for electronic PHI (ePHI) - Risk assessments mandatory - Uses required and addressable implementation specifications; “addressable” requires a documented reasonableness analysis rather than being optional - Requires applicable access, audit, integrity, authentication, transmission-security, risk-analysis, and risk-management controls

3. Breach Notification Rule: - Covered entities must notify patients of breaches of unsecured PHI - HHS and, for specified breaches, media notification have different timing and scope rules, including a 500-person threshold - Business associates have notice obligations to covered entities under the Rule and contract

4. Business Associate Agreements (BAAs): - Required when a covered entity or business associate engages another business associate to create, receive, maintain, or transmit PHI on its behalf - BAAs specify permitted uses, safeguard requirements, breach notification, and liability - A vendor is not automatically a business associate merely because it sells software. The role depends on the service and whether it handles PHI on behalf of a regulated entity (HHS software-vendor guidance)

HHS states that a cloud service provider maintaining encrypted ePHI is still a business associate even when it lacks the encryption key (HHS cloud-computing guidance). A BAA is necessary in defined relationships, but it is not a blanket permission slip for model training, product improvement, cross-customer learning, or commercialization.

HIPAA and De-Identification

HIPAA allows use of de-identified data without patient authorization. Two de-identification methods:

1. Safe Harbor Method: Remove 18 categories of identifiers: - Names, geographic subdivisions smaller than state (except first 3 digits of ZIP if >20,000 people) - Dates (except year; ages >89 aggregated) - Phone numbers, fax numbers, email addresses, SSNs, medical record numbers - Account numbers, certificate/license numbers, vehicle identifiers - Device identifiers, web URLs, IP addresses, biometric identifiers - Full-face photos, other unique identifying characteristics

2. Expert Determination: - A person with appropriate knowledge and experience determines and documents that identification risk is very small for the anticipated recipient and context - More flexible than Safe Harbor (can retain some identifiers if low risk) - Requires expertise and documentation

CRITICAL LIMITATION: HHS states that properly de-identified data retain some identification risk. Risk depends on the remaining data, reasonably available external information, anticipated recipient, access, and context (HHS de-identification guidance). Historic re-identification demonstrations illustrate possible linkage attacks, not a universal current probability for every dataset.

HIPAA’s Limitations for AI

HIPAA was written before modern AI existed. Key gaps:

1. Doesn’t Address AI-Specific Risks: - Model memorization of training data - Inference attacks (extracting information from model outputs) - Re-identification through AI-enabled linking

2. Limited Scope: - Applies to covered entities, business associates, and PHI within the Rules’ scope - Applicability does not end merely because data move to a vendor, research environment, or commercial service; legal role, data status, disclosure, and activity matter - Many consumer apps and wearables operate outside HIPAA, but coverage must be determined rather than assumed

3. Enforcement and Remedies: - HHS OCR investigates complaints and breaches and can impose corrective action and civil monetary penalties - HIPAA itself does not create a federal private right of action, but the same facts may implicate state law, contract, consumer protection, professional duties, or other claims - Detection and deterrence cannot be reduced to an unsupported comparison between penalties and company revenue

4. De-identification Boundaries: - Properly de-identified information is not PHI under the Privacy Rule - That does not eliminate contractual, ethical, re-identification, state-law, genomic, or group-privacy concerns

Proposed Security Rule update: HHS published a notice of proposed rulemaking in January 2025 to strengthen the Security Rule. It remains a proposed rule as of this review, not a final rule. The proposal would strengthen written asset inventory, risk analysis, encryption, segmentation, compliance verification, and other controls (Federal Register NPRM, January 2025). Do not present a proposed HIPAA requirement, an anticipated finalization date, or an uncited model-training interpretation as current law.

No defensible universal statement says that AI model training always requires individual authorization or that a BAA always permits it. Determine whose activity the training serves, whether data remain PHI, whether the use fits the BAA and Privacy Rule, and whether research, state law, 42 CFR Part 2, or other requirements apply.

Bottom Line: HIPAA compliance is necessary but not sufficient for protecting patient privacy in medical AI (Price & Cohen, 2019).

Privacy Risks Unique to Medical AI

Medical AI can amplify familiar privacy and security risks and add model-specific attack surfaces. The magnitude depends on the data, architecture, access, training, deployment, and controls.

1. Re-Identification Attacks

The Problem: “Anonymous” medical datasets can often be re-identified by linking to other datasets or public information.

Classic Example: Latanya Sweeney’s Research (Sweeney, 2000): - Massachusetts released “anonymized” hospital discharge data (removed names, addresses) - Sweeney re-identified Governor William Weld’s records by linking to voter registration database - Used ZIP code + birth date + sex (only 6 people shared these characteristics in his ZIP)

Why It Matters for AI: - AI training datasets often contain rich clinical detail (diagnoses, procedures, medications, lab values) - Even without direct identifiers, these clinical patterns can be unique - As more datasets become available, re-identification risk increases (more linkage opportunities)

Research Boundary: - Sweeney’s historical estimate concerned uniqueness in a population dataset under particular fields and assumptions. It is not a current probability that 87% of every clinical dataset can be re-identified. - Detailed clinical records may create linkage opportunities, but identification risk must be assessed for the actual dataset, anticipated recipient, available external data, and controls. - Commercial and public data can expand linkage opportunities; their availability and relevance must be evaluated rather than assumed.

Mitigation: - Differential privacy (add mathematical noise to data) - Limit granularity (age ranges instead of exact birth dates, regional instead of ZIP codes) - Restrict access to de-identified datasets (not publicly released) - Monitor for linkage attempts

2. Model Inversion and Membership Inference Attacks

Model Inversion: - An adversary attempts to infer properties or reconstruct information associated with training data through model access - Example: Given AI model predicting disease from genomic data, infer genome sequences of training set patients - Vulnerability varies by model, training, overfitting, access, data, and defenses

Membership Inference: - Adversary determines whether specific individual was in training dataset - Example: Query AI model repeatedly to determine if particular patient’s record was used in training - Can reveal sensitive information (person had HIV, mental health condition, etc.)

Documented Attack Class: Shokri and colleagues demonstrated black-box membership-inference attacks under tested model, dataset, and access conditions (Shokri et al., 2017). That study demonstrates feasibility, not a universal attack-success rate for clinical AI. Overfitting can increase risk, but sample size alone does not determine vulnerability.

Foundation Model Risks (Tonekaboni et al., 2025, preprint): - Healthcare foundation models trained on EHRs show similar memorization patterns - Vulnerable subgroups face elevated extraction risk; leaks involving HIV status or substance use carry higher harm than demographic attributes - Researchers recommend structured testing before model release to evaluate leakage risk

Why This Matters: - AI models themselves become privacy risks (even if training data is secured) - Releasing model parameters (e.g., for transparency or reproducibility) can leak patient information - Cloud-based AI creates additional exposure (vendor has model parameters)

Mitigation: - Differential privacy during training (add noise to gradients) - Limit model complexity (prevents overfitting/memorization) - Restrict access to model parameters - Audit models for memorization before deployment

Current Best Practice: No single technique is sufficient. The field currently recommends combining:

  1. Differential privacy during training (accept some performance reduction for formal guarantees)
  2. Pre-release evaluation (structured testing like Tonekaboni’s framework to measure actual risk)
  3. Access controls (API-only access, no model weights released)

Differential privacy provides a defined mathematical guarantee under a specified mechanism and privacy budget. It is not the only mathematically grounded privacy-enhancing method, and it does not solve purpose limitation, access, identity, endpoint security, or clinical validity. Evaluation should test the actual threat model before deployment.

3. Data Breaches

Sector Risk: - Health data and clinical operations are high-value targets because records are sensitive and care disruption can create immediate harm. - IBM’s 2025 industry report estimates an average breach cost for healthcare under its own sample and methodology. It is vendor-reported evidence, not a peer-reviewed census of all breaches (IBM, 2025). - Claims about a fixed dark-web price multiple for medical records are unstable and source-dependent, so the chapter does not use one as a planning metric.

AI-Specific Breach Risks: - Cloud infrastructure: Patient data stored by third-party AI vendors - Data aggregation: Centralized datasets create high-value targets - Supply chain vulnerabilities: Multiple parties (AI vendor, cloud provider, subcontractors) create attack surface - Insider threats: Employees with access to large datasets may misuse or sell data

Major Healthcare Breaches: - Anthem (2015): 78.8 million records - Premera Blue Cross (2015): 11 million records - UCLA Health (2015): 4.5 million records - Hundreds of smaller breaches annually

Consequences: - Identity theft, fraud, medical identity theft - Embarrassment, stigma (sensitive diagnoses exposed) - Discrimination (insurance, employment) - Loss of trust in healthcare system

Mitigation: - Encryption (data at rest and in transit) - Access controls (least privilege, multi-factor authentication) - Security audits and penetration testing - Incident response plans - Vendor risk assessment (verify AI vendors have robust security)

5. Algorithmic Inference of Sensitive Information

The Problem: AI can infer sensitive information patients haven’t disclosed.

Examples: - AI predicts sexual orientation, political views, personality traits from social media and digital traces - Medical AI could infer genetic risks, psychiatric diagnoses, substance use from indirect signals - Insurance companies could use AI to infer health risks from consumer data (shopping patterns, web searches, social media)

Why This Matters: - Patients may not realize sensitive information can be inferred - Can’t consent to disclosure of information they didn’t know they were revealing - Creates new forms of discrimination (denied insurance, employment based on inferred risks)

Example: Pregnancy Prediction: - Target’s AI predicted pregnancy from shopping patterns, sent maternity ads to teenage girl - Father complained to Target, later discovered daughter was pregnant - AI revealed sensitive information before patient had disclosed it

Medical Applications: - AI could infer HIV status from prescription patterns, doctor visits, lab tests - Could infer mental health conditions from activity patterns, communication metadata - Could infer genetic disease risk from relatives’ medical data

Ethical Concern: Right to “informational privacy”: not just protecting data you’ve shared, but also inferences drawn from that data.

Mitigation: - Transparency about what AI can infer - Patient control over data and its uses - Prohibitions on certain inferences (e.g., ban on using consumer data to infer health risks for insurance) - Regulations limiting use of inferred data for discrimination

Data Governance for Medical AI

Robust data governance is essential for protecting privacy while enabling beneficial AI.

Core Principles

1. Data Minimization: - Collect only data necessary for specific AI purpose - Don’t create large, multi-purpose datasets “just in case” - Define retention and deletion under applicable clinical-record, legal-hold, research, contract, backup, and operational requirements

2. Purpose Limitation: - Specify purpose for data collection - Do not repurpose data without an applicable legal basis, contractual authority, governance review, and any required consent or authorization - Restrict AI vendors from using data for other purposes (e.g., training models for other clients)

3. Transparency: - Inform patients about data use for AI - Make data practices discoverable (privacy policies in plain language) - Document data flows (where data goes, who has access)

4. Individual Control: - Support rights to access, amend or correct, request restrictions, object, erase, or opt out where the applicable law and system provide them - Do not promise deletion or opt-out rights that conflict with required record retention, research rules, legal holds, or statutory exceptions

5. Accountability: - Assign responsibility for data protection - Audit data practices regularly - Enforce consequences for violations

Practical Data Governance Framework

Essential Components of AI Data Governance

1. Data Inventory: - Catalog all patient data used for AI (sources, types, sensitivity levels) - Map data flows (where data moves from initial collection to AI use) - Identify all parties with access (internal teams, vendors, cloud providers)

2. Privacy Impact Assessment: - For each AI project, assess privacy risks - Consider: What data is collected? Who has access? What are re-identification risks? What harms could result from breach or misuse? - Identify mitigation strategies - Document assessment and decisions

3. Access Controls: - Least privilege (users only access data necessary for their role) - Role-based access control (RBAC) - Multi-factor authentication for sensitive data access - Audit logging (who accessed what data, when) - Regular access reviews (remove unnecessary permissions)

4. Data Use Agreements: - Formalize permitted uses, restrictions, and safeguards - Require a BAA when the legal relationship makes the vendor a business associate; use other agreements when different roles apply - Treat certifications as evidence with a defined scope and date, not as proof of legal compliance - Prohibit unauthorized secondary use, data sales, or sharing - Include breach notification requirements

5. Patient Consent Management: - Determine whether consent, HIPAA authorization, notice, waiver, IRB pathway, state-law permission, or another basis applies to each use - Offer granular choices when required or operationally promised, while stating any limits - Easy-to-understand language (avoid legalese) - Document consent and respect patient choices

6. Security Safeguards: - Encryption (data at rest and in transit) - Secure data storage (access-controlled, physically secured servers) - Network security (firewalls, intrusion detection) - Regular security testing (penetration tests, vulnerability scans) - Incident response plan (what to do if breach occurs)

7. Vendor Management: - Due diligence before selecting an AI vendor, including legal role, data use, security evidence, architecture, subprocessors, incidents, and financial capacity - Contractual protections (BAAs, data use restrictions, audit rights) - Ongoing monitoring at a risk-based cadence and after material changes; no universal annual interval fits every control - Exit strategy (data return or destruction when contract ends)

8. Training and Culture: - Educate staff about privacy obligations - Create culture of privacy-consciousness (not just compliance checkbox) - Empower employees to raise concerns about privacy risks

9. Ongoing Monitoring: - Regular audits of data access and use - Monitor for anomalous access patterns (insider threats) - Track AI model performance drift (could indicate data issues) - Review privacy practices on a defined cadence and after material changes, incidents, or regulatory updates

10. Accountability and Enforcement: - Assign ownership (Chief Privacy Officer, Data Governance Committee) - Consequences for violations (personnel action, contractual penalties for vendors) - Continuous improvement (learn from incidents, update practices)

Evaluating AI Vendor Privacy and Security

Healthcare organizations must carefully assess AI vendors before granting access to patient data.

Key Questions for AI Vendors

Due Diligence Checklist for AI Vendors

HIPAA and Regulatory Compliance:

  • Will vendor sign a Business Associate Agreement (BAA)?
  • How does the vendor support the regulated entity’s HIPAA obligations for the exact service and configuration?
  • What certifications does vendor hold (HITRUST, SOC 2, ISO 27001)?
  • Has vendor had any past privacy/security breaches or regulatory violations?

Data Use and Ownership:

  • What data does vendor need access to? Why is each element necessary?
  • Will data be used only for our institution, or to train models for other clients?
  • Who owns the AI model? Can vendor use our data to improve model for others?
  • Can vendor sell or share our data with third parties?
  • What happens to our data when contract ends (return, deletion)?

Data Storage and Security:

  • Where is data stored (vendor servers, cloud provider, which jurisdiction)?
  • Is data encrypted at rest and in transit? What encryption standards?
  • What access controls are in place? Who at vendor company can access our data?
  • Are vendor employees background-checked?
  • How often does vendor conduct security testing?

Breach and Incident Response:

  • What is vendor’s incident response plan?
  • How quickly will vendor notify us of a breach?
  • Does vendor have cyber insurance?
  • What is vendor’s history of breaches?

Transparency and Auditing:

  • Can we audit vendor’s security practices?
  • Will vendor provide access logs showing who accessed our data?
  • How does vendor demonstrate ongoing compliance?

International Data Transfers:

  • Will our data be transferred outside the U.S.?
  • What privacy protections apply in other jurisdictions?
  • Does vendor comply with GDPR if applicable?

Red Flags: - Vendor unwilling to sign BAA - Vague or evasive answers about data use - Claims that identifiability risk is zero without a method, recipient, context, and supporting analysis - Lack of security certifications - History of breaches or regulatory violations - Offshore data storage without adequate protections - No clear data deletion policy

Negotiating Vendor Contracts: - Compare the vendor’s standard terms with the organization’s risk and procurement requirements - Define data use restrictions, audit evidence, incident notice, subprocessors, material changes, deletion, and exit - Allocate indemnity and insurance in proportion to control, bargaining position, applicable law, and enforceability - Include termination rights if vendor violates terms - Get everything in writing (don’t rely on verbal assurances)

Privacy-Preserving AI Techniques

Emerging technologies can enable medical AI while better protecting privacy.

1. Federated Learning

How It Works: - AI model trained across multiple institutions without centralizing data - Each institution trains model on local data, shares only model parameters (not patient data) - Central server aggregates model updates to improve global model - Cycle repeats until model converges

Privacy Benefits: - Raw training records can remain at participating institutions - May reduce central collection of raw data - Can support multi-institutional training while exchanging updates rather than source records

Limitations: - Model updates can leak information; the magnitude is architecture-, access-, and defense-dependent - Requires technical infrastructure for federated training - Coordination challenges across institutions - Still vulnerable to some inference attacks

Medical Applications: - Multi-hospital AI model development (e.g., radiology AI trained across 10 hospitals without sharing images) - Rare disease research (aggregate learning across institutions with small patient numbers)

Evaluation boundary: Federated learning does not by itself establish privacy, consent, security, or clinical equivalence. A specific project must report the sites, data flow, threat model, aggregation, attacks, defenses, comparator, and performance endpoint before such claims can be made.

2. Differential Privacy

How It Works: - Mathematical technique that adds carefully calibrated noise to data or model outputs - Bounds how much the output distribution can change when one person’s record is added or removed, under a specified mechanism, adjacency definition, and privacy budget - Provides a formal, parameterized guarantee rather than making membership inference impossible

Privacy Benefits: - Rigorous mathematical privacy guarantee (unlike heuristic de-identification) - Can limit defined information leakage under the stated assumptions - Can be applied to training data or model outputs

Limitations: - Reduces accuracy (noise degrades model performance) - Trade-off between privacy and utility - Requires careful tuning of privacy parameters - Not intuitive for non-experts

Medical Applications: - Releasing aggregate health statistics without identifying individuals - Training AI models with privacy guarantees - Sharing genomic data for research

Implementation boundary: A product’s use of the phrase “differential privacy” should be evaluated for the mechanism, privacy budget, composition, population, telemetry, and release process. Do not transfer a consumer-telemetry implementation to clinical-model claims without a source.

3. Synthetic Data

How It Works: - Generate artificial dataset that preserves statistical properties of real data without containing actual patient records - Train generative AI model (e.g., GAN) on real data - Use model to generate synthetic patients (new combinations of characteristics)

Privacy Benefits: - Properly generated records are artificial, but a generator can reproduce or closely resemble training examples - Synthetic data may reduce some disclosure risks after empirical privacy testing - Sharing still requires governance, security, license, provenance, and use restrictions appropriate to the residual risk

Limitations: - Still possible to extract some real patient information from synthetic data (especially if generation model overfits) - Synthetic data may not perfectly replicate all patterns in real data - Quality depends on generation method and real data quality

Medical Applications: - Training AI when real data is scarce or restricted - Sharing data for research, competitions, or education - Testing AI systems before deployment on real patients

Example: Synthea (Synthetic Patient Generator): - Open-source tool generating realistic synthetic patient records - Used for EHR testing, research, and AI development - Not based on real patients (rule-based generation)

4. Homomorphic Encryption

How It Works: - Allows computations on encrypted data without decrypting it - Data remains encrypted throughout AI training and inference - Only data owner can decrypt results

Privacy Benefits: - Can protect data confidentiality during defined computations when implemented correctly - May reduce exposure to a computing party that lacks decryption capability - Can support selected outsourced or multiparty computations

Limitations: - Computational overhead and supported operations vary by scheme, implementation, hardware, and workload - Practicality must be benchmarked for the actual clinical latency and scale requirements - Requires specialized cryptographic expertise

Medical Applications: - Secure cloud-based AI inference (patient data encrypted, cloud performs prediction on encrypted data, result returned encrypted) - Multi-party computation (multiple institutions jointly analyze data without any party seeing others’ data)

Boundary: Encryption during computation does not secure endpoints, identities, keys, outputs, authorized misuse, data quality, or clinical validity. It is one control, not “ultimate” privacy. - Active research area: expect performance improvements

Patient Perspectives on Privacy

Privacy isn’t just regulatory compliance. It’s what patients expect and deserve.

What Patients Care About

Research Themes: - Preferences vary by population, data type, beneficiary, commercial involvement, governance, and question wording. - Transparency, trust, control, public benefit, and protection against misuse recur across ethics and engagement research. - Vayena and Blasimme analyze ethical and governance issues; the paper does not establish a universal majority preference or opt-out rule (Vayena and Blasimme, 2018).

Trust Factors to Evaluate: - identity and accountability of the data steward - public, clinical, research, or commercial purpose - expected benefit and distribution of benefit - data sensitivity, secondary use, access, and safeguards - community history and meaningful recourse

Vulnerable Populations: - Communities with history of medical exploitation (e.g., Tuskegee) understandably more skeptical - Marginalized groups face greater risks from data misuse (discrimination already prevalent) - Important to engage these communities, earn trust through transparency and protections

Building and Maintaining Trust

1. Transparency: - Tell patients when their data is used for AI - Explain purpose, risks, and benefits - Make privacy policies accessible and understandable

2. Consent: - Use meaningful consent or authorization when required or promised - Explain whether choices are granular, what can be withdrawn, and what cannot be undone - Do not substitute a consent screen for an applicable legal basis, institutional accountability, or security

3. Accountability: - Clear responsibility when things go wrong - Mechanisms to report concerns - Enforcement of privacy violations

4. Equity: - Ensure AI benefits reach all communities, not just privileged ones - Address algorithmic bias that harms marginalized groups - Prevent use of AI for discrimination

5. Patient Advocacy: - Physicians as advocates for patient privacy - Push back against exploitative data practices - Prioritize patient welfare over institutional or commercial interests

Regulatory Landscape Beyond HIPAA

HIPAA is just one piece of privacy regulation affecting medical AI.

GDPR (General Data Protection Regulation)

  • The General Data Protection Regulation applies to processing within its material and territorial scope.
  • Health data are a special category. Explicit consent is one possible Article 9 condition, not the only one. A controller also needs an Article 6 basis.
  • Data portability applies within defined conditions.
  • Article 17 provides a right to erasure with exceptions; it is not an unconditional right to delete every clinical or research record.
  • Article 22 concerns decisions based solely on automated processing that produce legal or similarly significant effects, subject to exceptions and safeguards. It is not a general ban on clinical algorithms or a universal right to a technical explanation.
  • Administrative fines depend on the violated provisions and case; avoid using one maximum percentage as a complete compliance summary.

Implications for Medical AI: - Determine territorial scope, controller and processor roles, Article 6 and 9 bases, information duties, data-subject rights, transfers, impact-assessment requirements, and automated-decision safeguards for the exact use. - Obtain EU and member-state advice rather than assuming that treating any EU resident always triggers GDPR.

State Laws (California Consumer Privacy Act, etc.)

  • Growing number of U.S. states passing privacy laws
  • CCPA (California) gives consumers rights to know what data is collected, delete data, opt out of sale
  • Health-data exemptions differ by law and may be entity-level, data-level, or activity-level. HIPAA status does not create one uniform exemption across state privacy laws.

FTC Enforcement Beyond HIPAA

The Federal Trade Commission (FTC) enforces privacy protections for health data that falls outside HIPAA’s scope. Two primary authorities apply:

Section 5 of the FTC Act: Prohibits “unfair or deceptive acts or practices.” Companies that promise to protect health data and then share it with advertisers violate this standard.

Health Breach Notification Rule (HBNR): Requires vendors of personal health records and related entities to notify consumers when their health information is breached. The FTC interprets “breach” broadly to include unauthorized disclosures to third parties, not just traditional cyberattacks (16 C.F.R. Part 318).

Why This Matters for Physicians:

Many consumer health apps are not HIPAA covered entities or business associates for the consumer-facing service. Others may operate in different roles for different functions. The 2024 HBNR amendments clarified coverage of many health apps and similar technologies outside HIPAA (FTC, 2024). Product-specific privacy practices must be checked before a recommendation.

FTC Enforcement Actions: Consumer Health Data
Company Year Violation Penalty Key Facts
GoodRx 2023 FTC alleged disclosure of prescription and health information to advertising platforms contrary to privacy promises and without required HBNR notices $1.5 million civil penalty First FTC HBNR enforcement action
BetterHelp 2023 Disclosed mental health intake data (depression, suicidal thoughts, medications) to Facebook, Snapchat, Pinterest for advertising $7.8M refund to consumers First FTC action returning funds to consumers for health data privacy violations
Flo Health 2021 FTC alleged disclosure of sensitive health information to analytics and marketing firms despite privacy promises Consent order Order imposed privacy-program and notice requirements
Easy Healthcare (Premom) 2023 FTC alleged that the company shared fertility and reproductive health data with Google, AppsFlyer, and China-based firms $100K civil penalty Second HBNR case

Common Pattern: These cases share a consistent fact pattern. Companies promised to keep health data private, then shared sensitive information (prescriptions, mental health status, pregnancy, fertility) with advertising platforms. The FTC treats such sharing as both deceptive (violated privacy promises) and unfair (consumers had no meaningful way to know or consent).

Practical Implications:

  • Health-app regulation depends on the entity, data, function, promises, and applicable federal and state law; “outside HIPAA” does not mean “unregulated.”
  • Privacy policies for consumer apps often permit broad data sharing buried in terms most users never read
  • Physicians recommending consumer health tools should consider whether the app’s privacy practices align with patient expectations
  • A BAA can impose important duties in a qualifying relationship, but it does not prove that an enterprise product is more secure or privacy-protective than every consumer product

For more on distinguishing consumer vs. enterprise AI platforms, see the Toolkit chapter.

ONC Algorithm Transparency

The ONC HTI-1 Final Rule established transparency requirements for predictive decision-support interventions supplied through certified health IT. HTI-1 is not a general privacy statute and does not apply to every AI product. It is relevant when evaluating which source attributes, validation information, and risk-management information are available through certified health IT.

Clinical AI Data-Flow Review

Map the complete lifecycle before contracting or deployment:

Stage Questions
Collection Which fields, images, audio, metadata, and identifiers enter the system?
Transmission Where do data travel, through which interfaces, and under whose account?
Processing Is inference local or remote? Are people, agents, tools, or subprocessors involved?
Storage Are prompts, recordings, output, embeddings, telemetry, and logs retained? For how long?
Training Can customer data improve a shared or customer-specific model? Is training optional?
Output Can output enter the EHR, patient portal, analytics system, or vendor dashboard?
Deletion Can primary copies, backups, caches, logs, embeddings, and derived datasets be deleted when required?
Incident response Who investigates, preserves evidence, notifies affected parties, and meets legal deadlines?

Every retained prompt, log, embedding, recording, and export needs an owner, purpose, access rule, retention period, and deletion pathway.

Proposed Federal Privacy Legislation

  • Federal privacy proposals change across congressional sessions.
  • A proposed bill is not current law and should not be used as a compliance requirement.
  • Any future edition that names a bill should link its official Congress record and state its current status and date.

Conclusion

Clinical AI privacy is not established by a product badge, encryption statement, security certification, or signed BAA. The relevant unit is the complete data lifecycle under the actual legal relationships and deployment configuration (Price and Cohen, 2019).

HIPAA provides a regulatory floor, not a ceiling. Physicians and healthcare organizations must go beyond compliance: implement robust data governance, carefully vet AI vendors, use privacy-preserving techniques when possible, and most importantly, advocate for patients.

Key Principles:

  1. Privacy is a patient right, not just a regulatory requirement
  2. Transparency builds trust: be honest about data use
  3. Minimize data collection and retention
  4. Secure data rigorously (encryption, access controls, monitoring)
  5. Support applicable patient rights and accurately describe their limits
  6. Demand accountability from AI vendors
  7. Use privacy-preserving techniques when possible
  8. Prioritize patient welfare over institutional or commercial interests

Map the data, minimize them, bind each use to a lawful purpose, verify technical controls, and maintain evidence that the system behaves as contracted. Trust cannot be guaranteed by compliance language, but privacy failures can undermine participation, care, research, and legitimacy.

Questions About Clinical AI Privacy

Can clinicians put patient information into ChatGPT or another general AI assistant?

Not by default. Whether protected health information may be entered into an AI service depends on the parties’ HIPAA roles, the permitted purpose, applicable agreements, configuration, safeguards, institutional approval, and other law. A BAA may be required, but a BAA alone does not make every use lawful or secure.

Is AI HIPAA compliant?

No product is HIPAA-compliant in the abstract. Compliance depends on whether HIPAA applies to the parties and function, the permitted use and disclosure, contract terms, deployment configuration, safeguards, risk analysis, workforce practices, and continuing operations. Vendor marketing or a security certification does not decide compliance.

What is a Business Associate Agreement and when is it required?

A BAA is required for a qualifying relationship between a HIPAA covered entity or business associate and another business associate. It defines permitted and required uses and disclosures, safeguards, breach reporting, subcontractors, and return or destruction. Whether a BAA is required depends on the legal roles and service, and the agreement does not authorize uses that the HIPAA Rules otherwise prohibit.

Can de-identified data be re-identified?

Proper HIPAA de-identification uses either Expert Determination or Safe Harbor. HHS notes that de-identified data retain some identification risk. Re-identification risk depends on the remaining data, external information, recipient capability, access, and context, so a historic population estimate should not be treated as a universal current probability.

Continue Reading