How healthcare organisations govern data for AI systems — HIPAA Safe Harbor and Expert Determination de-identification, consent frameworks for AI training and retrieval, access-controlled RAG over clinical knowledge bases, and the specific controls that satisfy both HIPAA and EU AI Act requirements.
Every healthcare organisation that deploys AI faces the same sequence of questions before any technical work begins: What data can we use to train this AI? What data can we index in the knowledge base? What data can we pass to the model at inference time? Who needs to consent to what? How do we de-identify data that can't be used in its original form? These are data governance questions, and they have specific answers under HIPAA — answers that most healthcare AI teams discover too late in the deployment cycle, after infrastructure decisions have already been made that need to be unwound.
This guide answers those questions with the specific HIPAA requirements, the technical implementations that satisfy them, and the AIZA-HexTyx controls that enforce the governance decisions in production. The HIPAA technical safeguards implementation guide: Healthcare AI Security: HIPAA-Compliant LLM Deployment →
HIPAA's de-identification standard (45 CFR §164.514) provides two pathways for removing the protected status from health information, enabling its use for AI training and development without individual patient consent. Understanding both pathways — and their limitations for AI applications — is the foundation of healthcare AI data governance.
Pathway 1 — Safe Harbor (§164.514(b)): Remove 18 specified identifier categories from the data. The 18 categories include: names, geographic data below state level (street address, city, zip code, county), dates other than year (birth date, admission date, discharge date, date of death), ages over 89, phone numbers, fax numbers, email addresses, Social Security Numbers, medical record numbers, health plan beneficiary numbers, account numbers, certificate/license numbers, VINs, device identifiers, URLs, IP addresses, biometric identifiers, full-face photographs, and any other unique identifying number or code. Additionally, the covered entity must not have actual knowledge that the information could be used alone or in combination to identify an individual.
For AI training and knowledge base construction, Safe Harbor de-identification has a critical limitation for LLM applications: LLMs are designed to reason about patterns across text, and they can potentially re-identify de-identified records by combining de-identified clinical notes with publicly available information. The "no actual knowledge" clause becomes difficult to satisfy when the AI system itself is capable of sophisticated inference. Safe Harbor is appropriate for structured training data used for traditional ML models; its adequacy for LLM training data should be assessed with counsel and a privacy expert who understands LLM capabilities.
Pathway 2 — Expert Determination (§164.514(b)(1)): A statistical or scientific expert applies generally accepted principles to determine that the risk of re-identification is very small. Expert Determination provides a stronger basis for LLM applications because it explicitly accounts for the re-identification risk posed by the specific downstream use — including uses by sophisticated AI systems capable of inference. For clinical knowledge bases powering RAG systems, Expert Determination should specify: the AI system's inferencing capabilities, the external data sources the AI has access to, and the population-level re-identification risk given both.
Expert Determination is more expensive and slower than Safe Harbor, but for clinical knowledge bases that will be used to power LLMs with broad knowledge and reasoning capability, it is the more defensible pathway. Document the expert's methodology, qualifications, and conclusions as part of the data governance record — this is the HIPAA audit evidence for data use decisions.
De-identification removes the regulatory requirement for individual consent, but doesn't resolve the ethical expectation of consent in healthcare contexts. Many patients expect that their health information will be used for their care — not for training commercial AI systems. This expectation gap is creating reputational and regulatory risk for healthcare organisations that use patient data for AI without disclosure, even when legally permitted to do so under HIPAA's de-identification provisions.
HIPAA Authorization for identifiable data use: Where de-identified data is insufficient (the AI requires identifiable data for accuracy), HIPAA requires a valid Authorization under §164.508. For AI training use, the Authorization must specifically describe the type of AI system being developed, the data that will be used, how the data will be protected, and whether the data may be used by any commercial partners. Generic "research use" Authorizations are insufficient for commercial AI training — the Authorization must be specific to the AI application.
Notice of Privacy Practices (NPP) disclosure: Even where AI use is permitted without individual Authorization (de-identified data, or uses permitted under the TPO exception), healthcare organisations should update their NPP to disclose AI use of health information. Patients who later discover their data was used for AI training without disclosure — even if legally permissible — create regulatory scrutiny and reputational risk. NPP disclosure is a governance control that prevents this.
Opt-out mechanisms for AI training: Several state laws (California CMIA, Texas THIPA, and others) impose consent requirements beyond HIPAA for commercial use of health data in AI training. Implement an opt-out mechanism for AI training use regardless of which state law applies — it simplifies compliance across multiple jurisdictions and reflects the ethical baseline most patients expect.
A clinical knowledge base for RAG typically contains: clinical guidelines, formulary information, drug interaction databases, published clinical literature, institutional protocols, and clinical pathways. Unlike patient records (which require PHI governance), most clinical knowledge content is not patient-specific — it is the institutional and scientific knowledge that clinicians use to make decisions. However, it still requires careful governance because the content is valuable, clinicians need confidence in its accuracy and currency, and adversarial manipulation of clinical knowledge content can directly affect patient care decisions.
Content validation for clinical knowledge RAG: Clinical knowledge changes rapidly — drug interactions are updated, guidelines are revised, new safety signals emerge. A clinical knowledge base that contains outdated information is a patient safety risk, not just an accuracy problem. Governance requires: source validation (only content from approved clinical sources is indexed), currency checking (content is re-validated against the source on a defined schedule, shorter for rapidly-changing content like drug safety), and version tracking (when was each chunk last validated, against which version of the source).
Aegis CP2 chunk validation serves dual purpose in clinical knowledge RAG: injection pattern detection (adversarial instructions embedded in clinical content) and provenance tracking (every chunk logged with source, validation date, and version). Rising CP2 block rates from a clinical knowledge source are both a security signal (potential corpus poisoning) and a content quality signal (the source may be updating in ways that introduce content validation failures).
Access-controlled retrieval for clinical knowledge: Not all clinical knowledge is appropriate for all contexts. Drug dosing information appropriate for a prescribing physician context may not be appropriate for a patient-facing chatbot. Clinical risk stratification tools appropriate for an ICU context may not be appropriate for a primary care context. RAG retrieval must be scoped to the appropriate clinical knowledge domain for the specific deployment context — not because different users can't be trusted, but because context-appropriate retrieval produces better clinical recommendations and reduces the risk of out-of-context information misapplication.
Implement context-scoping through retrieval metadata tagging at ingestion time: every chunk tagged with its appropriate clinical context (prescribing, patient education, specialist, generalist), the professional roles authorised to retrieve it, and the clinical settings in which it is applicable. Retrieval queries include the deployment context as a mandatory filter — an AI deployed in a patient-facing context can only retrieve chunks tagged as appropriate for patient education contexts.
The complete RAG pipeline security guide that covers all five ingestion-to-output stages: RAG Security: Complete Guide to Securing Retrieval Systems →
HIPAA's minimum necessary standard (§164.502(b)) requires that covered entities limit use of PHI to the minimum necessary to accomplish the intended purpose. Applied to LLM prompts: the model should receive only the PHI elements necessary for the specific clinical AI task, not the complete patient record.
This is harder to implement than it sounds because LLMs generally perform better with more context — clinicians and AI developers naturally want to provide complete patient context to maximise recommendation quality. The tension between "minimum necessary" and "maximum accuracy" must be resolved through clinical governance, not technical default. Establish a clinical AI data minimisation policy that specifies: for each clinical AI use case, which PHI elements are necessary and which are not. Implement this policy in the retrieval and prompt construction layer — retrieve and pass only the authorised PHI fields for each use case, not the complete patient record.
Configure Aegis CP4 with healthcare-specific PHI detection patterns beyond the standard 13 secret patterns: patient name + date of birth combinations (linkability risk), medical record number patterns, diagnosis code + patient identifier combinations, medication records + patient identifiers. The zero-retention mode for PHI-heavy AI sessions: AEGIS_ZERO_RETENTION=true ensures that session content containing PHI is not retained in the audit log — only metadata. This satisfies both the minimum necessary standard (PHI not retained beyond inference) and HIPAA §164.312(b) audit requirements (the audit log proves monitoring occurred without retaining PHI substance).
The complete RAG pipeline architecture that enforces minimum necessary retrieval at every stage: RAG Pipeline Architecture Security: Complete Guide End-to-End →
The HIPAA technical safeguards that govern PHI processing at inference time: HIPAA Technical Safeguards for AI Systems: PHI in LLM Prompts →
For healthcare organisations operating in the EU, the EU AI Act's high-risk classification applies to AI systems used "for the purpose of making decisions or assisting in making decisions related to access to health and life insurance and medical services" and AI used in clinical decision support that influences treatment decisions. High-risk classification requires: risk management system (AI SBOM + risk register), technical documentation (development and validation documentation), logging and audit trail (Aegis audit log), transparency disclosure (patients informed when AI is used), human oversight (clinician review of AI recommendations), and accuracy and robustness testing (HexTyx adversarial testing).
The GDPR interaction compounds complexity: healthcare data is "special category" personal data under GDPR Article 9, requiring explicit consent or a derogation for processing. AI training on health data requires either: explicit consent (Article 9(2)(a)), processing for preventive or occupational medicine (Article 9(2)(h)), or processing for scientific research with appropriate safeguards (Article 9(2)(j)). The same data use that HIPAA permits under de-identification may require explicit consent under GDPR if EU patients are involved.
Resolve the GDPR/EU AI Act compliance with the same zero-retention configuration used for HIPAA: session content not retained, metadata-only audit log satisfies EU AI Act Art. 12 logging requirements without GDPR Art. 5(1)(e) storage limitation conflict. The SOC2 evidence package that overlaps with healthcare AI compliance requirements: SOC2 for AI Products: Practical Guide for LLM Platforms →
De-identification:
Consent:
Clinical knowledge RAG:
PHI minimisation:
Run a scan and verify your PHI detection configuration, zero-retention mode operation, RAG chunk validation for clinical content, and minimum necessary retrieval enforcement.
Validate Healthcare AI Data Governance →