There is no authoritative record of AI agents systematically suppressing adverse events, so this is a threat model. It matters because a malicious input must not be able to silently change what investigators believe about a study’s safety.
Part of the AI Security for Research Institutions guide series.
Clinical research depends on accurate safety reporting.
Adverse events must be captured, evaluated, escalated, and documented according to the applicable study processes and requirements.
AI can help with that work.
It can also introduce a new class of integrity risk:
What happens if an AI workflow is manipulated into suppressing, changing, delaying, or misclassifying a safety signal?
There is not a broad authoritative record showing autonomous AI agents systematically suppressing clinical-trial adverse events. The concern is therefore best treated as a security threat model.
But it is an important threat model because the consequences can be unusually high.
A normal enterprise workflow can tolerate some operational error.
Safety reporting cannot be treated the same way.
An AI system involved in adverse-event processing may:
Each step can influence what a human investigator sees.
If malicious content manipulates the agent, the result may be a distorted safety signal.
Attackers do not necessarily need the AI to invent an adverse event.
They could attempt to make the system:
This is an integrity attack.
The objective is to change what the research team believes is happening.
Clinical and research records are often text-rich.
A report may contain free text.
A document may be imported.
An investigator note may be retrieved.
A message may enter the workflow.
That creates opportunities for indirect prompt injection.
A malicious instruction hidden inside one of those sources could potentially influence an AI component that processes the text.
The human user may have done nothing malicious.
The safety workflow itself became the attack surface.
Human review helps.
But only if the reviewer receives sufficient information.
Suppose an AI system presents:
“Low-priority event — no escalation recommended.”
A reviewer may accept the recommendation.
But what if the AI had silently omitted a critical piece of the source narrative?
A trustworthy workflow should preserve the evidence needed to reconstruct the recommendation.
The reviewer should be able to see:
Research safety systems should preserve provenance across the AI pipeline.
The institution should know:
Where did the information originate?
Which AI system processed it?
Which version was running?
What transformations occurred?
Which human approved the final action?
That provenance is useful for both security investigations and research integrity.
An AI model can be useful for identifying potential safety signals.
The security architecture should not automatically give the same model authority to erase, suppress, or close the record.
This is another application of a broader rule:
The model can recommend. The control system decides.
High-impact operations should require explicit authorization.
A safety workflow should have expected behavior.
For example:
Sudden changes deserve investigation.
A sharp drop in escalation rates may look like “improved efficiency.”
It could also indicate:
The key is to monitor behavior, not only system uptime.
Security testing should simulate:
The test question is not whether the AI gives a perfect answer.
It is whether an attacker can systematically move the workflow away from truthful, complete, auditable safety reporting.
Clinical research AI security ultimately intersects with research integrity.
A security control that prevents unauthorized data access is important.
A control that preserves accurate safety evidence can be equally important.
The institution should treat the provenance and integrity of AI-assisted safety processing as part of the research record.
For consequential clinical-research AI, consider:
AI can help research teams process safety information faster.
But speed is not the same as safety.
For adverse-event workflows, the security objective should be stronger:
A malicious input must not be able to silently change what investigators believe about the safety of a study.
HexTyx tests AI agents against live, LLM-backed applications that use synthetic data in controlled environments. Our record covers 28 domain applications. These are not research-institution deployments and no real participants or studies were involved, but the failure patterns map directly onto the boundaries in this guide.
The guarantee has to come from the system, not the model’s judgment. In HexTyx testing, live Claude Haiku 4.5 consistently declined purely autonomous injected instructions in LLM-mediated apps, yet complied when the same manipulation was framed as a direct conversational claim. Where a pipeline executed directives in deterministic server code, the exploit worked regardless of model judgment.
The most serious patient-safety result was in a synthetic clinical-records app (Concord): a fake-confirmation attack executed a medication order for a warfarin patient despite a detected interaction. A safety-reporting workflow needs the same design this guide recommends: independent authorization at the point of action and a reviewer who can see the source.
Run the free playground scanner against your own AI application, or read the other guides in this series.