Legion Health: The AI Decides Whether a Psychiatric Refill Reaches a Clinician

The AI Can Complete the Renewal

The signed agreement authorizes Legion’s “Mental Health Refill Autopilot” to renew a limited group of existing, non-controlled psychiatric medications.

The system may:

• Verify the patient and prescription.
• Collect information about efficacy, side effects, allergies, clinical changes, and symptoms.
• Screen for conditions requiring escalation.
• Determine whether the request is eligible.
• Authorize the renewal.
• Transmit it to the patient’s pharmacy.

This is not an AI recommendation awaiting a physician’s decision. When the request falls within scope and the system detects no risk or ambiguity, the AI completes the renewal.

The Scope Is Narrow but the Decision Is Consequential

The pilot excludes controlled substances, benzodiazepines, antipsychotics, lithium, valproate, clozapine, new prescriptions, dose changes, medication switches, and renewals requiring new laboratory work or an ECG.

It includes commonly used medications for depression, anxiety, panic disorder, PTSD, OCD, insomnia, and related conditions. Automated renewals are limited to ten between provider reviews or six months, whichever occurs first.

Those boundaries reduce the number of eligible cases. They do not change who makes the decision inside those boundaries.

Patients and Pharmacists Must Be Told AI Is Acting

Before the AI engages with a Utah patient about a renewal, Legion must disclose that the patient is interacting with generative AI, explain how the patient’s information and correspondence will be used or shared, identify the temporary Utah program, and obtain an acknowledgement. Patients may request human review.

Pharmacists must be told that the renewal was generated by AI and receive a direct way to contact Legion and Utah.

Those are meaningful disclosures. They make the AI’s participation visible. They do not establish which model made the decision, who approved its operating rules, or who is responsible for the design. A patient’s acknowledgement authorizes participation; it does not transfer responsibility for the system.

Utah Created a Named-Prescriber Structure

Schedule A allows an affiliated professional entity or provider to rely on Legion’s AI “solely by being the named prescriber” without interacting directly with the patient or another healthcare provider.

Utah’s public explanation says each refill is approved by a licensed physician, either directly or “vicariously through the AI system’s protocol.”

That word—vicariously—carries the responsibility structure.

The physician’s professional authority remains attached to the prescription even when the physician did not review the individual renewal.

Human Review Declines as the Pilot Advances

During Phase A, a clinician can stop an individual renewal before it reaches the pharmacy.

During Phase B, the review occurs after the AI has acted. During Phase C, most individual decisions are not scheduled for clinician review at all.

The phrase “clinician in the loop” therefore describes the operating system as a whole. It does not establish that a clinician participated in a particular patient’s renewal.

A Physician’s Name Is Not Evidence of Physician Judgment

For an automated renewal, the named physician may not have:

• Seen the patient’s answers.
• Reviewed the refill packet.
• Evaluated the safety flags.
• Approved the individual renewal.
• Had an opportunity to intervene before transmission.

The name makes the prescription attributable to a licensed professional. It does not prove that the professional formed or adopted the judgment reflected in the prescription.

Legion’s design exposes the difference between having a human name on the outcome and having evidence that a human made the decision.

Safety Screens Control Access to a Clinician

Legion’s proposal establishes hard stops for conditions including:

• Suicidality or self-harm risk.
• Mania or hypomania.
• Pregnancy-related changes.
• New or worsening symptoms.
• Loss of medication efficacy.
• Severe adverse effects.
• Contraindications or prescription discrepancies.

If the system detects one of those conditions, encounters ambiguity, or cannot verify the patient or prescription, the request enters a clinician queue. Patients and pharmacists may also request escalation.

If the system detects no qualifying condition, the renewal proceeds.

The human safeguard begins only after the automated system decides that human judgment is necessary.

A Deterministic Rule Does Not Make Detection Deterministic

The agreement says each hard stop is tested through deterministic unit tests and synthetic red-team scripts. Those tests can establish that a known trigger produces the expected action.

They do not establish that the system will correctly recognize every real patient statement that should activate the trigger.

A rule can reliably escalate a positive suicidality flag while the language system still fails to recognize that a patient’s words indicate suicidality. If the flag is never generated, the deterministic safeguard never begins.

Naming a missed escalation would identify the technical failure. It would not establish who designed the threshold, who validated it, who determined its performance was sufficient, or who accepted the risk of letting it control access to a clinician.

Legion’s Broader Architecture Uses Multiple Frontier Models

In a November 2025 interview, Legion said its broader AI stack uses frontier models from OpenAI, Google, and Anthropic and switches among them as needed. The company said most of its performance comes from prompts and workflow design rather than a custom-trained model.

Legion also described an API-first architecture connecting LLMs to scheduling, billing, clinical records, patient support, and clinical workflows. The interview was presented in partnership with Legion, so it is evidence of the company’s stated design and strategy—not independent validation of its performance. Legion Health interview

The agreement does not identify which model or model version powers the Utah refill workflow. It does not establish whether the workflow uses one provider consistently or can move among providers.

The Regulated System Has Multiple Moving Parts

The operating system is not merely “the AI.” It includes:

model + model version + prompts + deterministic rails + workflow logic + APIs + clinical data + escalation thresholds + vendor systems

What the Agreement Allows to Change

The agreement allows deterministic-rail updates and prompt or template revisions informed by internal quality review. Formal amendments to the approved proposal require written approval from the parties.

What the Agreement Does Not Answer

The public record does not clearly establish:

• Whether replacing the underlying model counts as an amendment.
• Whether every replacement model must repeat Phase A.
• Who decides that two models are sufficiently equivalent.
• How silent updates by a model vendor are identified.
• How model-specific errors are separated from prompt, workflow, or data errors.
• Whether Utah receives notice before or after a material system change.

If the model changes while the named workflow remains the same, the regulated label may remain stable while the behavior beneath it changes.

Legion Uses a Safety-Weighted Calculation

The agreement classifies clinician comparisons as follows:

Excluding unnecessary escalations from the advancement penalty creates a conservative safety preference. That is a defensible design choice.

It also means that “98% concordance” or “99% concordance” is not ordinary agreement accuracy. One category of disagreement is omitted from the performance threshold.

The Headline Percentage Cannot Stand Alone

Questions the Percentage Cannot Answer

A decision-maker needs the full matrix:

• How often did the AI renew when the clinician would have escalated?
• How often did the AI escalate when the clinician would have renewed?
• Which disagreements counted against advancement?
• Were clinicians reviewing independently, or did they see the AI’s decision and rationale first?
• Did the same disagreement pattern concentrate around a medication, symptom, model, or prompt version?

The percentage describes a scoring rule. It does not by itself establish that the AI and clinicians agreed in 98% or 99% of all reviewed cases.

The Agreement Combines Entities That the Terms Separate

Legion’s Terms of Service distinguish between two organizations:

The Utah agreement takes a stronger approach by naming Legion Health Inc. and Legion Health PA jointly as the “Participant.” Both are therefore parties to the state agreement. Jonathan Kole, M.D., signed for Legion Health PA, and Yash Patel signed for Legion Health Inc.

That prevents the pilot from being presented to Utah as the responsibility of only one side of the corporate structure. It still does not establish which entity performed each act after a specific failure.

The Current Public Terms Describe a Different Prescription Sequence

Legion’s Terms of Service, last updated in October 2025, say a prescription requires a provider consultation, a provider determination that the medication is appropriate, and a prescription written by that provider. The Utah agreement signed in March 2026 allows an AI-authorized renewal under a named prescriber who may not interact with the patient.

The terms predate the pilot and may be revised before patients participate. As currently published, however, they do not explain the individual authority Utah has authorized the AI to exercise. Legion Health Terms of Service

Utah Remains in the Responsibility Chain

Utah granted regulatory relief that makes the automated renewal structure possible. The agreement says Utah does not endorse Legion, requires Legion to protect the state from claims, and preserves legally available remedies for patients and third parties.

Those provisions allocate legal and financial positions after harm. They do not erase Utah’s authorizing act.

Utah decided that a physician could supply professional authority through an approved AI protocol without reviewing every prescription. Responsibility Reconstruction must retain that decision in the record.

The Refill Packet Creates Traceability

Every AI decision is supposed to produce a structured refill packet containing:

• Patient inputs.
• Rule checks.
• Safety flags.
• A decision rationale.

Legion must report volumes, dispositions, escalation reasons, clinician agreement, complaints, adverse outcomes, safety signals, and selected case excerpts to Utah. This is a stronger evidence design than a system that produces only a final prescription.

The packet may help establish what the system recorded and why it reported making the decision.

It is not necessarily the complete evidence needed to reconstruct the decision.

The Audit Record Has an Expiration Problem

What Legion Plans to Preserve

The proposal says clinical records will follow Legion’s standard medical-record retention policy. Pilot-specific audit artifacts will be retained through the pilot and Utah’s closeout, then moved to reduced retention and purged.

The agreement does not state a precise public retention period for those audit artifacts.

What Utah Receives

Utah’s monthly reports are treated as protected records. The state ordinarily receives de-identified data and redacted excerpts rather than complete chat logs. Legion therefore remains central to holding, selecting, explaining, and preserving much of the evidence.

Traceability exists only as long as the necessary record survives.

These are the questions that matter after a patient is harmed. “AI error,” “prompt failure,” “physician oversight,” or “missed escalation” would describe possible mechanisms. None would establish the responsibility chain.

Legion designed a clinic around progressively transferring software-mediated work from humans to LLMs.

Utah authorized the AI to determine whether a psychiatric medication renewal can proceed and whether the patient’s circumstances require clinician involvement.

A physician’s name remains attached to the prescription. That name does not establish that the physician reviewed or adopted the individual decision.

The safety process contains deterministic rules, but those rules depend on the system first recognizing the condition that should activate them.

The performance thresholds use a concordance calculation that does not count every disagreement equally.

The broader Legion architecture can use models from multiple outside providers, while the public agreement does not identify the model and version governing the refill workflow.

Legion creates a structured operating record, but some pilot-specific evidence is scheduled for reduced retention and eventual deletion.

Utah’s pilot therefore preserves a familiar human name on the prescription while permitting the clinical act beneath that name to be performed by a changing AI system.

If a patient is harmed, the presence of a physician’s name will make the prescription attributable. It will not prove that a physician made the decision. Responsibility Reconstruction must determine who designed the decision, who authorized the system, who validated its controls, who could have intervened, whether anyone did, and what evidence survives to prove each connection.

Naming the error is not owning responsibility.