How a consulting firm can put a trustworthy natural-language query model on top of its own data, without putting client data, state agreements, or its reputation at risk.
Medicaid consulting runs on questions that are individually answerable but expensive to answer: what a state pays for a service, how a rate moved across cycles, how a client's experience compares to its peers. Today those answers are assembled by hand from internal engagement files and scattered public sources. A general-purpose AI chat assistant does not fix this, for two reasons: it cannot be trusted with client data under the firm's compliance obligations, and it cannot be trusted with Medicaid semantics, where a plausible-sounding answer built on the wrong scope is worse than no answer.
This document describes an architecture that addresses both problems. It is a reference design for discussion, not a vendor proposal. Three principles drive everything below.
The model is rented; the ontology is owned. Any firm can call a frontier model. The durable asset is the semantic layer that teaches the system what the firm's data actually means, and that layer is built by Medicaid experts, not by the model.
Every claim cites a source. An answer that cannot be traced to specific rows and source documents is a liability, not a deliverable. Verification is a separate mechanical layer, not a politeness request in a prompt.
Authorization is enforced below the AI. Engagement walls and state data-use restrictions are applied at the data layer, where the model physically cannot reach out-of-scope data, rather than in a prompt the model is asked to respect.
A question moves through the system in one direction: it is scoped by the requester's engagement authorization, resolved into precise meaning by the semantic layer, translated into a governed query, executed against zoned data, and then every claim in the draft answer is checked against the returned rows before the analyst sees it.
Swipe sideways to see the full diagram →
The foundation is a governed warehouse in whatever cloud the firm already operates, with hard separation between three zones that carry different obligations:
Every table in every zone carries provenance: source document, publication date, ingestion date, and transformation lineage. In this domain an answer that cannot be traced to a primary source is not an answer.
Natural-language-to-SQL over raw tables fails in Medicaid, not because the models are weak but because the semantics are treacherous. The same English phrase maps to materially different data depending on scope decisions an expert makes without thinking:
The semantic layer is a machine-readable description of what every table and column means, which joins are valid, which scope questions a given metric requires, and what the known quirks of each source are. It is built table by table, by people who know Medicaid; a model can draft scaffolding, but the layer cannot be reliably autogenerated and must be validated by experts. That is exactly why it is the firm's durable, compounding asset: the model is interchangeable and improves for free, while the ontology is proprietary and appreciates with every engagement that refines it.
The AI model does one job: translate a scoped, semantically resolved question into a query. Three other jobs are deliberately handled by separate mechanisms, because collapsing them into one prompt is how these systems fail quietly:
The system accelerates the analyst; it does not sign the work. Numbers destined for client deliverables get a verification pass against primary sources, and the citation chips the system attaches make that pass fast instead of forensic. The goal is not to remove judgment from the workflow but to move it up the stack, from assembling data to interrogating it.
Choose a question to watch it move through the pipeline. States, engagements, rates, and dollar figures below are invented for illustration; the mechanics are the point.
The architecture above only matters if it operates inside the right contractual and regulatory perimeter. The load-bearing facts, current as of August 2026:
| Item | What it means for this build |
|---|---|
| BAA coverage | Anthropic signs HIPAA Business Associate Agreements for qualifying customers on the Enterprise plan and direct API only, as a configuration the organization must request and activate rather than an automatic property of every Enterprise contract. Consumer plans (Free, Pro, Max) and Team are excluded. Any path that touches PHI must run through BAA-covered services. |
| Zero data retention | Available on the API for approved customers: inputs and outputs are not stored beyond what law and abuse prevention require. One material carve-out: the vendor's frontier-tier “Covered Models” require 30-day safety retention and are unavailable to zero-retention organizations, so the firm chooses between ZDR and the newest model tier. Standard-tier models under ZDR are a sound default for this design. |
| In-tenancy option | If the firm's security posture requires inference inside its cloud provider's boundary, the same models are available through the major providers (for example AWS Bedrock, which is HIPAA-eligible) under the firm's existing cloud BAA. Caveat: model policies follow the model. Frontier-tier “Covered Models” carry mandatory 30-day safety retention wherever they run, including Bedrock, with limited data sharing back to the model vendor. Model selection is itself a compliance decision, not just a capability one. |
| Consumer AI plans | Individual Pro/Max subscriptions are fine as personal productivity tools on public materials. They carry no BAA, default to consumer data settings, and have no admin or audit surface. Client data through a consumer account is a compliance incident, not a cost saving. This split should be written policy. |
| State data-use agreements | State DUAs routinely restrict where data may live, who may access it, and what processing is permitted. Each engagement's DUA must be reviewed before its data enters any zone, and the engagement gate must encode those restrictions. |
| De-identification | The biggest lever in the whole design. Data de-identified under the HIPAA standard is no longer PHI, and a system operating on de-identified plus public data avoids most of the perimeter above. If most engagements can be shaped on de-identified data, the PHI zone can be deferred, possibly forever. |
| 42 CFR Part 2 | Records originating from federally assisted substance-use-disorder treatment programs carry consent requirements beyond HIPAA. If any engagement involves Part 2 data, it needs its own gating, separate from the general PHI zone rules. |
The architecture has a natural seam, and nothing about it requires an outside vendor. The internal half, the de-identified and PHI zones, the engagement gate, and the parts of the ontology that describe the firm's own data, must be owned and built by the firm, because it encodes the firm's obligations and its clients' trust. It is also where the work can start today: the phased approach below runs entirely on the firm's own data and the public sources its analysts already use.
The public half, fee schedules, rate notices, SPAs, and enrollment data across all states, together with the ontology that makes them queryable, is separable. The firm can build it in-house scoped to the states it works most, grow it over time, or eventually evaluate commercial public-data intelligence layers that exist in this space; because nothing confidential ever reaches such a product, that evaluation involves no BAA, no DUA, and no exchange of client data. The decision that matters now is not build-versus-buy. It is keeping the seam clean, so the two halves stay independent and every sourcing option remains open later.
Each phase is defined by what it must demonstrate before the next one starts. Scope discipline matters more than speed: the failure mode for projects like this is a broad, shallow rollout that produces uncited answers and burns trust.
Public zone plus one engagement's de-identified data. A curated set of question types that recur across engagements, each wired through the semantic layer with full citations. No PHI, no open-ended chat.
Gate to phase 2: analysts on the pilot engagement accept the system's cited answers as deliverable-grade starting points, verified against primary sources.
Extend the semantic layer table by table across practice areas. Wire the engagement gate to the staffing system. Stand up the verification workflow as a formal step in deliverable QA.
Gate to phase 3: ontology coverage for the firm's most common recurring analyses, and the access gate demonstrably enforcing engagement walls in testing.
Broader analyst rollout. A PHI zone only if specific engagements require identified data, and only after BAA or in-tenancy arrangements and the relevant DUA reviews are in place. If de-identified data keeps covering the work, this zone stays closed.
Standing rule: the perimeter grows only when an engagement's requirements force it, never for convenience.
What this is not. This is not a chatbot pointed at a document pile, and it is not a replacement for analyst judgment. Systems that skip the semantic layer produce fluent, confidently wrong answers about Medicaid data, and systems that skip the sourcing validator make it impossible to tell. The design above exists to make the fast answer and the defensible answer the same answer.