Private LLM Deployment for Law Firms: On-Prem vs VPC vs Hybrid

Most firms evaluating a private LLM assume the choice is binary: send prompts to a public model behind a business agreement, or stand up an on-prem cluster in a hardened closet. The real decision has three doors, and the differences between them show up in the audit trail, the annual budget, and the malpractice risk register long before they show up in the demo.

Single-tenant cloud VPC, on-prem GPU cluster, and hybrid burst are all defensible answers for a firm handling privileged material. They fail differently, they scale differently, and they impose very different obligations on the CIO who signs the contract. This piece walks through the cost drivers, break-even volumes, and confidentiality trade-offs at each tier, then closes with a short list of what to demand in writing before a single matter runs through the system.

Why the Deployment Model Is a Confidentiality Decision, Not an IT Decision

Any private LLM deployment for law firms sits inside a well-defined ethics perimeter. ABA Formal Opinion 512 implicates competence (MR 1.1), confidentiality (MR 1.6, 1.9(c), 1.18(b)), communication (MR 1.4), candor toward the tribunal (MR 3.3), supervisory responsibilities (MR 5.1 and 5.3), and fees (MR 1.5) when lawyers use generative AI. Each of those rules pushes back on a different part of the architecture: MR 1.6 constrains where inference can physically run, MR 5.3 constrains how vendor engineers touch the system, and MR 3.3 constrains what happens when a model hallucinates a citation in the style of Mata v. Avianca, in which a federal judge sanctioned two attorneys $5,000 for a brief containing six fabricated citations.

Regulators are moving in the same direction. NIST released the Generative AI Profile (NIST-AI-600-1) on July 26, 2024 to help organizations identify risks unique to generative systems, and the EU AI Act entered into force on 1 August 2024, with high-risk obligations phasing in on a schedule firms with European clients cannot ignore. Choosing a deployment model is, in effect, choosing which of those obligations you will inherit and which you will outsource.

Single-Tenant Cloud VPC Is the Default for a Reason

A single-tenant LLM inside a virtual private cloud gives the firm a logically isolated network with private IP ranges, dedicated subnets, and its own gateways, running on hyperscaler hardware. AWS documents the model in its Amazon VPC user guide, and the same shape exists on Azure and GCP. Model weights, embeddings, and vector stores live in accounts the firm owns; inference runs on GPUs the firm reserves; nothing touches a shared endpoint.

Cost drivers for a VPC deployment are almost entirely operational. Reserved H100 capacity from specialty providers now sits in a wide band: cloud alternatives start at $1.49/hr for H100 and $2.15/hr for H200, with AWS at roughly $3.90/hr after June 2025 cuts. A serious contract-review workload, with two reserved 8×H100 nodes plus vector storage, RAG orchestration, logging, and observability, typically lands in the mid six figures per year, before staff.

Staffing load is the honest number. A VPC deployment needs a small platform team: one MLOps engineer, one security engineer with cloud posture experience, and shared time from a data engineer. A single-tenant LLM for a law firm is realistic for a mid-size practice because the hyperscaler owns the physical layer, and infrastructure security guidance from AWS spells out tenant isolation, security-group controls, and VPC endpoints the firm can inherit rather than build. The confidentiality trade-off is the honest one: the provider still runs the hypervisor, and any subpoena served on the provider is a subpoena the firm has to be ready to answer.

On-Prem GPU Clusters Buy Certainty at a Real Price

An on-prem cluster is the answer when the client agreement, the matter, or the jurisdiction forbids inference in a cloud region at all. Weights, embeddings, prompts, and retrieval indexes never leave the firm's data center. That is the ceiling of what confidentiality architecture can offer, and it is the reason air-gapped deployments are typically reserved for national-security work, sovereign M&A, and the largest litigation books.

The price of that certainty is significant. A single server equipped with 8x NVIDIA H100 GPUs can cost over $250,000, and industry-average lead times for GPU clusters run 5-6 months. Hardware is only the entry ticket: an Introl five-year TCO model finds that $3M in GPUs actually costs $15.7M over five years once power, cooling, staff, and maintenance are included. The corollary is well documented: break-even analysis shifted to favor cloud below roughly 60-70% utilization, and few firms outside the AmLaw 20 sustain that kind of continuous inference load.

Staffing is the second half of the bill. On-prem requires a systems administrator with GPU experience, a network engineer, a security engineer, and enough MLOps depth to handle model updates, driver upgrades, and CUDA regressions without external help. Refresh cadence matters as well: GPU generations move every 2-3 years, and a cluster purchased in 2026 will look tired against the Blackwell successor generation before its depreciation schedule ends.

None of that argues against on-prem. It argues that on-prem is a decision to run a small AI operations group in-house, in perpetuity, with the same discipline the firm applies to its document management system. Firms that succeed with it usually approach the build through an enterprise AI deployment engagement rather than a hardware purchase order, and pair it with formal legal AI governance before the first matter runs through it.

Hybrid Burst Splits the Workload Along the Confidentiality Line

Hybrid deployments are the pragmatic middle. Sensitive matters (privileged communications, sealed filings, PHI, deal team materials under NDA) stay on a small on-prem or VPC island. Less sensitive workloads (general research, precedent search over public case law, template drafting) burst to a larger reserved VPC pool. Third-party analysis of the pattern describes it as the pragmatic choice for mid-size organizations balancing compliance, cost, and flexibility.

The engineering discipline is routing. The firm needs a policy layer that decides, per prompt, which environment can see which data, based on client, matter, jurisdiction, and document classification. Well-designed hybrids resemble the private hybrid AI deployment pattern: an internal gateway with deterministic routing rules, encrypted transport between tiers, and per-tier audit logs that can be reconciled to a single matter number. Policy-as-code frameworks make those routing rules reviewable in the same way as any other firm policy.

The cost story is not "cheaper than both." It is "closer to VPC economics, with a small on-prem footprint for the most sensitive slice." Break-even math favors hybrid when at least 60-70% of inference volume is safe to burst; below that threshold, the on-prem footprint dominates and the firm is essentially paying for on-prem with an unnecessary cloud dependency on top.

How to Choose, in Concrete Terms

Three variables drive the decision more than any vendor pitch: sustained inference volume, the mix of matter sensitivity, and the firm's tolerance for running an infrastructure team. The following mapping is not universal, but it holds for most firms LAW.co works with.

Firm Profile	Recommended Tier	Break-Even Signal	Staffing Floor
Boutique / <100 lawyers	Single-tenant VPC	Any inference volume	1 MLOps + shared security
Mid-size / 100-500 lawyers	VPC, hybrid at 18 mo.	>2M tokens/day sustained	2 platform + 1 security
Large / 500+ lawyers	Hybrid burst	>60% burst-safe volume	Full platform team
Sovereign / air-gapped matters	On-prem cluster	Client mandate, not volume	Dedicated AI ops group
In-house legal dept.	VPC inside enterprise cloud	Uses parent-org tenancy	Shared with enterprise IT

Adoption pressure is the reason none of this is theoretical anymore. Clio's 2025 Legal Trends Report found 87% of legal professionals at large firms reported adopting AI, while 53% said their firm had no AI policy or that they were unaware of one, and a Thomson Reuters survey found the share of legal organizations actively integrating generative AI rose from 14% in 2024 to 26% in 2025. The gap between adoption and governance is where breach cost lives, and breach cost has not moderated: IBM's 2025 report puts the U.S. average breach at an all-time high of $10.22 million.

What to Demand in the Contract

Regardless of tier, the buying-side terms that actually protect the firm are narrow and specific. A vendor unwilling to write these into the master agreement is telling you something.

• Tenant isolation, in writing. Named accounts, named subnets, named model instances. No shared inference endpoints. No cross-tenant caching of embeddings or prompts.
• No training on firm data, ever. Explicit prohibition on using prompts, completions, retrieved documents, or logs for provider model training, fine-tuning, or evaluation.
• Data residency and retention. Region locks that match client commitments, and configurable retention windows tied to the firm's data retention compliance policy rather than a vendor default.
• Attorney oversight at the model boundary. Human-in-the-loop controls for any output that reaches a client, a tribunal, or a filed document, with reviewer identity captured in the audit log.
• Full audit export. Prompt, retrieval context, model version, latency, and reviewer action, exportable to the firm's SIEM in a documented schema.
• Subpoena and government-access notice. Contractual commitment to notify the firm of any third-party demand for access to systems or logs, to the extent legally permitted.
• Model version pinning and rollback. The right to pin a model version for the life of a matter, and to roll back after any provider-side update.
• Exit rights. Weights the firm has fine-tuned, embeddings the firm has built, and logs the firm has generated must be exportable in a documented format on termination.

Those provisions apply whether the model runs in a VPC managed by a partner, in a rack the firm owns, or across both. They are the substrate on which everything else, from agentic AI for law firms to secure legal AI sandboxing, sits.

Where the Decision Usually Lands

Most firms under 300 lawyers should start on a single-tenant VPC with strict contract terms, then re-evaluate at eighteen months of production usage. Firms with sustained high-volume, high-sensitivity workloads (or client commitments that forbid cloud inference) should build hybrid, treating the on-prem island as an inference vault rather than a general-purpose data center. Pure on-prem is the right answer for a specific slice of practice, not a default. The buying decision is less about which architecture is best in the abstract and more about which one the firm can honestly staff, audit, and defend to a client asking where its documents live.