Views: 0

Every LLM interaction. Logged. Immutable. Attributed. Deterministic governance for chat and MCP tool calls in high-stakes environments.

What Is LLM Provost?

LLM Provost is a complete enterprise AI governance platform. It wraps the entire AI stack — from the chat UI down to the LLM model — in a deterministic governance layer that captures, attributes, and locks every interaction.

Unlike tools that only monitor prompts, LLM Provost sits in the data path. Every chat message, every MCP tool call, every model response passes through the proxy and is logged with 4-layer identity before it reaches the user or the external system.

The Full Stack

Component	Technology
Chat Interface	LibreChat — enterprise UI with conversation history, search, and multi-model support
Governance Proxy	OpenResty / Lua — intercepts all traffic, enforces rules, captures bodies, attributes identity
LLM Provider	AWS Bedrock (production) or OpenWire / local Llama (dev) — your data never trains external models
Database	MongoDB (dev) / Amazon DocumentDB (production) — conversation persistence
Search	Meilisearch — full-text conversation search across all logged interactions
Log Pipeline	Fluent Bit → Amazon S3 (Object Lock) — immutable WORM storage, tamper-proof
Auth	AWS Cognito / Azure Entra ID — enterprise SSO, JWT-based identity injection

4-Layer Identity Logging

Every transaction — chat or tool call — is logged with four immutable identity layers:

Field	What It Captures
user_id	Who is asking — extracted from Cognito JWT, cannot be faked
customer_id	Who is the subject — extracted from MCP tool arguments (e.g. patient name, account number)
conversation_id	Which session — injected by LibreChat, ties all messages in a chat together
request_id	Which transaction — unique per request, links request and response in the audit trail

MCP Tool Governance

When the AI calls external tools — Salesforce, Athena, your EHR, your CRM — LLM Provost intercepts and governs every call:

• Tool allowlists: Only approved MCP tools can be called
• Tool blocklists: Destructive tools (delete, export) are blocked at the proxy
• Rate limits: Per-tool, per-user throttling prevents runaway loops
• Time-based rules: Restrict sensitive tools to business hours
• Token caps: Reject oversized prompts before they reach the model
• Full body capture: Request and response bodies logged for every tool call

Compliance & Sovereignty

• HIPAA: Immutable audit trails, private LLM endpoints, no data leakage to external training
• SEC / FINRA: Tamper-proof WORM logging on S3 Object Lock — 7-year retention configurable
• GDPR / EU AI Act: Full interaction records, right-to-explanation support, data sovereignty in your VPC
• On-prem capable: Deploy entirely in your data center with local Llama models — no cloud required

The Four Pillars