Views: 1019

Every LLM interaction. Logged. Immutable. Attributed. Human-directed AI assistance with deterministic governance for chat and MCP tool calls.

What Is LLM Provost?

LLM Provost is a complete enterprise AI governance platform. It wraps the entire AI stack — from the chat UI down to the LLM model — in a deterministic governance layer that captures, attributes, and locks every interaction.

Unlike tools that only monitor prompts, LLM Provost sits in the data path. Every chat message, every MCP tool call, every model response passes through the proxy and is logged with 4-layer identity before it reaches the user or the external system.

Human-in-the-Loop by Design

LLM Provost is not an autonomous decision-making system. Professionals use LibreChat UI to interact with AI using text, documents, and images. The AI retrieves information through governed MCP tools and generates recommendations, summaries, and analyses. Humans are always responsible for reviewing, approving, and executing decisions. LLM Provost ensures every step of that human-directed process is logged, attributed, and immutable. This empowers your team and removes the shadow AI from their pockets.

The Full Stack

Component	Technology
Chat Interface	LibreChat — enterprise UI with conversation history, search, and multi-model support
Governance Proxy	OpenResty / Lua — intercepts all traffic, enforces rules, captures bodies, attributes identity
LLM Provider	AWS Bedrock (production) or OpenWire / local Llama (dev) — your data never trains external models
Database	MongoDB (dev) / Amazon DocumentDB (production) — conversation persistence
Search	Meilisearch — full-text conversation search across all logged interactions
Log Pipeline	Fluent Bit → Amazon S3 (Object Lock) — immutable WORM storage, tamper-proof
Auth	AWS Cognito / Azure Entra ID — enterprise SSO, JWT-based identity injection

4-Layer Identity Logging

Every transaction — chat or tool call — is logged with four immutable identity layers:

Field	What It Captures
user_id	Who is asking — extracted from Cognito JWT, cannot be faked
customer_id	Who is the subject — extracted from MCP tool arguments (e.g. patient name, account number)
conversation_id	Which session — injected by LibreChat, ties all messages in a chat together
request_id	Which transaction — unique per request, links request and response in the audit trail

Example: Wealth Management

A financial advisor opens a chat for Client ID Craig-001. They upload portfolio documents and ask the AI for rebalancing options. The AI retrieves records via governed MCP tools and proposes actions. The advisor reviews the recommendations, approves the changes, States exactly what he desires, and executes. Every step — the prompt, the tool calls, the AI response, the user identity — is logged and auditable, and rules are easily applied.

MCP Tool Governance

When your staff uses AI, it calls external tools on their behalf — Salesforce, Athena, your EHR, your CRM — LLM Provost intercepts and governs every call:

• Tool allowlists: Only approved MCP tools can be called
• Tool blocklists: Destructive tools (delete, export) are blocked at the proxy
• Rate limits: Per-tool, per-user throttling prevents runaway loops
• Time-based rules: Restrict sensitive tools to business hours
• Token caps: Reject oversized prompts before they reach the model
• Full body capture: Request and response bodies logged for every tool call

Compliance & Sovereignty

• HIPAA: Immutable audit trails, private LLM endpoints, no data leakage to external training
• SEC / FINRA: Tamper-proof WORM logging on S3 Object Lock — 7-year retention configurable
• GDPR / EU AI Act: Full interaction records, right-to-explanation support, data sovereignty in your VPC
• On-prem capable: Deploy entirely in your data center with local Llama models — no cloud required

The Four Pillars