Private Model as a Service: Zero-Trust Blueprint for Protecting AI Weights- M. Entenza & J. Freimann

Private Model as a Service: Zero-Trust Blueprint for Protecting AI Weights - Marcos Entenza & Jens Freimann, Red Hat In the agentic era, deploying proprietary AI on-premises raises a critical question: how do you protect model IP when infrastructure admins have full hardware access? This session introduces Private Model as a Service (PMaaS), a production-ready reference architecture that secures AI model weights across their entire lifecycle using hardware-rooted Trusted Execution Environments (TEEs). We dive into the technical orchestration of Confidential Containers (CoCo) and KServe to build a cryptographically verified inference pipeline with vLLM. Model weights are distributed and decrypted exclusively inside hardware-verified CPU TEEs (Intel TDX, AMD SEV-SNP) with GPU memory protection (NVIDIA H100/B200). Remote attestation via a Key Broker Service (KBS) ensures decryption keys are only released to policy-compliant, verified environments. We also cover the challenges of running vLLM inside restricted TEEs and our work upstreaming GPU attestation logic into Kata Containers and CoCo. Attendees leave with a practical blueprint for deploying zero-trust confidential AI workloads that decouple model security from infrastructure trust.