Deploy AI agents on Kubernetes: Move from Ollama to vLLM
How do you move an AI agent from a laptop to a Kubernetes cluster? Learn how to deploy AI agents and serve models using vLLM on Red Hat OpenShift. Transitioning AI agents from local development to cloud infrastructure requires shifting serving runtimes and containerizing workloads. In this episode of AgentOps Unlocked, we demonstrate how to take a local agent setup and run it side by side with a vLLM model endpoint on Red Hat OpenShift. Join Grace Ableidinger to walk through building containers with Podman, pushing images to Quay, and deploying manifests directly to the cluster. Watch live tool execution as the cluster-deployed agent interacts with web search and GitHub using Model Context Protocol (MCP) servers, with chat logs monitored in Red Hat OpenShift AI. Finally, we highlight the critical operational gaps between basic cluster deployment and true enterprise containment. 00:18 Moving AI agents from laptop to production 00:42 What changes when moving to a cluster 01:22 Use Red Hat OpenShift AI to serve models at scale with vLLM 01:52 Using AI Hub to pull models from Hugging Face 02:33 Change .env config to switch to vLLM from Ollama 03:01 Containerizing the agent application 03:18 Building containers with Podman and pushing to Quay 03:52 Testing custom agent tools: web search and GitHub MCP 04:12 Monitoring chat logs in Red Hat OpenShift AI 04:29 Deployed versus production ready operational gaps Don't miss these open source AgentOps and Red Hat AI tools: 📂 Access the AgentOps GitHub repository → https://red.ht/github-agent-ops 🤖 Check out Red Hat AI's Hugging Face → https://red.ht/rhai-hugging-face 💡 Learn more about AI inference → https://www.redhat.com/en/topics/ai/what-is-ai-inference?sc_cid=RHCTG0260000496899 Watch the previous episode of AgentOps Unlocked ""How to run local AI agents for free"" → https://youtu.be/i4A8jN41NNs Want to learn more? Explore the full AgentOps Unlocked video series → https://www.youtube.com/playlist?list=PLEvHfaWT80U4 #AgentOps #AIAgents #OpenShift #vLLM #MCP #RedHat #OpenSource #Kubernetes