Why Deployment Is Still Hard

“Open source is easy to release but hard to deploy,” says Mozilla CTO Raffi Krikorian. “There’s still a bunch of questions of How do I host this thing? How do I provision for it correctly? What do I do about guardrails?” Mozilla’s State of Open Source AI report, which surveyed roughly 1,500 developers, backs up his contention. Teams must make decisions about hosting, provisioning, guardrails, and memory before they can run an open model inside an organization. Commercial APIs handle those choices for users. Open models ask them to manage deployment themselves, with the added benefit of more control. If developers can build installation and deployment tools that reduce the setup work organizations face when they run open source AI, that work could bring cost savings and faster response times for many tasks. Follow O'Reilly on: LinkedIn: https://www.linkedin.com/company/oreilly/ Facebook: http://facebook.com/OReilly Instagram: https://www.instagram.com/oreillymedia BlueSky: https://bsky.app/profile/oreilly.bsky.social TikTok: https://www.tiktok.com/@oreillymedia