What Is an AI context window? Understanding short-term memory in LLMs

This short explainer breaks down what an AI context window is and how it works as a model’s short-term memory. It’s for people using or building with LLMs who want clearer, practical intuition about how much context a model can really handle. GT Hill from Cribl explains how tokens, previous messages, web results, documents, and images all consume space in the context window, and why that window has a finite size measured in thousands of tokens. You’ll learn how overloading the context window can actually degrade answer quality, how this is similar to human short-term memory, and what that means for designing prompts and AI-powered features. By the end, you’ll understand: - What a context window is and how it relates to tokens and short-term memory - Which inputs (chat history, web searches, files, images) fill up the context window - Why a “too full” context window leads to worse results instead of better ones - How engineers (including at Cribl) think about context window limits when adding AI into their products ## Follow Cribl LinkedIn: https://www.linkedin.com/company/cribl/ Twitter: https://www.twitter.com/cribl_io Sign up for a Cribl.Cloud account: https://cribl.cloud/signup/ Learn more about Cribl: https://cribl.io