RAG On-Premise

On-premise RAG without a GPU By Hubert García Gordon Project page: https://towardsdatascience.com/rag-on-premise/ ShipAI is a Towards Data Science project. Watch all the videos here: https://towardsdatascience.com/videos/ Key takeaways. 1. A semantic cache skipped the slow CPU-only pipeline, but questions one word apart landed almost on top of each other. 2. Paraphrases and opposite questions still overlap in similarity, so no cutoff is safe and the cache ships off. 3. A matched control moved the score from 0.3556 to 0.9333: his first test could be passed by counting shared words. Chapters: 0:00 On-premise RAG, explained 1:04 Qdrant, Ollama, ASP.NET Core 1:51 The QueryAsync pipeline 2:42 Adding a semantic cache 3:50 One word, opposite answers 5:30 What the graph showed 6:56 Three mistakes, all documented 7:47 Why the cache ships off