From analytics engineering to context engineering in 4 minutes
Are you ready to level up from analytics engineering into context engineering? In this demo, I take my local Claude Code and Codex chat transcripts, load them into Snowflake, and use the dbt_context_engineering package to chunk, embed, and search them, then have dbt Wizard write a skill from what we found. π Links dbt_context_engineering package: https://github.com/dbt-labs/dbt-context-engineering Example repo from this video: https://github.com/C00ldudeNoonan/dbt-chat-context-eng Wizard Desktop (beta, join the waitlist): https://docs.getdbt.com/docs/dbt-ai/wizard-desktop β±οΈ Chapters 0:00 Analytics engineering β context engineering 0:07 Just-in-time vs. just-in-case context 0:26 Warehouse AI functions and classic search techniques 0:48 The pipeline shape: extract, transform, chunk, embed, search 1:04 The dbt context engineering package 1:23 Demo: chat transcripts β Snowflake 1:54 Wizard Desktop setup 2:15 Summarizing sessions with AI 2:24 Parallel pipelines: testing chunk sizes 2:39 How embeddings and search work 3:19 Eval searches 3:25 Writing a SKILL.md from the results 3:31 Principles: sample, iterate, use your existing tools 3:57 Try the package What's in the example repo - Local ingestion with secret and email redaction before anything leaves your machine - Chunking, classification, embedding, and vector search as ordinary dbt models - Spend caps, dev sampling, and a run log for every AI call (AI is off by default) - Incremental embedding: only new or changed chunks get re-embedded - An eval set to compare chunk sizes on the same questions The package is early and we're actively building it, so issues and pull requests are welcome. #dbt #ContextEngineering #Snowflake #AnalyticsEngineering #RAG #AIAgents