Introduction to RAG and Agents

We build a RAG system over real documentation and evolve it into an agentic search workflow. The knowledge base is the Evidently AI documentation, a real, evolving set of Markdown files that makes RAG genuinely useful, since LLMs can't keep up with library docs on their own.

What we cover:

• Build a classic RAG pipeline: index documents with minsearch and retrieve highlighted snippets
• Implement search (returns snippets) and get_file (returns the full document)
• Let the LLM decide which tool to call
• Wire the tools into an agent using toyaikit and PydanticAI
• The agent mirrors how humans read docs: search, scan snippets, open the promising one

This workshop was originally delivered at DataMakersFest 2026 in Porto, Portugal. There is no recording available.

The system you will build

The architecture has one LLM agent with two tools and two data sources:

The agent has two tools. search returns short highlighted snippets so the agent can decide which documents are worth reading. get_file returns the full document so the agent can read it end-to-end. The pattern mirrors how humans read documentation: search, scan snippets, open the promising one.

Links

Resources not included in the workshop materials list:

• toyaikit - teaching framework for agents
• PydanticAI - production agent framework