All Tags

Tag

search

4 items tagged with "search"

Event

From RAG to Agents: Implementing Agentic Search

We build a classic RAG system over real documentation, then evolve it into an agentic search workflow where the LLM decides what to search for and whether to open a full document. Along the way we implement two tools - search (returns highlighted snippets) and get_file (returns the full document) - and wire them into an agent using both toyaikit and PydanticAI. The knowledge base is the Evidently AI documentation: a real, evolving set of Markdown files that makes RAG genuinely useful, since LLMs can't keep up with library docs on their own. This workshop was originally delivered at DataMakersFest 2026 in Porto, Portugal. There is no recording available. The system you will build The architecture has one LLM agent with two tools and two data sources: flowchart LR USER["User"] AGENT["Agent (LLM)"] SEARCH["search tool<br/>highlighted snippets"] GETFILE["get_file tool<br/>full document"] MINSEARCH["minsearch index<br/>Evidently docs"] FILEINDEX["file_index<br/>filename -> content"] USER -->|question| AGENT AGENT -->|decides which tool| SEARCH AGENT -->|decides which tool| GETFILE SEARCH -->|query| MINSEARCH MINSEARCH -->|snippets| SEARCH GETFILE -->|filename| FILEINDEX FILEINDEX -->|full text| GETFILE SEARCH -->|results| AGENT GETFILE -->|results| AGENT AGENT -->|answer| USER The agent has two tools and decides which one to use. search returns short highlighted snippets so the agent can decide which documents are worth reading. get_file returns the full document so the agent can read it end-to-end. The pattern mirrors how humans read documentation: search, scan snippets, open the promising one. Links Resources not included in the workshop materials list: toyaikit - teaching framework for agents PydanticAI - production agent framework

May 4, 2026
rag ai-agents llm-engineering search tooling-architecture
Event

Build Your Own Search Engine

We build a search engine from scratch over DataTalks.Club Zoomcamp FAQ documents. We start with TF-IDF text search, then add cosine similarity and field boosting. From there we move through SVD/LSA and BERT embeddings to vector search. The TextSearch class built during the workshop became the basis for the minsearch library used in later RAG and agent workshops. What we cover: TF-IDF text search with sklearn, cosine similarity, field boosting and keyword filtering A reusable TextSearch class that becomes minsearch Vector search using SVD and NMF embeddings BERT embeddings for semantic search that respects word order Originally delivered at a DataTalks.Club live session in 2024, updated in 2026 with refreshed examples and tooling. Links Resources not included in the workshop materials list: DataTalks.Club YouTube channel The search engine you will build We take two approaches to search over the same FAQ data: flowchart LR DOCS["FAQ documents<br/>DE/ML/MLOps Zoomcamp"] TEXT["Text search<br/>TF-IDF + cosine similarity<br/>field boosting + filtering"] VEC["Vector search<br/>SVD / NMF / BERT embeddings<br/>cosine similarity"] CLASS["TextSearch class<br/>(became minsearch)"] DOCS --> TEXT DOCS --> VEC TEXT --> CLASS VEC --> CLASS Text search uses TF-IDF vectorization, weights the question field three times as much as the others, and filters by keyword. Vector search replaces sparse representations with dense embeddings (SVD, NMF, then BERT) to handle synonyms and word order. Both paths share the TextSearch class we build along the way, which combines TF-IDF across multiple fields with boost weights and keyword filters.

May 21, 2024
search rag llm-engineering information-retrieval