All Tags

Tag

llm-engineering

7 items tagged with "llm-engineering"

Event

From RAG to Agents: Implementing Agentic Search

We build a classic RAG system over real documentation, then evolve it into an agentic search workflow where the LLM decides what to search for and whether to open a full document. Along the way we implement two tools - search (returns highlighted snippets) and get_file (returns the full document) - and wire them into an agent using both toyaikit and PydanticAI. The knowledge base is the Evidently AI documentation: a real, evolving set of Markdown files that makes RAG genuinely useful, since LLMs can't keep up with library docs on their own. This workshop was originally delivered at DataMakersFest 2026 in Porto, Portugal. There is no recording available. The system you will build The architecture has one LLM agent with two tools and two data sources: flowchart LR USER["User"] AGENT["Agent (LLM)"] SEARCH["search tool<br/>highlighted snippets"] GETFILE["get_file tool<br/>full document"] MINSEARCH["minsearch index<br/>Evidently docs"] FILEINDEX["file_index<br/>filename -> content"] USER -->|question| AGENT AGENT -->|decides which tool| SEARCH AGENT -->|decides which tool| GETFILE SEARCH -->|query| MINSEARCH MINSEARCH -->|snippets| SEARCH GETFILE -->|filename| FILEINDEX FILEINDEX -->|full text| GETFILE SEARCH -->|results| AGENT GETFILE -->|results| AGENT AGENT -->|answer| USER The agent has two tools and decides which one to use. search returns short highlighted snippets so the agent can decide which documents are worth reading. get_file returns the full document so the agent can read it end-to-end. The pattern mirrors how humans read documentation: search, scan snippets, open the promising one. Links Resources not included in the workshop materials list: toyaikit - teaching framework for agents PydanticAI - production agent framework

May 4, 2026
rag ai-agents llm-engineering search tooling-architecture
Event

Build Your Own AI Coding Agent

We build a coding agent from scratch: tool calls, an agentic loop, a skills system that loads reusable instruction files on demand, and a final version running on PydanticAI. The agent can read and write files, run shell commands, search the codebase, and load specialized skills - all inside a structured Django project template. This workshop was originally delivered at MLCon 2026 Amsterdam. There is no video recording available. The agent you will build The architecture of the final agent: flowchart LR USER["User"] RUNNER["Runner<br/>ToyAIKit / OpenAI Responses"] LLM["LLM<br/>gpt-5.4-mini"] TOOLS["Tools<br/>read_file, write_file<br/>see_file_tree<br/>execute_bash_command<br/>search_in_files"] SKILL["SkillTool"] LOADER["SkillLoader"] SKILLS["SKILL.md files"] USER -->|prompt| RUNNER RUNNER -->|model call| LLM LLM -->|tool call or response| RUNNER RUNNER -->|execute tool| TOOLS RUNNER -->|load skill| SKILL SKILL --> LOADER LOADER -->|read| SKILLS RUNNER -->|answer| USER The agent runs inside a Django project template. It has five core tools for interacting with the filesystem and shell, plus a skills tool that loads extra instructions from SKILL.md files when the agent decides they are relevant. The workshop progresses through four parts: understanding tool calls, building the coding agent, adding skills, and porting to PydanticAI. Links Related workshops and resources: Previous coding-agent workshop - the v1 version without skills agent-skills workshop - the standalone skills workshop

Apr 21, 2026
ai-agents llm-engineering agent-systems tooling-architecture
Event

Skills.md from Scratch: Build a Skill-Driven Coding Agent

A newer, combined version of this workshop is available: Coding Agent with Skills. We start from the coding agent in the prerequisite workshop. We turn it into a general-purpose coding agent with two reusable behavior layers: skills and slash commands. The agent loads a skill when your request matches its description. You invoke a command with a leading slash, and the system renders it into a prompt before the agent acts. Links External resources for this workshop: Starting notebook GitHub fetch skill Prototype implementation Prerequisite coding-agent workshop ToyAIKit OpenCode OpenCode skills documentation AgentSkills spec The agent you will build By the end you'll have built a system that looks like this: flowchart LR USER["User"] RUNNER["ToyAIKit runner<br/>OpenAI Responses"] LLM["OpenAI model"] CODETOOLS["Coding tools<br/>read, write, tree, bash, search"] SKILLTOOL["skill(name) tool"] LOADER["SkillLoader"] SKILLS["skills/*/SKILL.md<br/>scripts and templates"] COMMANDS["commands/*.md"] COMMANDTOOL["execute_command(name, args)"] USER -->|plain request| RUNNER USER -->|/command| RUNNER RUNNER --> LLM RUNNER --> CODETOOLS RUNNER --> SKILLTOOL SKILLTOOL --> LOADER LOADER --> SKILLS RUNNER --> COMMANDTOOL COMMANDTOOL --> COMMANDS The project stays small enough to understand in a notebook, but it mirrors the pieces used by real coding agents. The coding tools let the model read, write, search, and run commands. The skill loader turns SKILL.md files with YAML frontmatter into tool-loadable instructions. The command loader turns markdown files like review.md or test.md into reusable prompt templates.

Jan 16, 2026
ai-agents llm-engineering agent-systems tooling-architecture
Event

Building Safe AI Agents with Guardrails

We start with a DataTalks.Club Data Engineering Zoomcamp FAQ assistant. Then we add checks that keep the agent on topic and block unsafe responses. The checks also show how to cancel wasted work when a guardrail fails. We first use the OpenAI Agents SDK for its built-in guardrails. Then we rebuild the same idea with tools and plain asyncio, so it works with other agent frameworks. Links The external resources: Related course: AI Bootcamp: From RAG to Agents FAQ data used by the agent AI Hero email course for the docs.py loader OpenAI Agents SDK guardrails documentation The notebook you will build By the end, you wrap a tool-using FAQ agent in guardrails: flowchart LR USER["User question"] INPUT["Input topic guardrail"] FAQ["FAQ assistant agent"] SEARCH["search_faq tool<br/>minsearch index"] OUTPUT["Output safety guardrail"] ANSWER["User-facing answer"] OPENAI["OpenAI model calls"] USER --> INPUT INPUT -->|passes| FAQ INPUT -->|trips| ANSWER FAQ --> SEARCH FAQ --> OPENAI INPUT --> OPENAI OUTPUT --> OPENAI FAQ --> OUTPUT OUTPUT -->|passes| ANSWER OUTPUT -->|trips| ANSWER The base agent can already search the FAQ, but it tries to answer unrelated questions too. The input guardrail blocks questions outside the course domain. The output guardrail checks the agent response for policy problems. Examples are promising deadline extensions or writing homework for a student. In the later parts we show the same checks as tools. We also build a small async runner that can cancel work when a guardrail trips.

Jan 6, 2026
ai-agents llm-engineering agent-safety tooling-architecture async-control
Event

Build a Course FAQ Agent with MCP, PydanticAI and OpenAI

We build a course FAQ assistant from the bottom up. First we expose a plain Python search(query) function to the OpenAI Responses API. Then we turn the same idea into a reusable agent loop and compare toyaikit, OpenAI Agents SDK, and PydanticAI. Finally we move the FAQ tools behind an MCP server. From there a notebook, PydanticAI, Cursor, and VS Code can all reach them. Links The main resources: AI Bootcamp: From RAG to Agents Prerequisite workshop: Building a Coding Agent Data Engineering Zoomcamp FAQ source document Parsed FAQ JSON FAQ parsing notebook The system you will build The final setup looks like this: flowchart LR NOTEBOOK["Jupyter notebook"] OPENAI["OpenAI Responses API"] FRAMEWORKS["Agents SDK<br/>PydanticAI"] MCPCLIENT["MCP clients<br/>toyaikit, PydanticAI, Cursor"] MCPSERVER["FastMCP server<br/>SSE or stdio"] TOOLS["FAQ tools<br/>search, add_entry"] INDEX["minsearch index<br/>FAQ JSON"] NOTEBOOK -->|function calling| OPENAI NOTEBOOK --> FRAMEWORKS FRAMEWORKS -->|tool calls| TOOLS NOTEBOOK -->|MCP client| MCPCLIENT MCPCLIENT -->|MCP protocol| MCPSERVER MCPSERVER --> TOOLS TOOLS --> INDEX The FAQ data comes from the Data Engineering Zoomcamp FAQ. The first half of the workshop keeps the tools inside the notebook so you can see the agent loop directly. The second half moves the same tools into mcp_faq/, which makes them reusable by any MCP client.

Sep 1, 2025
ai-agents llm-engineering agent-systems tooling-architecture mcp rag
Event

Building a Coding Agent: Python/Django Edition

For a newer, combined take, see Coding Agent with Skills. We build a small project bootstrapper for Django, a coding agent that takes a plain-language app request and copies a working Django template. From there it reads and writes files through tools, iterating until the generated app runs. The first version uses the OpenAI Responses API through ToyAIKit. Then we try the same idea with OpenAI Agents SDK, PydanticAI, Anthropic, and Z.AI. Links The main resources: ToyAIKit Django template repo Todo app made with Z.AI Related course: AI Bootcamp: From RAG to Agents Related workshop: Hands-on with AI Agents and MCP The app you will build You chat with the coding agent in a notebook. It's backed by an LLM and a small set of filesystem tools. You give it a request like to-do list, and the agent edits a copied Django template and leaves you with a project you can run. flowchart LR USER["You<br/>short app request"] CHAT["Jupyter chat UI<br/>ToyAIKit"] RUNNER["Agent runner<br/>Responses API or framework"] TOOLS["AgentTools<br/>read, write, tree, grep, bash"] DJANGO["Copied Django template<br/>project folder"] LLM["LLM provider<br/>OpenAI, Anthropic, Z.AI"] USER -->|type request| CHAT CHAT --> RUNNER RUNNER -->|tool calls| TOOLS TOOLS -->|modify files| DJANGO RUNNER -->|messages and tools| LLM DJANGO -->|make run| USER Two screenshots show what the finished workshop output looks like. The first one shows the notebook chat after the agent plans and starts calling file tools: The second one shows one of the generated Django todo apps: Result The simplest version is intentionally small. It runs in Jupyter, uses local filesystem tools, and edits one copied Django project folder. That's enough to understand how larger coding agents work under the hood. The same four steps scale to any size of agent: Prepare a template. Expose the right tools. Give the model precise instructions. Iterate on the generated code.

Aug 14, 2025
ai-agents coding-agents django python llm-engineering
Event

Build Your Own Search Engine

We build a search engine from scratch over DataTalks.Club Zoomcamp FAQ documents. We start with TF-IDF text search, then add cosine similarity and field boosting. From there we move through SVD/LSA and BERT embeddings to vector search. The TextSearch class built during the workshop became the basis for the minsearch library used in later RAG and agent workshops. What we cover: TF-IDF text search with sklearn, cosine similarity, field boosting and keyword filtering A reusable TextSearch class that becomes minsearch Vector search using SVD and NMF embeddings BERT embeddings for semantic search that respects word order Originally delivered at a DataTalks.Club live session in 2024, updated in 2026 with refreshed examples and tooling. Links Resources not included in the workshop materials list: DataTalks.Club YouTube channel The search engine you will build We take two approaches to search over the same FAQ data: flowchart LR DOCS["FAQ documents<br/>DE/ML/MLOps Zoomcamp"] TEXT["Text search<br/>TF-IDF + cosine similarity<br/>field boosting + filtering"] VEC["Vector search<br/>SVD / NMF / BERT embeddings<br/>cosine similarity"] CLASS["TextSearch class<br/>(became minsearch)"] DOCS --> TEXT DOCS --> VEC TEXT --> CLASS VEC --> CLASS Text search uses TF-IDF vectorization, weights the question field three times as much as the others, and filters by keyword. Vector search replaces sparse representations with dense embeddings (SVD, NMF, then BERT) to handle synonyms and word order. Both paths share the TextSearch class we build along the way, which combines TF-IDF across multiple fields with boost weights and keyword filters.

May 21, 2024
search rag llm-engineering information-retrieval