Turn AI ideas into real projects

We take the FAQ agent from the [End-to-End Agent Deployment](/workshops/2026-04-21-end-to-end-agent-deployment) workshop and change how it finds answers. That agent searched the FAQ with minsearch, an in-memory keyword index rebuilt from scratch on every boot. We swap it for semantic vector search that lives in a single SQLite file. Then we host that file on Turso, so the data survives restarts on a free host. The app runs without Postgres, without a rented disk, and without a server billed while it sits idle. A coding assistant drives each step, and we include the prompts so you can reproduce the workflow with any agent. ## The two changes We make two moves, in order. 1. Swap minsearch for [`sqlitesearch`](https://github.com/alexeygrigorev/sqlitesearch). Retrieval becomes semantic, and it persists in one SQLite file instead of memory. Embeddings come from a lightweight ONNX model, so PyTorch stays out of the dependency tree. 2. Host that SQLite file on [Turso](https://turso.tech), a hosted libSQL service with a generous free plan that does not expire. The agent reads it through a local replica. The data survives restarts, and the searches stay fast. ```mermaid flowchart LR subgraph ingest["ingest (offline, once)"] FAQ["FAQ JSON"] --> EMB1["ONNX embed"] EMB1 --> LSH["sqlitesearch<br/>vector index"] LSH -->|writes| TURSO[("Turso<br/>hosted libSQL")] end subgraph serve["serve (every request)"] UI["client"] -->|POST /ask| API["FastAPI"] API --> AGENT["agent loop"] AGENT -->|search tool| SEARCH["sqlitesearch"] SEARCH -->|local reads| REPLICA["embedded replica"] AGENT -->|model call| OPENAI["OpenAI"] end TURSO -.->|syncs down on boot| REPLICA ``` The system splits into two halves that never run at the same time. Ingest is an offline job. It fetches the FAQ and embeds every entry with the ONNX model. Then it builds a vector index and writes it straight to Turso. Serve is the running app. On boot it opens an embedded replica that syncs the data down from Turso once. Every `search` tool call then reads from that local copy. The agent loop, the `search` tool, and the OpenAI call are unchanged from the previous workshop. Only what sits behind the tool is different. I made the change for a practical reason. We deployed the previous workshop to Railway on a trial, and the trial ended. Most free tiers drop Postgres and persistent disk once the trial is over. That wipes anything stored on local disk at the next restart. Turso keeps the data outside the host on a generous free plan. The app can move to any free platform and still find its vectors. ## Links Resources used here: - [End-to-End Agent Deployment](/workshops/2026-04-21-end-to-end-agent-deployment) - the workshop this one builds on. - [sqlitesearch](https://github.com/alexeygrigorev/sqlitesearch) - the vector search library, with the libSQL backend used here. - [LLM Zoomcamp ONNX embedder lesson](https://github.com/DataTalksClub/llm-zoomcamp/blob/main/02-vector-search/lessons/09-onnx-embedder.md) - the source of the lightweight embedder.

Serving an open-source model yourself means renting a GPU and running an inference server on it. In this workshop we take a quantized DeepSeek model and rent a GPU by the hour on RunPod. We serve it with vLLM behind an OpenAI-compatible API. Then we point a small tool-calling FAQ agent at it, the same way we would point it at OpenAI. Most of the code is written by an AI assistant (Codex), and the prompts are quoted verbatim. This was a freestyle session with no prep, so it also surfaces the dead ends: - A model that does not fit the card. - A full container disk. - An SSH session that keeps dropping. - A tool parser that needs a flag we did not set. Each one has a fix worth keeping. ## Links These materials are related: - [Previous workshop: end-to-end agent deployment](/workshops/2026-04-21-end-to-end-agent-deployment) - [RunPod console](https://console.runpod.io/) - [vLLM documentation](https://docs.vllm.ai/) - [Modal vLLM inference tutorial](https://modal.com/docs/examples/vllm_inference) ## The system you build The model runs on the rented GPU, not on your laptop. Everything you write locally talks to it over HTTP, exactly the way it talks to OpenAI. Only the base URL and the API key change. The setup has these pieces: - A GPU pod on RunPod runs the model. - vLLM serves it behind an OpenAI-compatible HTTP API. - RunPod's HTTPS proxy exposes that API, guarded by a bearer token. - The FAQ agent runs on your laptop and calls one `search` tool. ```mermaid flowchart LR AGENT["FAQ agent<br/>(your laptop)"] PROXY["RunPod HTTPS proxy<br/>&lt;pod-id&gt;-8000.proxy.runpod.net"] VLLM["vLLM server<br/>OpenAI-compatible API"] GPU["GPU pod<br/>DeepSeek-R1-Distill-Qwen-14B-AWQ"] FAQ["DataTalks.Club FAQ"] AGENT -->|"POST /v1/chat/completions + Bearer token"| PROXY PROXY --> VLLM VLLM --> GPU AGENT -->|search tool| FAQ ``` The model is [stelterlab/DeepSeek-R1-Distill-Qwen-14B-AWQ](https://huggingface.co/stelterlab/DeepSeek-R1-Distill-Qwen-14B-AWQ), a 14-billion-parameter model quantized to fit a single 24 GB consumer GPU. The agent is the FAQ search agent from the [end-to-end agent deployment workshop](/workshops/2026-04-21-end-to-end-agent-deployment). We port it from the OpenAI Responses API to vLLM's chat completions API. It answers Data Engineering Zoomcamp questions with one `search` tool against the course FAQ. ## Workshop flow We build the workshop in this order: - Rent a GPU by the hour on RunPod and connect over SSH. - Install vLLM and serve a quantized model. - Fit the model on the card, then move the caches to the pod volume. - Port the FAQ agent and get the open model to make tool calls. - Reach the model over the network with an API key. - Make the whole thing reproducible with a one-command deploy. - Shut everything down, including the storage that keeps billing after the pod is gone. The reproducible deploy recreates the part the live session ran out of time to finish. The teardown step is the one I learned the hard way: stopping the pod does not stop the storage bill.

We solve a real AI engineer take-home assignment live. The product is a small investment coaching bot that answers public-company research questions without crossing into personalized financial advice. The flow of the session: - Brainstorm scope and survey data sources. - Build a CLI agent with PydanticAI and OpenAI. - Pivot from a paid market-data API to free SEC EDGAR data after the free tier blocks us. - Clean up the pointless tests the coding agent generated and write test guidelines we actually want to follow. - Add a small evaluation harness with scenarios and an LLM judge. - Wrap the result as a Telegram bot. Every step is done alongside a coding agent. The prompts are included verbatim so you can reproduce the workflow with any agent you like. ## Links Useful resources for this workshop: - [AI engineering take-home assignments research](https://maven.com/p/250595/ai-engineering-take-home-assignments) - [AI engineering field guide: home assignments](https://github.com/alexeygrigorev/ai-engineering-field-guide/blob/main/interview/questions/06-home-assignments.md) ## The app you will build The final app looks like this: ```mermaid flowchart LR USER["User"] TG["Telegram bot"] CLI["CLI app"] AGENT["PydanticAI agent<br/>safety + structured output"] TOOLS["SEC tools<br/>search, snapshot,<br/>filings, digest"] EDGAR["SEC EDGAR<br/>company facts + filings"] OPENAI["OpenAI Responses API"] USER --> TG USER --> CLI TG --> AGENT CLI --> AGENT AGENT -->|tool call| TOOLS TOOLS -->|HTTP JSON| EDGAR AGENT -->|model call| OPENAI ``` The agent has four tools, all backed by free SEC EDGAR endpoints: - `search_company(query)` resolves a ticker or company name to a CIK. - `get_financial_snapshot(ticker_or_cik)` fetches recent annual revenue, profit, balance sheet, and cash facts from companyfacts XBRL data. - `get_latest_filings(ticker_or_cik)` returns recent filing metadata and SEC URLs. - `get_filing_digest(ticker_or_cik, form_type)` pulls the latest 10-K or 10-Q text and extracts business, revenue, risk, and MD&A snippets. The agent has a strict safety boundary baked into its instructions: - no buy/sell/hold recommendations - no position sizing - no predictions of price direction A buy/sell question gets transformed into an educational research brief instead. The evaluation harness lives in `evals/manual/`. It runs a list of scenarios from `scenarios.csv` through the agent and captures the full tool-call trajectory. An LLM-as-judge then scores whether the answer stayed inside the safety boundary and addressed the question.