Where to go from here
You now have a working agentic search system. The LLM searches, reads snippets, opens documents, and synthesizes answers. Here are the natural next steps. They are covered in the full AI Engineering Buildcamp course.
Adding more tools
The two-tool pattern is a starting point.
Useful additions include these tools:
- A re-ranker that scores and sorts the search results before the agent sees them
- A "list files" tool so the agent can browse the corpus structure
- A "fetch GitHub issue" tool so the agent can pull in community Q&A
Each new tool follows the same pattern. Add type hints, write a docstring, and register it with the framework.
Trying different search backends
minsearch is fine for learning, but real deployments use dedicated search
infrastructure. Elasticsearch and Qdrant are common choices. The search
function signature stays the same, so the agent does not change. You swap the
backend and the agent keeps working.
Evaluating the agent
Before shipping, you need to know whether the agent actually answers correctly.
Common metrics include these checks:
- Hit Rate - does the right document appear in the search results?
- MRR (Mean Reciprocal Rank) - how high does the right document rank?
- LLM-as-judge - use another LLM to score the quality of the answer
Evaluation is where you close the loop. Measure once, change the prompt or the tools, and measure again.
Adding guardrails
An agent that can call tools in a loop can also get stuck in a loop. It can return sensitive information or spend more tokens than you budgeted.
Consider these guardrails for the agent:
- Input/output checks on every tool call
- A maximum number of tool-call iterations
- Content filtering on the final answer
The full course
The full AI Engineering Buildcamp course covers structured exercises and production topics. That includes evaluation, guardrails, and deployment. This workshop compresses two course modules.