Solving a Real AI Engineer Take-Home Assignment Live
Event times are shown in Europe/Berlin. Sign in to use your timezone
Continue with the workshop writeup
Open the canonical pages, recording, materials, and code repo.
We solve a real AI engineer take-home assignment live. The product is a small investment coaching bot that answers public-company research questions without crossing into personalized financial advice.
The flow of the session:
- Brainstorm scope and survey data sources.
- Build a CLI agent with PydanticAI and OpenAI.
-
Switch from a paid market-data API to free SEC EDGAR data after the free tier blocks us.
-
Clean up the pointless tests the coding agent generated and write test guidelines we actually want to follow.
-
Add a small evaluation harness with scenarios and an LLM judge.
- Wrap the result as a Telegram bot.
We do every step with a coding agent. We include the prompts verbatim, so you can repeat the same steps with any agent you like.
Links
Useful resources for this workshop:
The app you will build
The final app looks like this:
The agent has four tools, all backed by free SEC EDGAR endpoints:
search_company(query)resolves a ticker or company name to a CIK.-
get_financial_snapshot(ticker_or_cik)fetches recent annual revenue, profit, balance sheet, and cash facts from companyfacts XBRL data. -
get_latest_filings(ticker_or_cik)returns recent filing metadata and SEC URLs. -
get_filing_digest(ticker_or_cik, form_type)pulls the latest 10-K or 10-Q text and extracts business, revenue, risk, and MD&A snippets.
The agent's instructions set a strict safety boundary:
- no buy/sell/hold recommendations
- no position sizing
- no predictions of price direction
The agent turns a buy/sell question into an educational research brief instead.
The evaluation harness is in evals/manual/. It runs a list of
scenarios from scenarios.csv through the agent and records the full
trajectory (the sequence of tool calls the agent made). An LLM-as-judge
then scores whether the answer stayed inside the safety boundary and
answered the question.
Hosted by
Alexey Grigorev
Chief Agent Officer at AI Shipping Labs
Software engineer and machine learning practitioner with 15+ years of experience building production ML systems. I focus on practical, production-grade ML and AI systems, from early prototypes to reliable systems in production.
I'm the founder of DataTalks.Club, a free community that connects tens of thousands of practitioners worldwide, and the creator of the Zoomcamp series, free, code-first programs that have reached 100,000+ learners globally.
At AI Shipping Labs, I'm building the kind of environment that would have accelerated my own career growth. After years of teaching at scale, I wanted something more focused: a space for action-oriented builders who want to turn AI ideas into real projects. The community gives members the structure, accountability, and peer support to ship practical AI products consistently, even alongside their main jobs.
More from AI Shipping Labs
Durability
Agents run for minutes or hours, call slow tools and sometimes wait for a human. If the process dies halfway, the run is gone and you start over, sometimes repeating actions that …
January 12, 2027
MCP
MCP lets you write a tool once and use it from many clients: your own agent, Claude Code, Cursor, VS Code. Our previous MCP workshop is more than a year old, and a lot has changed…
January 5, 2027
Freestyle Workshop
A freestyle workshop: you come with your own topic, problem or question, and we work on it live. We try to turn something useful into working code during the session. Previous fre…
January 4, 2027