Inference Engineering Book Club - Kickoff

We met on 10 August 2026 to start a shared reading of Inference Engineering by Philip Kiely. We use the book to connect product decisions to the systems behind a deployed model. We start with model selection and hardware. Later sessions cover serving, optimization, modalities, and production systems.

If you're joining after the kickoff, use this checklist to start. You can follow the reading and add notes on the AI Shipping Labs book page.

The book's role

We usually start with hosted model APIs because they're easy to use, but they hide the decisions behind the API. Those decisions include cost, latency, hardware, and serving. We may need to run a model ourselves when data must stay in a particular region or when one provider isn't enough. Latency and inference cost are also reasons to evaluate self-hosting.

The book starts with product decisions and ends with production operations. Between those points, it explains the model and hardware choices that determine how we serve and optimize inference.

Club format

We read a few chapters before each meeting, compare the parts that made sense and the parts that didn't, then choose the next assignment. We mark progress and add rough notes on the book page. We can later combine those notes with the meeting discussions into a shared chapter summary.

The group also wants to connect the theory to small experiments. A useful experiment compares one model or technique under two conditions. Record one or two measures, such as latency and memory use. Add throughput, cost, or output quality when they matter for the experiment. We don't need a frontier model to learn the underlying idea.

Next actions

Use this checklist before the first discussion.

[ ] Get the book from the publisher's page.
[ ] Read chapters 0, 1, and 2. Start with the inference stack, then study the decisions before optimization and the model bottlenecks.
[ ] Add one note or question to the matching chapter on the book page. A rough question is enough.
[ ] Choose one concept to make concrete. Draw a diagram, write a small program, or explain it aloud. Attention, the KV cache, model selection, and GPU memory were all suggested as good candidates.
[ ] If you run an experiment, record the model, hardware, input, output, and measurement so someone else can reproduce the comparison.

Reading strategy

Read a chapter once to understand its structure. Return to the sections that matter for your use case. Ask an AI assistant to explain a difficult passage or quiz you after you have read it. Turn one idea into a diagram, a small program, or an explanation in your own words. Those actions expose gaps that a familiar-looking paragraph can hide.

Resources

These links support the reading and the exercises discussed in the session.

Inference Engineering, including the publisher's digital edition.
Inference Engineering interactive guide with diagrams, quizzes, and exercises.
Fast and Efficient LLM Inference with vLLM for hands-on background on serving and optimization.
The Feynman technique for testing whether you can explain a concept simply.

Recording

Watch the kickoff recording on YouTube.

Recording chapters

Use these timestamps to jump to the main parts of the discussion.

Continue with Week 1

The next session covered chapters 0, 1, and 2. Read the Week 1 recap or browse the Inference Engineering Book Club series.