What Does an AI Engineer Do?

This is chapter 1 of The AI Engineering Field Guide.

The title "AI engineer" is young, and companies use it loosely. Before you spend months preparing to move into this role, you deserve a clear picture of the work.

In this chapter I'll give you my definition of the job and walk through the day-to-day responsibilities. After that, I'll put the role next to data science and ML engineering. Finally, I'll back it all up with what companies write in thousands of real job descriptions.

The core responsibility

Strip away the buzzwords and the job is this: integrating AI into a product in a way that solves a real user problem.

In practice, the job has two sides:

• You work with models from providers like OpenAI and Anthropic through their APIs.
• You work with product managers to find real user problems, because good AI engineering starts with the problem and treats AI as the tool, not the other way around.

Projects that start with "AI is cool, let's use it somewhere" tend to die in a demo.

Notice what's missing from this definition: models. AI engineers don't create them from scratch, don't design custom architectures, and don't spend their days on heavy feature engineering. The model already exists, so your job is everything that turns it into a product.

Calling the API is the easy part

For a toy demo, one API call is the whole system. Suppose you want to extract attributes from uploaded pictures: write a prompt, call the API, return the response. You can build that in an afternoon.

A professional implementation of the same feature looks different.

Each of these practices shows up in real production systems:

1. Prompt tests, with known inputs and expected outputs, so you notice immediately when a change breaks something
2. An evaluation dataset: a fixed set of inputs that produces a quality number you can track over time
3. Iterating on the prompt: change it, rerun the evaluation set, confirm nothing got worse
4. A careful rollout, with an A/B test on a small share of users before releasing to everyone
5. Production monitoring: a dashboard for error rates and failure cases once real traffic arrives
6. Log collection, so you can look at recorded inputs and outputs to find where the system disagrees with user expectations
7. Human annotators who sample production data, verify quality, and feed problem cases back into the evaluation set
8. Model update checks: when the provider ships a new model, you rerun the evaluation set and look for regressions
9. Prompt versioning and experiment tracking, with tools like Git and MLflow
10. Feedback from users, both explicit (thumbs up and down) and implicit (a user correcting the output by hand)

That gap between step one and step ten is the actual job. You can see a small but real version in the simple-sell marketplace example on GitHub, a working implementation with tests and CI/CD.

The complexity ladder

The simplest version runs in one step: user input goes into a prompt, the prompt goes to an LLM API, a response comes back. From there, complexity climbs quickly.

Add a search layer and you get RAG, which I put at roughly five times harder than the simple case. Now you build data pipelines and run a search engine, vector or text-based. Retrieval adds another layer, together with the infrastructure and reliability work that comes with any data system.

Add tools and multiple model calls, and you get agents, which I put at roughly ten times harder than the simple case. The work now includes tool calls and several LLM rounds per task. On top comes multi-step evaluation, trace instrumentation, and careful management of how each new tool rolls out.

This progression matters when you plan work or estimate projects. A prompt feature behaves like a script, and a RAG feature behaves like a service, so budget accordingly.

The role versus data science, ML, and software engineering

Next to an ML engineer, the role is close. The dividing line is model ownership: ML engineers own model weights, while AI engineers use third-party models through APIs. Mentally, swap a call to a locally hosted model for a call to OpenAI, and the rest of the job barely changes.

That's also why ML engineering is the easiest transition into AI engineering. The main addition is evaluation skills: knowing how to measure whether a probabilistic system works.

Next to a data scientist, the difference is bigger. Data science centers on model creation, where data scientists translate requirements into ML problems, design datasets, and train models. AI engineers do science and engineering, but without real modeling, so most of the effort goes into prompt tuning instead of model training.

A team doesn't necessarily need a separate data scientist for this work. Data scientists moving this way need to add engineering skills such as tests, CI/CD, and deployment.

Next to a software engineer, the difference is the AI layer on top. AI engineering is engineering, and the usual disciplines of building reliable services apply in full. What's new is the probabilistic component. Prompts, retrieval, evaluation, and monitoring of model behavior had no equivalent in classical software engineering.

In bigger organizations, AI responsibilities are often split between existing data scientists and ML engineers. Data scientists take prompt tuning and validation frameworks. ML engineers take the engineering and deployment side, and eventually a dedicated AI engineer gets hired. And LLMs aren't the answer to every problem, because traditional ML work continues alongside.

The data from 4,894 job descriptions

My view of the role is one input, and the data is the other. I extracted 33,957 responsibilities from 4,894 job descriptions collected in six monthly scrapes between February and June 2026. I categorized them with AI assistance and measured how many jobs mention each area. Most of those responsibilities (73.4%) belong to AI-first roles, 24.3% come from AI-support roles, and only 2.4% from traditional ML or unclassified roles.

Here's how common each responsibility area is:

• Building AI systems: 98.1% of jobs
• Deployment and production: 78.3%
• Infrastructure and platforms: 69.2%
• Evaluation and quality: 68.5%
• Collaboration and communication: 68.2%
• Monitoring and maintenance: 64.3%
• Integrating APIs and services: 62.4%
• Performance optimization: 57.2%
• Data processing: 49.0%
• Agents and agentic workflows: 48.4%
• RAG and retrieval: 42.0%
• Security and compliance: 40.1%
• Research and experimentation: 34.9%
• Working with customers: 26.8%
• Frontend and user interfaces: 17.1%
• fine-tuning: 13.8%
• Prompt engineering as a named requirement: 10.0%
• Self-hosting models: 2.5%

Three things stand out.

Building dominates the list more than any other area. Some form of "build AI systems" appears in 98.1% of jobs, which makes it the definition of the role rather than one responsibility among many.

Production is most of the work. Put deployment (78.3%), infrastructure (69.2%), and monitoring (64.3%) together, and "make it run reliably" rivals "make it work" as the core task.

The hype items are rare. fine-tuning shows up in 13.8% of jobs and self-hosting in 2.5%. Prompt engineering appears as a named requirement in only 10.0%, while integrating provider APIs appears in 62.4%. The market runs on APIs, not on training or hosting models.

The data also shows where the role is heading. Between the February scrape (895 jobs) and the June scrape (888 jobs), agents climbed from 41.8% to 54.8% of jobs. Agentic work is now a majority responsibility, and it got there in five months. Monitoring rose to 70.6% and security to 44.9%, while research and experimentation fell from 43.1% to 30.1%. In short: the job is becoming more agentic and more operational, with less exploration.

Even the vocabulary agrees with the data: across all 33,957 responsibilities, the most common words are data and build. Systems, design, and implement follow close behind. These are verbs of construction, not research.

A note on job titles

Titles are messier than the work. In the dataset, "AI Engineer" is the most common title with 222 postings, followed by Senior AI Engineer (161). Next come Applied AI Engineer (62) and AI/ML Engineer (53), plus the Staff, Lead, and Principal variants. You'll also find AI Product Engineer and Forward Deployed AI Engineer.

The title alone doesn't tell you what the job is. Some roles labeled "AI Engineer" build platforms and pipelines near AI instead of working on AI systems directly, so read the responsibilities, not the headline. In this dataset, 91.9% of postings with "AI Engineer" in the title describe work on AI systems.

The always-fresh version of the data

I adapted this chapter from the open-source AI Engineering Field Guide. The job description dataset, the full analysis, and the interview reports are all in the repo, and I update them as new scrapes come in. The numbers there are always fresher than the ones printed here.

Up next in the series: The AI Engineering Interview Process.

Build this

The fastest way to understand the role is to do a small version of it. Pick an idea from the projects page. Good candidates are a RAG search over documents you care about or a small agent with a couple of tools. Take it past the demo stage by adding an evaluation set, setting up basic logging, and iterating on the prompt until the metric stops improving. That exercise touches every practice from the "calling the API" section, and it doubles as interview material.

When you're ready for the rest of the series, become a member to unlock the negotiation chapter.