AI engineer interview questions, by stage, and what a strong answer shows

We did not source a published standard. Anthropic's careers page says that for technical roles it uses live coding tools such as Colab and CodeSignal, and that candidates can look things up ⁹. Job postings say more about content. Databricks' US posting for an AI Engineer in forward deployed engineering asks for experience building GenAI applications including RAG and multi-agent systems, and for expertise in deploying production-grade GenAI applications, including evaluation and optimizations ¹. Scale AI's Applied AI Engineer posting for its Global Public Sector team (listed for Doha and London) lists designing and maintaining evaluation frameworks as a responsibility, and gives building generative or agentic AI applications or setting up evaluation pipelines as examples of the 2+ years of production AI/ML experience it asks for ².

We assume interviews test what the job asks for, so we grouped the practice questions the same way. That is an assumption based on two postings read on October 23, 2026, not a measurement of real loops, which may differ by company, level and team. Databricks says its role is intended for experienced engineers and not for internship, new graduate or entry-level applicants ¹, and the Scale AI posting asks for 7+ years of post-graduation engineering experience ². Ask your recruiter what each round covers.

Table 1. Scroll sideways for every column.

Expect ordinary programming with a model in the loop. These are practice prompts; none comes from a real company's interview.

• "Call an LLM API and return JSON that matches a schema. What happens when the output is malformed?" A strong answer validates the output against the schema, retries a bounded number of times with backoff, does not retry errors that cannot succeed, logs the raw response, and says what the caller gets when every attempt fails.
• "Build a minimal retriever: split documents into chunks, embed them, return the top k for a query." A strong answer explains your chunk size and overlap, uses cosine similarity or an equivalent, and says how you would test retrieval. It also notes when retrieval may be unnecessary: Anthropic said in September 2024 that a knowledge base smaller than 200,000 tokens, about 500 pages, can go directly in the prompt with no need for RAG ¹⁰. Check the context window of the model you would actually use.
• "Run 10,000 documents through a model with a concurrency limit, without losing progress if the job dies." A strong answer uses async tasks or a worker pool, respects rate limits, makes each item safe to repeat, saves results as it goes, and estimates cost first.
• "Write a loop where the model can call two tools. How do you stop it running forever?" This is the reason-and-act pattern from the ReAct paper, where a model generates reasoning steps and actions in an interleaved way, and the actions let it interface with external sources such as knowledge bases ¹¹. A strong answer sets a step budget, validates tool arguments, returns tool errors to the model as text, and logs every step.
• "How do you test a function whose output comes from a model?" A strong answer tests the deterministic parts normally, mocks the model in unit tests, and keeps a small suite of real-model checks that assert properties (valid structure, required fields) rather than exact strings.

System design rounds turn a vague product request into an architecture, and the strongest signal is restraint. Anthropic's December 2024 guide to building agents (it now carries a note that much of the tooling it describes has changed) separates workflows, where LLMs and tools are orchestrated through predefined code paths, from agents, where the LLM dynamically directs its own process and tool usage. It recommends finding the simplest solution possible and increasing complexity only when needed, because agentic systems often trade latency and cost for better task performance ³.

• "Design a question-answering assistant over a company's internal documents." A strong answer asks who uses it, how fresh the data must be and who may see which document, then proposes a baseline and names upgrades and how to test each. In Anthropic's own experiments, adding explanatory context to each chunk before embedding cut the top-20 retrieval failure rate from 5.7% to 3.7%; combining it with keyword (BM25) search reached 2.9%, and adding reranking reached 1.9% ¹⁰. You need not recite the figures, but know that hybrid search and reranking exist and that gains must be confirmed on your own data.
• "Answers ignore facts that were in the retrieved context. What do you check?" A strong answer knows models can use information worse when it sits in the middle of a long input: the "Lost in the Middle" study found performance often highest when relevant information was at the beginning or end of the input and significantly worse when it was in the middle, in the models it tested in 2023 ⁵. Practical responses, which are our suggestion and not the paper's, are fewer, better-ranked chunks with the most relevant first, then measuring the change.
• "When would you build an agent instead of a fixed workflow?" A strong answer picks a workflow for well-defined tasks, reserves agents for open-ended problems that need flexibility, and admits the cost in latency and money ³.
• "An assistant can read email and send replies. What could go wrong?" The expected answer is prompt injection. In indirect prompt injection, attackers inject instructions into data an application is likely to retrieve ⁶, and NIST's generative AI profile says generative AI is itself vulnerable to attacks like prompt injection ¹². A strong answer treats retrieved text as untrusted, gives tools the least privilege that works, requires human approval for irreversible actions, and logs every action.
• "The feature works but costs too much and is too slow. What do you change?" A strong answer measures where the tokens and seconds go, then considers a smaller model for easy cases, shorter prompts, caching and fewer calls, and re-runs the test set after every change.

Both postings above name evaluation, one in its requirements and one in its responsibilities ^1,2. We would expect to be asked how you would know a system is good, not only how to build it.

• "How do you know your RAG system is any good?" A strong answer evaluates retrieval and generation separately. The Ragas paper proposes metrics for three dimensions: whether the retriever finds relevant, focused passages, whether the model uses them faithfully, and the quality of the generation itself, without relying on human-annotated ground truth ¹³. It then builds a test set that mirrors real traffic and includes edge cases, as Anthropic's evaluation guide advises ¹⁴. OpenAI's guide warns against "vibe-based evals", judging quality by impression ¹⁵.
• "Would you use an LLM to grade outputs?" A strong answer says yes, with controls. A 2023 study of LLM judges examined position, verbosity and self-enhancement biases, and reported that strong judges such as GPT-4 reached over 80% agreement with human preferences, the same level of agreement as between humans ⁴. OpenAI suggests pairwise or pass/fail grading, controlling for response length and validating against human labels ¹⁵. Anthropic's evaluation guide notes, in its grading code examples, that it is generally best practice to evaluate with a different model from the one that generated the output ¹⁴.
• "Quality dropped after a model or prompt change. Walk me through debugging." A strong answer starts from logged inputs and a fixed test set, not from rewriting the prompt. Run old and new versions on the same cases, sort the failures into categories, change one thing at a time (prompt, model version, retrieval index, source data), and roll back if the cause is unclear.
• "How do you reduce hallucinations?" NIST defines confabulation as confidently stated but erroneous or false content, and says it is a natural result of how generative models are designed ¹². A strong answer therefore does not promise to eliminate it. It reduces and contains it: grounding in retrieved sources, citations, permission to say "I don't know", a verification step, measured faithfulness, and human review where the stakes are high.
• "How do you test an agent?" OpenAI's guide recommends testing tool selection, argument extraction and handoffs between agents ¹⁵. Anthropic recommends extensive testing in sandboxed environments with guardrails, because autonomy can compound errors ³.

We found no published breakdown of behavioral rounds for AI engineers, so these are practice questions, not reported ones. They focus on unclear requirements and trade-offs: tell me about a time a model feature failed in production, a time you pushed back on shipping, or how you would explain a nondeterministic system to someone who wants a guarantee. In our judgment, a strong answer is specific and uses your own numbers. Do not invent metrics, because interviewers may follow up on them.

Classic machine learning theory depends on the role. The Databricks posting asks for extensive hands-on industry data science experience with tools such as pandas, scikit-learn and PyTorch ¹, so for that role we would prepare for questions on evaluation metrics, overfitting and data splits. Scale AI's posting emphasizes building and deploying production AI applications and evaluation frameworks, and also mentions fine-tuning ². We would expect more theory the more a role leans toward training models; our comparison of AI engineer and ML engineer roles explains the split. Whatever the role, we suggest being able to answer in two sentences: what is an embedding, what does temperature change, what limits a context window, and when would you choose prompting, retrieval or fine-tuning.

It depends on the company and the round, so ask. Anthropic's candidate guidance is one published example: it encourages using Claude to prepare, expects live interviews without AI assistance unless Anthropic indicates otherwise, and asks candidates to do take-homes without Claude unless told otherwise ⁷. Rules can differ inside one company. A January 2026 Anthropic engineering post says the take-home it uses to hire performance engineers explicitly allows AI tools, and its author, who redesigned the test, wrote that people should be able to distinguish themselves in a setting with AI, like they would have on the job ⁸. Treat both as examples from one employer, not a norm.

Build one small system and be able to defend every choice in it. A document question-answering app or a tool-using assistant is enough, if it has a test set you wrote yourself (a few dozen cases, including some the system should refuse), a recorded baseline score, and a log of each change with the score before and after. That log gives you real numbers, failures and stories for the system design, evaluation and behavioral rounds at once. Then practice out loud: narrate trade-offs, and when you do not know something, say how you would find out. This is our suggestion, not a researched formula. For the wider career picture, read our guide to moving from software engineer to AI engineer.