Domain 4: Prompt engineering and structured output

You force a model's output through a schema and the parse errors stop. Every record is well formed, every field is there. A few weeks later someone notices that some of those perfectly valid records say things that aren't true. Nothing failed, and the output is still wrong.

That gap, between valid and correct, runs through all of Prompt Engineering & Structured Output. It is 20% of the exam, roughly 12 questions, and it rewards having shipped something with a model in it and lived with what came out.

Read it as one pipeline

This domain is easy to hold as a sequence, because its modules follow the life of a single request, from the instruction going in to the bill coming out:

1. 1Explicit criteria (4.1). Words like important or severe feel precise to whoever wrote them and mean something slightly different to the model every run. Name the boundary instead.
2. 2Few-shot examples (4.2). When stating the boundary still isn't enough, show it. The module is as much about which symptoms examples fix as which they don't.
3. 3Structured output (4.3). Getting a guaranteed shape back, and stopping that guarantee from inventing data.
4. 4Validation and retry (4.4). What to check once the output is well formed, and which failures a second attempt can repair.
5. 5The Batch API (4.5). Trading waiting time for a lower bill, and when that trade is off the table.
6. 6Multi-pass review (4.6). Keeping quality even across a big job, and who should be trusted to judge it.

Explicit criteria and few-shot examples

Picture a support inbox where the model is told to "flag anything urgent". To one run, urgent means a customer who can't log in. To the next, it means any message with an exclamation mark. Neither run is wrong by the instruction's own standard, because the instruction never said. The fix the exam wants is a boundary with edges: which situations count, and which look similar and don't.

The same module covers automated reviewers. If one category of finding is noisy, people stop trusting the categories that aren't. One unreliable label can sink a whole tool, and tuning the good categories harder does not fix it.

Few-shot examples pick up where criteria leave off. The exam tests how many to use and what each one has to contain, and the content matters more than the count. An example that shows only its answer teaches the model to copy, while one that shows why teaches it something it can reuse. The exam also asks when examples are the wrong tool, because some symptoms that look like inconsistency have a different cause.

The guarantee and its limits

Asking for JSON in a prompt can still return something else, and a pipeline cannot be built on that. Forcing the output through a schema removes that failure. The module spends at least as long on what the guarantee doesn't cover.

Under time pressure this is easy to answer backwards. Making a field mandatory feels like it should make the model more careful. The fix is to give the model an honest way to say it doesn't know.

Which failures a retry can fix

Once output is well formed, what's left is semantic, and the exam wants to know whether you can tell the two kinds of semantic failure apart. Some can be repaired by sending the right information back: a total that doesn't add up, a value in the wrong format. Some can't, because a retry only reworks what the model was given in the first place. If the source never held the answer, asking again gets a different guess.

A retry that only says "try again" hands the model nothing it didn't already have, which is why it appears as a distractor. A useful retry names what failed, such as the total that doesn't add up, and sends it back with the original document.

Paying for it, and checking it

The Batch API halves the price of a job in exchange for giving up any promise about when it finishes. Ask who is waiting on the result. A nightly job that tags yesterday's tickets can wait. A reply to someone in a chat window can't.

Multi-pass review is the last step. A model is a poor judge of work it produced in the same conversation, because it is still carrying the logic it used to produce that work. And one review over a large batch spreads attention thin in the same way Domain 1 describes for decomposition. If one of those two ideas clicks, the other follows from it, which is one reason Spaced mixes domains on the same evening.

What's left for the modules

The modules go deeper on each step of that pipeline: how many examples to use and what each one has to show, how to stop a schema forcing invention, what a useful retry sends back, and when a batch is the right call. Full Access opens all six modules, and the practice runs itself around your evenings.