Prompt Engineering — Design, Test, Revise
The four-part triage prompt worked on the helmet email, so the owner switched it on for the whole inbox. By lunchtime it had filed "please fix my wheel and refund the broken bell" as repair and a spam newsletter as order. One good result had felt like proof. It was a sample of one.
The rule
Prompt engineering is design, test, revise — measured on real examples.
- Design — write a prompt with a clear purpose and all four parts.
- Test — run it on a set of realistic inputs where you already know the right answer.
- Revise — study the failures, change one thing, and test again.
The word engineering matters. A builder does not declare a bridge safe because one car crossed it. A prompt is judged the same way: by how often it succeeds across varied cases, including awkward ones.
A small test set for the triage prompt might look like this:
Email Expected
"My gears click on every hill." repair
"Where is order 4471?" order
"Please fix my wheel and refund the broken bell." refund
"Do you have parking for customers?" otherTwenty or thirty such rows, chosen to include the tricky cases, tell you more than any amount of rereading your own prompt.
The shortcut
Treat every prompt change like an experiment: one change, full re-run, compare the score.
Measure
Run the prompt on all 20 test emails. It gets 16 right. Four misses, three of them emails that ask for both a repair and a refund.
Changing one thing at a time is what makes the result readable. If you rewrite the whole prompt and the score moves, you cannot tell which edit helped and which hurt. Re-running the full set matters too: a fix for one case can quietly break another.
Good prompt engineering is also plain good writing. Most failures come from ambiguity a human would also stumble on: undefined labels, missing facts, unstated formats. The techniques in the rest of this course are tools for removing that ambiguity, and the test set is how you know they worked.
Drills
Answer fast — trust the rule.
A prompt worked perfectly on one example. What should you conclude?
Why change only one thing between test runs?
Which test set is most useful for the triage prompt?
After fixing a failure, why re-run the whole test set and not just the failing email?
Remember
- Prompt engineering is design, test, revise, measured on real examples.
- Shortcut: one change, full re-run, compare the score.
- Trap: trusting a prompt because it worked once.