Sample content for editorial review. Examples are illustrative.
Write down the job in plain language
Before tuning a prompt, describe what the system is supposed to help a person do. Include the information available to it and the conditions under which it should decline or ask for clarification. This description is the beginning of an evaluation plan. Without it, prompt changes tend to optimize whichever example is most memorable in the current conversation.
Collect a small but varied starting set
Thirty examples can be a practical starting exercise, not a claim that thirty is enough to establish reliability. Include common requests, ambiguous inputs, missing evidence, conflicting information, and tasks outside the intended scope. Keep the original input and the expected behavior together. If examples contain sensitive information, use an appropriate review and handling process before placing them in a shared collection.
Describe what good means for each case
Some tasks have a checkable answer. Others need a rubric that separates factual support, completeness, clarity, and compliance with the requested format. Identify failures that should block a release even if the average score improves. A response that sounds more polished can still be worse at the part of the task that matters most to the user.
Keep development and checking distinct
Use one group of examples while developing the prompt and keep another group for checking changes. Record the model configuration and application version with each evaluation run. Compare outcomes by failure type, then inspect a sample manually. If a change fixes one category while damaging another, that tradeoff should be visible before someone approves the release.
Let real use expand the collection
When a user corrects an answer or the system encounters an incident, turn the lesson into an evaluation example after reviewing the data involved. Keep the collection representative rather than allowing it to become a pile of unusual failures. The goal is a repeatable way to ask whether a proposed change improves the actual workflow. Prompt writing becomes easier when the evidence for judging a change already exists.
