Few-Shot Prompting Examples: Build Better Examples for Classification and Extraction

Overview

Few-shot prompting is an in-context learning technique in which you place a small set of input-and-output examples before a new task. The model uses those examples to infer the expected labels, format, tone, or transformation. Unlike fine-tuning, the examples live in the prompt and can be changed without retraining a model.

The quality of a few-shot prompt depends less on the number of examples than on their coverage. A useful set demonstrates the normal case, separates similar categories, and makes important edge-case decisions visible. Poor examples can make a classification prompt inconsistent, encourage verbose extraction, or cause the model to copy irrelevant details.

Start with a written task contract before choosing examples. Define the input, the allowed output, the decision rules, and what the model should do when evidence is missing. For example:

Task: Classify each support message.
Allowed labels: billing, technical, account, other.
Return exactly one label in lowercase.
If the message contains multiple issues, choose the issue that should be handled first.

This contract gives every demonstration a clear purpose. If you are building a broader LLM workflow, compare the prompt against a fixed evaluation set rather than judging it from one successful response. The prompt evaluation framework provides a useful companion process for testing and scoring prompt changes.

How to estimate

Think of few-shot design as an optimization problem with three competing factors: coverage, consistency, and prompt cost. More examples may clarify the task, but they also consume context and can introduce conflicting patterns. Estimate the effect of an example set with a repeatable workflow.

1. List the decisions. Write down every class, field, transformation, or formatting rule the model must apply.
2. Collect candidate examples. Use real or carefully anonymized inputs from the situations your application actually receives.
3. Group by behavior. Separate typical cases, confusing cases, missing-data cases, long inputs, and multi-intent inputs.
4. Choose a small first set. Include at least one clear example for each important behavior, then add contrastive examples where two outcomes are easy to confuse.
5. Measure baseline performance. Run the same evaluation records with zero-shot instructions and with the candidate examples.
6. Calculate the change in cost and quality. Compare input tokens, output tokens, valid-format rate, label accuracy, field-level extraction accuracy, and failure categories.

For a rough prompt-cost estimate, use:

Estimated input tokens per request
= fixed instructions + example tokens + user input tokens

Then multiply the result by the expected request volume. Token counts vary by model, tokenizer, language, and formatting, so treat a character-based estimate as directional only. Before production decisions, inspect token usage through the model or API you plan to use. Pricing and context limits are provider-specific and can change, so record the model name, API configuration, and date alongside your estimate.

Quality should be estimated separately from cost. A prompt that uses fewer examples but produces invalid JSON may create more downstream work than a longer prompt with reliable output. Use a small labeled test set and report results by category rather than relying only on an overall score.

Inputs and assumptions

Select representative inputs

Representative does not mean average. Select examples that cover the distribution of real requests and the decisions that matter operationally. For classification, include common messages, short messages, ambiguous wording, spelling mistakes, and messages that mention more than one topic. For extraction, include complete records, missing fields, repeated entities, conflicting values, and values that resemble the target entity but should not be extracted.

Use contrastive pairs when a boundary is important. Two inputs that differ by one meaningful detail can teach the model why one belongs to a category and the other does not:

Input: I was charged twice for the same order.
Output: billing

Input: The payment page freezes before I can submit my order.
Output: technical

Balance categories without forcing artificial symmetry

If one label has many more real examples than another, do not automatically duplicate the smaller class. Instead, ensure that every allowed label has at least one unambiguous demonstration and that easily confused labels have comparable coverage. Duplicating near-identical examples wastes context and can make a label appear more important than intended.

Label edge cases explicitly

Examples should reveal how the system handles uncertainty. Decide whether missing evidence should produce unknown, an empty field, other, or a clarification request. Do not imply a policy through one inconsistent example. State the rule in the instructions and reinforce it with a demonstration.

Keep formatting consistent

Use the same delimiters, field names, capitalization, and output schema in every example. If the application expects JSON, show valid JSON rather than a code-like approximation. If the model must return only a label, do not include explanations in the example outputs. A structured output mode or schema constraint may also be available depending on the model and API, but the supported format should be verified for the specific configuration.

Worked examples

Classification prompt example

Classify the message using exactly one label: billing, technical, account, other.
Return only the label.

Message: I cannot reset my password because the reset email never arrives.
Label: account

Message: The dashboard shows a blank page after I sign in.
Label: technical

Message: My invoice includes a charge for a plan I cancelled.
Label: billing

Message: Can you add an export-to-calendar feature?
Label: other

Message: {{message}}
Label:

This set covers each label and includes a useful distinction between access problems and application failures. If account and technical issues are frequently confused, add a pair that makes the boundary clearer. Do not add examples merely because they sound different; add them because they cover a decision not already represented.

Entity extraction prompt template

Extract the fields requested below. Use null when a field is not present.
Return valid JSON with exactly these keys: product, version, operating_system.
Do not infer values that are not stated.

Text: The client runs Atlas 4.2 on Windows 11.
JSON: {"product":"Atlas","version":"4.2","operating_system":"Windows 11"}

Text: Atlas crashes on macOS after the latest update.
JSON: {"product":"Atlas","version":null,"operating_system":"macOS"}

Text: {{text}}
JSON:

The second example is important because it demonstrates the missing-version behavior. Add a negative example when a phrase could be mistaken for an entity, such as a company name that is not the product being requested. For long documents, also state whether the output should include every occurrence or one normalized value.

Rewriting and JSON generation

For rewriting, examples should show the degree of change expected. Include one ordinary input, one input with unwanted tone, and one case where meaning must be preserved despite awkward wording. For JSON generation, demonstrate optional fields, arrays, empty values, escaping of quotation marks, and the required response when the input lacks enough information. The examples should be valid instances of the same schema, not a mixture of prose and structured output.

Selection checklist

• Does every allowed class or output behavior appear?
• Are the most easily confused classes represented by contrastive examples?
• Is at least one missing-data or refusal boundary demonstrated?
• Do examples use realistic length, spelling, and terminology?
• Are labels and JSON keys consistent throughout?
• Could any example be interpreted in more than one way?
• Have near-duplicate examples been removed?
• Has the example set been tested on inputs that were not used to create it?

Keep examples separate from your evaluation records. Reusing test inputs in the prompt can make results look better without showing whether the prompt generalizes.

When to recalculate

Revisit a few-shot prompt whenever the task, data, model, or output contract changes. Recalculate token usage when you add examples, lengthen instructions, change delimiters, or begin sending larger user inputs. Recheck quality when new categories are introduced, category volumes shift, or production logs reveal a new failure pattern.

Model updates can change how examples are interpreted, even when the wording remains unchanged. Treat a model or API configuration change as a reason to rerun the same regression set. Also verify supported structured-output behavior, context limits, and token accounting for the selected model rather than carrying assumptions from an earlier setup.

A practical maintenance loop is:

1. Save the prompt, model configuration, example set, and evaluation results as one version.
2. Review misclassified or incorrectly extracted records on a regular schedule.
3. Turn recurring failure modes into candidate examples or clearer rules.
4. Remove examples that no longer represent current traffic.
5. Compare the revised prompt with the previous version for both quality and token use.
6. Roll back if improvements in one category create regressions elsewhere.

For team workflows, a versioned change log makes these decisions easier to audit; see prompt versioning for teams for a practical approach. If the task depends on retrieved documents, test the examples together with retrieval quality rather than treating the prompt as the only variable. The RAG tutorial covers that broader evaluation context.

The best few-shot prompt is not the longest one. It is the smallest, clearest example set that covers real decisions, makes boundaries explicit, and continues to perform well on unseen inputs. Keep a measured baseline, update examples when the underlying data changes, and recalculate cost whenever the prompt grows.