Prompt Engineering: The Core Moves

Ask a model “explain why my Python code is slow” and you get a generic paragraph about interpreted languages. The context window contains almost no signal about format, audience, or level of detail — so the model samples from a broad swath of “Python performance” content in its training distribution. The most probable completion for a vague prompt is a vague answer.

Every technique in this lesson works by adding signal to the context window, narrowing the output distribution from “anything that could plausibly follow these tokens” toward exactly the shape of response you want.

The first high-leverage move: add a role prefix before any content. One sentence that tells the model what kind of person is speaking — or what kind of response this is — shifts the entire prior distribution for every token that follows.

The mechanism is the same token statistics you saw in the previous lesson: the training corpus contains millions of documents. Conversations that start with “You are a senior security engineer” are overwhelmingly written by or for security engineers. That vocabulary, those risk framings, that assumed knowledge — all of it becomes more probable as soon as the role tokens appear in context.

The widget below shows how four different role framings shift the predicted output character for the same underlying question. Notice where two roles converge — that is where corpus coverage is thin and the role signal washes out.

If role prompting sets the prior, few-shot examples move the posterior directly. You place completed input → output pairs into the context window; the model reads the pattern and continues it.

The key insight from the GPT-3 paper ^{[Language Models are Few-Shot Learners]} : you do not need to fine-tune the weights. You can reshape model behavior at inference time by placing high-quality examples in the context. Each example is just more context tokens — and at every decoding step, the model attends to all of them.

Tweet: "Best meal I've had this year." Label: POS
Tweet: "Service was shockingly slow." Label: NEG
Tweet: "Arrived as expected." Label: NEU
Tweet: "The queue took 45 minutes but the food made up for it." Label:

The widget below lets you vary shot count from 0 to 4 and observe how format-lock behavior changes. Notice the cliff between 0 and 1 is almost always larger than between 2 and 4 — a single example carries enormous format signal.

For tasks requiring multi-step reasoning — math, logic, code debugging — asking the model to “think step by step” before the answer can dramatically improve accuracy. ^{[Chain-of-Thought Prompting Elicits Reasoning in Large Language Models]} This is chain-of-thought (CoT) prompting, and the mechanism connects directly to what you already know about next-token prediction.

When the model is forced to produce intermediate reasoning tokens before the answer token, those tokens become part of its context for the answer. It is literally reading its own scratch work.

The simplest CoT trigger is appending “Let’s think step by step.” to any prompt — this phrase alone was found to elicit reasoning steps without any examples. ^{[Large Language Models as Zero-Shot Reasoners]}

The widget below steps through a CoT trace token by token. You can see which tokens are “scratch work” and where a direct-answer completion would have diverged — that divergence point is the threshold concept.

The final move requires no new mental model. You already know that context shapes the distribution. Format constraints are simply the most direct application: specify the exact shape of the output, and the set of valid completions collapses.

Three highest-leverage format instructions:

1. Name the container. “JSON object,” “markdown table,” “numbered list,” “one sentence” — each maps to a distinct corpus genre with predictable structure.
2. Name the fields explicitly. Return JSON with keys: summary, risk_level, action_items is less ambiguous than “return a JSON summary.” The model pattern-matches to documents with those exact field names.
3. Set a length constraint. “In under 100 words” or “exactly 3 bullet points” tells the model which part of the completion-length distribution to sample from.

Extract the entities from this sentence. Return JSON only — no prose, no markdown fences.
Schema: {"people": [...], "places": [...], "dates": [...]}
Sentence: "Alice and Bob met in Berlin on 14 March to finalize the treaty."

{"people": ["Alice", "Bob"], "places": ["Berlin"], "dates": ["14 March"]}

The four moves stack. A well-structured prompt has:

1. Role — sets the genre and expertise register.
2. Few-shot examples — demonstrates the transformation pattern.
3. Chain-of-thought instruction (when multi-step reasoning is needed) — forces intermediate context.
4. Format constraint — locks the output schema.

Not every prompt needs all four. A factual lookup needs none. A complex analytical pipeline benefits from all four.

// Role (prior)
You are a senior security engineer reviewing Python code.

// Format constraint (output shape)
For each issue found, respond with exactly this JSON:
[{ "severity": "critical|high|medium", "line": number, "issue": string, "fix": string }]
No preamble. No explanation outside the JSON array.

// Few-shot example (pattern lock)
Example:
Input: password = request.get("pw")
Output: [{"severity": "critical", "line": 1,
          "issue": "Unsanitised user input stored as password",
          "fix": "Hash with bcrypt before storing"}]

// Chain-of-thought instruction (for complex code)
Before generating the JSON, think step by step about each line's security
implications. Then produce only the JSON array.

// Task
Review: [CODE_HERE]

flowchart TD
  A[Task] --> B{Multi-step reasoning?}
  B -- Yes --> C[Add chain-of-thought]
  B -- No --> D[Skip CoT]
  C --> E{Consistent format needed?}
  D --> E
  E -- Yes --> F[Add format constraint]
  E -- No --> G[Skip format constraint]
  F --> H{Training data coverage thin?}
  G --> H
  H -- Yes --> I[Add role + few-shot examples]
  H -- No --> J[Minimal prompt may suffice]
  I --> K[Send prompt]
  J --> K