Talking to a Machine That Only Reads

How to Actually Write the Prompt
Part 2 of Demystifying Prompt Engineering

In Part 1 we landed on one idea that an LLM isn’t a mind you’re chatting with, it’s a document-completion engine. It reads what you give it and predicts how that document most likely continues.

Great. So if it’s finishing a document, your job is to write a good one. Not a longer one, a better one. Here’s what that actually means. 💡

Most people, when a prompt underperforms, do the same thing: they add more words. More instructions, more caveats, more “make sure you don’t…” The prompt balloons and the output gets worse, because you’ve buried the signal in noise. A good prompt is the opposite move. It’s a tight document built so the only natural way to continue it is the answer you want.

Four things make that happen.

1. Show, don’t tell

The single highest-leverage thing you can add to a prompt is an example.

This is few-shot prompting. You give the model a couple of worked examples of the input-and-output pattern you want, and let it extrapolate. A prompt with zero examples is zero-shot; one with a few is few-shot. And the reason it works so well is exactly the document model from Part 1. The model is compelled to continue patterns. Show it two input → output pairs and the third one practically writes itself.

Why not just describe the format in words? Because some things are nearly impossible to state as rules. Tone. Structure. That “I know it when I see it” house style. You can write a paragraph trying to pin down the exact JSON shape and edge-case handling you want, or you can show two examples and let the pattern speak. As the book puts it: implicit is often better than explicit.

Two cautions, because few-shot has sharp edges:

• Your examples set the “normal.” The model assumes whatever distribution your examples imply. Show it three happy-path cases and it’ll quietly decide errors don’t happen. Cover the edge cases you actually care about.
• It’ll copy patterns you didn’t mean to teach. If all your examples are in ascending order, or all “positive” cases come first, the model may latch onto that instead of the real signal. Mix them up.

When the task is genuinely simple, skip the examples. They cost tokens and add risk. But the moment format or style matters, stop describing and start demonstrating. 🛠️

2. Give it the right context, not all of it

Here’s a trap that feels like diligence. You dump everything possibly relevant into the prompt (the whole file, the entire doc, all twelve past messages), figuring more context can’t hurt.

It can. Remember the model is finishing a document, and in a well-formed document, every detail is there for a reason. So the model feels compelled to use whatever you include. Drop in an irrelevant snippet and it won’t ignore it. It’ll find a way to make it matter, and steer the answer somewhere you didn’t want. Call it the Chekhov’s-gun problem: if the gun’s on the wall, the model assumes it’s going to be fired.

So the real skill is curating context. Two moves cover most cases.

• Retrieval, zoom in. When the answer lives in a big pile of material (docs, a codebase, past tickets), pull only the snippets actually relevant to this question and inject those. This is what RAG (retrieval-augmented generation) is underneath the buzzword. You fetch the relevant bits, then prompt. The whole game is fetching the right bits.
• Summarization, zoom out. When the material’s too big to fit and you can’t just grab a few snippets, compress it. For something huge, summarize in chunks and then summarize the summaries. You lose detail, so summarize with the end task in mind, and keep what you’ll need.

Either way, the goal is the same: the model should see exactly what’s relevant and nothing that’ll distract it.

3. Put what matters where the model will see it

Position matters more than almost anyone realizes. Models pay the most attention to the start and the end of a prompt, and tend to get fuzzy in the middle, and the longer the prompt, the deeper that dead zone gets. Bury your most important instruction in paragraph six of ten and you’ve hidden it in the worst possible spot.

So treat the start and end as prime real estate:

• State the ask up front, then, after the context, restate it at the end. A long prompt pulls the model’s attention every which way; that closing restatement re-aims it right before it starts generating. State the question, give the context, then ask the question again.
• End by pointing at the answer. The last thing in the prompt should transition from describing the problem to starting the solution. For a chat model that can be as simple as a direct question. For a raw completion, you can even write the first line of the answer yourself and let the model continue from there.

If a piece of content is critical, it does not belong in the murky middle. Filter hard, and park what matters at the edges.

4. Pick a document the model has already seen

Last one, and it ties the whole thing together. Your prompt-plus-answer is a document, so make it look like a kind of document the model has read a million times. It already knows how those flow, so the completion comes out cleaner and more predictable.

Three reliable shapes:

• The conversation, a back-and-forth between a user and a helpful assistant. Natural for chat, great for multi-step tasks. This is the default most chat models are tuned for.
• The report, an objective, structured write-up with an intro, analysis, and conclusion. Perfect when you want reasoning and analysis, and it dodges the “social” weirdness of role-play. Add a short scope line (“this focuses only on the API layer, not the frontend”) and the model honors the boundary better than it would in a chat.
• The structured document (XML, YAML, or JSON), for when you need to parse the output. Pick a format and the model fills in the shape; now your code can rely on it.

You’re borrowing a format the model is already fluent in. That’s the move.

The takeaway: tighter, not longer

Every one of these is the same instinct applied four ways: a good prompt is a clean document, not a big one. Show the pattern instead of describing it. Feed it the right context, not all of it. Put what matters at the edges. Borrow a shape it already knows.

Do that and you’ll watch a bloated, unreliable prompt turn into a short one that just works. You didn’t find the magic words. You built a document with one obvious continuation.

In Part 3, From Prompt to Product, we leave the single prompt behind. Real systems chain prompts together, hand the model actual tools, and (the part everyone skips) measure whether the thing works. That’s where prompting quietly turns into engineering.

What’s the worst “I just kept adding more instructions” prompt you’ve fought with? Tell me what finally fixed it. I want to hear what worked.