The term RAG has become associated with a particular kind of infrastructure: chunk documents, embed them into vectors, store them in a specialised database, retrieve the most relevant chunks at query time, inject them into the prompt. This pipeline is genuinely useful when the knowledge base is large, when relevance is hard to determine in advance, and when you need to query across hundreds or thousands of documents.
Most of the time, that is not the situation. Most of the time, you know roughly what context the agent needs. You just need a reliable way to provide it. Markdown makes that straightforward.
What RAG actually does
At its core, RAG solves a simple problem: AI models have a fixed training cutoff and no knowledge of your specific domain, your codebase, your documents, or your conventions. If you want the model to reason about your situation rather than a generic one, you have to supply that information somehow.
The standard RAG pipeline solves this by automatically finding and retrieving the most relevant pieces of a large knowledge base. It is an engineering solution to a retrieval problem — necessary when you cannot reasonably decide in advance what context is needed.
If you already know what context the agent needs, retrieval is not the bottleneck. Formatting is.
When the relevant context is knowable — when you can write it down — the retrieval step is unnecessary. What remains is just the injection: getting the right background information into the model's context window before it starts reasoning.
When the pipeline is overkill
Consider what "RAG" actually needs to accomplish in common practical scenarios:
It needs to know your conventions, your architecture decisions, the libraries you use, the patterns you follow. This is stable, curated knowledge you can write down. A CONTEXT.md or CLAUDE.md file in the project root gives the agent exactly what it needs — with no retrieval step.
It needs product knowledge: features, pricing, known issues, response policies. That knowledge fits in a single well-organised Markdown document. The agent reads it at the start of each conversation. No vector store required.
It needs to know your tone, your terminology preferences, what to avoid, how you structure different document types. A style guide is already a natural Markdown document. Feed it in at the start and the assistant behaves consistently without being reminded every time.
In each of these cases, the knowledge is not vast. It is specific, curated, and stable enough to maintain as a text file. The infrastructure overhead of a proper RAG pipeline would dwarf the actual work.
Why Markdown is the right format
When injecting context directly into a model's prompt, the format of that context matters more than it might seem. Markdown has specific properties that make it significantly better than the alternatives for this use case.
Markdown structure — headings, lists, code blocks — carries semantic meaning that language models recognise and reason with. A ## heading signals a conceptual boundary. A bullet list signals enumeration. A fenced code block signals: treat this literally. The model reads this structure the same way it reads the rest of its training data, because most of that training data was Markdown.
A Word document carrying the same information as a Markdown file is much larger — not because it contains more meaning, but because it carries layout, fonts, styles, revision history, and binary metadata. None of that information is useful to a language model. It is noise that occupies token budget without contributing to understanding. A PDF is similar: structure that was designed for print rendering, not for machine comprehension. Markdown contains only what matters.
When you inject a Word document into an AI pipeline, you cannot easily see what the model actually received — the extraction step (whether via library or API) may lose formatting, merge paragraphs, or drop tables. Markdown is plain text. What you write is what the model reads. You can verify the context with your own eyes, in any text editor, in seconds.
A context file that is tedious to edit tends not to get updated. A Markdown file in your project folder can be edited in any editor, committed to version control, reviewed in a diff, and updated incrementally as your knowledge evolves. The lower the friction, the more current the context stays.
What this looks like in practice
A concrete example: a Markdown context file for a coding agent working on a backend API project.
## Project: Invoice API
Internal REST API for processing invoices from supplier EDI feeds.
Built with Node.js (Express), PostgreSQL, deployed on Azure.
### Architecture
- `src/routes/` — HTTP route handlers, thin layer only
- `src/services/` — business logic, no direct DB access
- `src/db/` — query functions, never used outside this folder
- `src/types/` — shared TypeScript interfaces
### Conventions
- All dates stored and returned as ISO 8601 strings (no Date objects in API responses)
- Errors returned as `{ error: string, code: string }` — never expose stack traces
- Database queries use parameterised statements, never string interpolation
- Service functions are pure where possible — side effects isolated to route handlers
### Dependencies
- **Do use:** zod (validation), dayjs (date manipulation), pg (database)
- **Do not introduce:** ORM libraries, date-fns, axios (use built-in fetch)
### Known issues
- The `parseSwedishDate` utility does not handle two-digit years — treat as invalid
- Supplier ID 0041 sends malformed VAT fields — skip validation for this supplier only
This file takes perhaps twenty minutes to write for a mature project. Once it exists, any AI session on this codebase starts with full context. The agent won't suggest an ORM, won't use the wrong date format, and knows about the one supplier that breaks the rules. None of that required a retrieval pipeline — just a well-maintained text file.
This pattern connects directly to spec-driven development: a spec written in Markdown before implementation is already context for the agent that writes the code. The spec doesn't become obsolete after the first session. It persists, accumulates, and becomes exactly this kind of knowledge file. The two patterns feed each other. If you're not already writing specs in Markdown, that's a good place to start — it's covered in Spec-driven development — this is where Markdown really shines.
Format comparison: Markdown vs. alternatives
| Format | Token efficiency | AI-readable structure | Human-editable | Versionable |
|---|---|---|---|---|
| Markdown | High | Native | Yes | Yes (plain text) |
| Plain text (no structure) | High | Partial — no semantic sections | Yes | Yes |
| Word (.docx) | Low — format overhead | Requires extraction step | Yes | Binary diffs only |
| Low — print metadata | Extraction often lossy | Difficult | Not practical | |
| JSON/YAML | Medium | Structured, but not prose-friendly | Requires care | Yes |
Plain text without structure works, but misses an opportunity: the model cannot tell where one concept ends and another begins without signals like headings and lists. Markdown provides those signals at no cost in readability.
Where this approach has limits
Markdown-as-context works well when your knowledge base is small enough to fit in the context window — or when you can curate a subset that will. Current models have large context windows, so this covers a wide range of practical use cases. But there are situations where a proper RAG pipeline earns its complexity:
If the relevant documents collectively run to hundreds of thousands of tokens, you need retrieval. A context file that includes everything will exceed the window or become unwieldy to maintain.
If the agent might need any of several hundred documents depending on the user's query, automated retrieval is more reliable than manual curation. This is the case RAG was designed for.
A context file is only as good as its last update. If the relevant information changes daily and keeping the file current is impractical, a pipeline that pulls from a live source is more accurate.
Below those thresholds, the Markdown approach is not a compromise — it is the sensible choice. The retrieval pipeline adds latency, infrastructure, and maintenance cost. If you don't need it, you don't want it.
Getting started
The entry cost is low enough to start today:
Call it CONTEXT.md, AGENT.md, or CLAUDE.md — the name doesn't matter. Write what the agent needs to know to work well in this domain. Keep it concise and factual.
Paste it as a system prompt or prepend it to your first message. Many AI coding tools and agent frameworks support a configuration file that gets included automatically — use that if it's available.
When the agent makes an assumption you have to correct, that correction belongs in the context file. When a convention changes, update the file. It takes thirty seconds and makes every future session better.
Write the context. Inject it. Maintain it.
Markdown's transparency is its advantage here: you can see exactly what the agent reads, edit it without tools, and keep it in version control alongside everything else. Compare that to a vector store — opaque, infrastructure-dependent, and overkill for most needs.
For most practical AI workflows, the knowledge isn't the hard part. Getting it into a form the model can reliably use is. Markdown solves that problem in the most direct way possible: structured plain text, written once, reused everywhere, with no pipeline between you and the model's context window.
If you're building specs for development tasks, the spec is the context file — written before the code, reused in every session that touches that component. That combination is covered in Spec-driven development.
Related reading: Spec-driven development — this is where Markdown really shines · Which Markdown flavor for AI? · Plain text formatted — what "AI-ready" really means
Further reading: Plain text, formatted — what AI-ready really means · Spec-driven development — this is where Markdown really shines · Build your own agent with Markdown