Context engineering is becoming one of the most important disciplines in serious AI development. A 2024 study in the Transactions of the Association for Computational Linguistics found that language models could perform worse when relevant information was buried in the middle of a long context. A larger context window does not guarantee that an agent will use the right information well.
My rule for multi-step workflows is simple: give the model the smallest sufficient set of information needed for the next decision. Not the entire history. The next decision.
Key Takeaways
- Context engineering controls what information an AI system receives at each step.
- Long histories, noisy tool output and stale memory can reduce reliability.
- Dynamic retrieval, structured state, compaction and isolation are core strategies.
- Multi-agent systems should pass structured findings, not giant transcripts.
Why Long Context Can Still Fail
It is tempting to solve context problems by increasing the token limit. That is often the easy answer rather than the engineering answer.
The TACL paper, Lost in the Middle, found that relevant information near the beginning or end of long inputs was often easier for models to use than information in the middle. The researchers also found diminishing gains from adding more retrieved documents.
That is why I treat context as a working set, not a storage bin. Microsoft’s multi-agent reference architecture similarly recommends reducing outdated, redundant and noisy information while prioritising context that is useful and timely.
Â
Strategy 1: Build Context for the Current Step
A researcher, verifier and writer should not receive the same context.
The researcher needs the question and source requirements. The verifier needs claims, evidence and provenance. The writer needs verified facts and output rules.
Passing everything to everyone creates noise.
LangChain groups the main techniques into write, select, compress and isolate in its context engineering guide. That framing turns context engineering into a system-design problem rather than a prompt-writing exercise.
Planner → Researcher → Verifier → WriterEach stage gets a deliberately assembled context, while its output becomes structured input for the next stage.
Strategy 2: Retrieve Information Just in Time
Anthropic’s guidance on context engineering recommends keeping lightweight references and loading underlying information only when it becomes necessary.
{
"document_id": "report_2026_09",
"page": 43,
"claim": "Revenue increased 18%"
}The system can fetch the original passage when verification starts instead of loading the full report at the beginning. You preserve recoverability without turning the active context into a document dump.
Strategy 3: Replace Transcript Memory with Structured State
A long conversation is a poor database. Use explicit state instead:
{
"goal": "Verify product launch report",
"completed": ["primary-source review"],
"open_questions": ["pricing", "release date"],
"decisions": ["use company announcement"],
"next_step": "check official documentation"
}This is easier to inspect, resume and debug. I also recommend separating facts, assumptions, decisions and errors so an assumption cannot quietly become a “fact” later.
Strategy 4: Compress Without Destroying Evidence
Anthropic recommends compaction for long-running agents: summarise accumulated work and continue from a fresh context. The danger is over-compression.
A weak summary says:
The customer is nearing renewal.
A useful state record preserves the details:
Contract expiry: 31 October
Notice period: 60 days
Current usage: 72%
Budget: $75,000Recent work on Agentic Context Engineering (ACE) describes “context collapse”, where repeated rewriting gradually strips useful detail. Its authors propose structured, incremental updates instead.
Strategy 5: Isolate Agents and Tool Output
Multi-agent systems become harder to control when every agent sees every tool and every result.
Microsoft recommends concise, AI-friendly tool definitions and clear input/output formats. Deterministic work should stay outside the model when practical.
Instead of returning a huge API payload, return only what the decision needs:
{
"customer_id": "C1042",
"risk_flags": ["recent_chargeback"],
"balance": 45000,
"currency": "NGN"
}For agents, tool design is context design.
The same principle applies between agents. A research agent can return five verified findings with source references; it does not need to send the orchestrator its entire search transcript.
A Practical Workflow
- Define the goal, hard constraints and output contract.
- Retrieve only the sources needed for the current subtask.
- Convert raw results into claims with provenance.
- Check contradictions before generation.
- Save durable state and discard unnecessary history.
- Build fresh context for the next step.
- Validate against the original requirements.
Every stage should produce a clean hand-off.
Measure the Context, not Just the Answer
Do not optimise context because a diagram looks elegant. Measure it.
- Tokens per model call
- Retrieval precision and stale context
- Task success rate
- Tool errors
- Latency and cost per successful task
Microsoft recommends an iterative loop: build, measure, identify the biggest problem, make a targeted change, then measure again.
That prevents a common mistake: celebrating lower token usage when the agent has simply lost information it needed.
The Bigger Idea
Context engineering is best understood as runtime information architecture for AI.
The model does the reasoning, but the surrounding system determines what it sees, what it can retrieve, what it should remember, what it should ignore and what another agent is allowed to receive.
Better prompts still help. But once an agent researches, calls tools, stores state and hands work to other agents, the central engineering problem becomes much more concrete:
What should the model see at this exact step?
