How I Do AI Engineering

The best AI engineering setup I have built is mostly an exercise in subtraction. Fewer permanent instructions. Fewer integrations. Less context loaded by default. Against that, I use a much stricter definition of done.

My rule is simple: minimise what the agent has to carry, and maximise the evidence it must produce.

I have used coding agents on almost every project I have worked on over the past year. Not for autocomplete or the occasional code snippet, but for complete features, migrations, debugging, performance work, code reviews, and maintenance. The biggest improvement did not come from discovering a magic prompt. It came from designing an environment in which the agent can inspect the work, make sensible decisions, prove that those decisions work, and continue when its first attempt fails.

At this point, working with an agent feels less like prompting a chatbot and more like managing a capable engineer who writes code extremely well but has never worked in your company. It can move very fast. It does not automatically know what your team values, which business rules are sacred, or what “finished” means in your codebase. Your job is to make those boundaries clear without burying the agent in instructions.

The Model Is Only Part of the Product

People compare coding models as if the model is the entire experience. In practice, the harness around it matters just as much. The harness determines how the model searches a repository, edits files, runs commands, manages context, asks for approval, and recovers from failed attempts.

The difference becomes obvious when a request leaves room for interpretation. Does the agent ask before widening the scope? Does it search for an existing abstraction or create a new one? Does it optimise for the visible demo or for the smallest maintainable diff? Does it stop when the requested feature appears to work, or inspect neighbouring flows for regressions?

Those defaults shape the codebase long after the first prompt is forgotten.

I worked with Claude Code for around six months and have used Codex since its launch. Both are very capable, but they feel noticeably different in daily work.

Claude Code is trigger-happy. That is both its superpower and its biggest weakness. Give it a request and it will make assumptions, build the supporting pieces, and push hard towards something that works. When starting a new project, or when the person directing it is not technical, that can feel unbeatable. A single request produces an impressive amount of visible progress.

The problem is that it can optimise too aggressively for completing the request in front of it. It may create more machinery than the problem needs or make decisions that are useful today but expensive for a growing team to live with later. It is also very good at selling its own work. A questionable implementation can arrive wrapped in such a polished explanation that it feels more considered than it really is.

Fluency is not correctness. Persuasive communication can make a short-sighted solution harder to question.

Codex has generally felt more restrained. It is less likely to start building before it understands the repository, and it tends to look for the smallest change that solves the actual problem. Across existing codebases, I have found that it more consistently considers maintainability, regressions, and the conventions already present in the system. It is the harness I recommend most often to technical teams.

Codex rarely charms me. Its communication can be blunt and overly terse. Sometimes it implements a clean solution without properly explaining why it chose that approach. When asked, however, it usually gives a short, direct explanation without trying to sell the decision. For technical work, I prefer that restraint.

These are tendencies from my projects, not a controlled benchmark or a permanent ranking. Models and harnesses change quickly, and either tool can be guided towards a different working style. The useful comparison is not which agent produces the most impressive first response. It is how the agent behaves after the fiftieth request, when every unnecessary abstraction and unchecked assumption has become part of the codebase.

I judge a harness on scope discipline, repository fit, verification behaviour, recovery after failure, and whether its confidence matches its evidence. The last point is underrated. A detailed explanation can hide weak work; a terse one can hide a careful solution. I read the diff and the evidence before I read the agent's review of itself.

Tendency	Claude Code	Codex
Default posture	Acts quickly and fills gaps	Inspects before widening scope
Initial result	More visibly complete	Smaller and more focused
Main risk	Short-term overbuilding	Under-explaining decisions
Communication	Persuasive and expansive	Direct and terse
Best fit in my work	Prototypes and greenfield work	Established technical codebases

These are tendencies from my own projects, not a universal model benchmark.

One-shot output is a demo. The real product is the codebase that remains.

Context Is Not Free

Models thrive on context, but context has a cost. Every extra instruction competes for attention. Large instruction files accumulate stale decisions. Documentation drifts away from the implementation. Eventually, the agent has to decide whether the code or the document is telling the truth.

The advertised context window only tells you how much a model can receive. It does not tell you how reliably the model will use every part of it. The researchers behind Lost in the Middle found that models often performed better when relevant information appeared near the beginning or end of a long input, and worse when it sat in the middle. Adding more retrieved documents also produced diminishing improvements even while retrieval recall continued to rise.

The exact numbers came from older models, so I would not apply them directly to today's agents. The distinction remains useful: context capacity and effective attention are not the same thing. More context can add distractors as easily as it adds knowledge.

I get the best results with a small AGENTS.md file and very little else loaded by default. Modern coding agents are exceptionally good at searching repositories. They can follow imports, inspect call sites, find related tests, and infer local patterns. They do not need an essay explaining every directory, and the code does not need comments written for an AI on every function.

My rule of thumb is that the code is the documentation. The agent file is for things the code cannot reveal.

Mine usually contains short bullets covering:

• How feature branches should be named.
• How to start and seed the project.
• Whether package installation must happen inside Docker instead of on the host.
• Which checks must pass before work is complete.
• How to run browser and end-to-end tests.
• Small workflows I ask the agent to repeat regularly.

Everything else is layered. Durable repository rules are always available. Architecture and local conventions are discovered from the code. Deployment guides, infrastructure notes, and domain explanations live in separate files that the agent loads when relevant. The ticket and its acceptance criteria belong to the current task.

These layers age differently. A test command may remain correct for months. A written explanation of a feature can become false after one refactor. Keeping volatile knowledge close to the implementation reduces the number of competing truths.

It also prevents AGENTS.md from becoming a graveyard of rules added after individual mistakes. Large instruction files tend to collect overlapping constraints, outdated warnings, and solutions for code that no longer exists. The file looks comprehensive while becoming less reliable with every addition.

Small instructions are easier for humans and agents to keep correct.

Keep the Tool Surface Small

I apply the same principle to MCP servers, skills, and other extensions. Less is more.

An agent does not need a custom integration for everything. It already understands Git, shell commands, and the GitHub CLI. Wrapping familiar workflows in another abstraction can create more ambiguity than value.

The integrations that consistently earn a place in my setup provide information or capabilities the repository cannot provide by itself: access to the ticket system, a real browser for testing, and logs or monitoring data for understanding behaviour in a running environment. That is usually enough. I do not maintain a large library of skills, and I add a tool only when a recurring task cannot be handled reliably with what the agent already has.

Every integration expands both the cognitive and security surface. The agent has another tool to choose between, another data shape to understand, and another system it can accidentally touch.

An MCP server is a trust boundary, not a convenience toggle. The MCP project's security guidance recommends progressive, least-privilege scopes and warns that broad tokens increase the impact of compromise while making actions harder to audit. Local MCP servers may also run with the same operating-system privileges as the client.

Before adding an integration, I want to know what unique capability it provides, whether it can be read-only by default, how narrowly its credentials can be scoped, and whether its actions can be reconstructed afterwards. If I cannot answer those questions, it does not belong in an unattended workflow.

Treat the Agent Like a New Engineer

The “new engineer” comparison is more than a metaphor. METR's current time-horizon research measures how difficult a task can become, based on the time a human expert would need, before an agent's probability of success falls below a given level. METR makes an important qualification: its tasks are self-contained, well-specified, and automatically scorable. It says the comparison is closer to a low-context new hire or contractor than an experienced employee with years of tacit knowledge. Performance also drops on messier work and when success is judged holistically rather than through an algorithmic score.

That matches my experience. Agents are strongest when the outcome is measurable. Production engineering is full of things that are not: unclear ownership, undocumented business rules, competing priorities, and consequences that only appear several systems away.

Once I started managing agents from that perspective, the quality of their work improved dramatically. I stopped ending prompts with “build this” and started defining the delivery loop.

Before, I would let an agent implement a feature, test it myself, review the code, and return with feedback. We might repeat that cycle several times before the work was ready. Now I make the agent own much more of that loop. It must use the real interface, test unhappy paths, inspect screenshots, hunt for regressions, review the diff for unnecessary abstraction, and check permissions, validation, migrations, and error handling.

A green test suite is evidence, not proof.

I normally expect four layers of evidence, depending on the change:

1. Static evidence: formatting, linting, types, dependency checks, and a clean diff.
2. Programmatic evidence: unit and integration tests for business rules, boundaries, and failure cases.
3. Behavioural evidence: browser tests that exercise the same public interface a user touches.
4. Visual evidence: screenshots or DOM snapshots that expose broken layout, missing states, and confusing feedback.

The agent owns the verification loop. Product judgement and the final merge remain human responsibilities.

Each layer catches a different class of mistake. A type checker cannot tell me that a save button is hidden below a modal. A screenshot cannot prove that permissions are enforced on the server. A browser test that inserts records directly into the database can prove the backend works while completely missing a broken onboarding flow.

For browser tests, I prefer roles, labels, and visible text over long CSS selectors. That forces the test to interact with the semantics exposed to a user. It also follows Playwright's testing guidance, which prioritises user-facing locators and recommends traces when debugging failed CI runs. A trace is especially useful because it combines the action timeline, DOM snapshots, console output, and network activity. Screenshots help with visual judgement, but they do not replace assertions.

After the implementation passes, I ask the agent to review it again from the perspective of a maintainer who did not write it. Can code be deleted without changing the behaviour? Which assumptions are not enforced by a test or type? What neighbouring flows share the changed state? Could another endpoint cross the same permission boundary? Would a rollback leave the schema and application compatible?

The same model can repeat its original blind spot, so this is not independent review. It is a cheap extra filter before human review.

For larger tasks, I ask the agent to create a goal containing the full definition of done and continue until it is confident in the evidence. At the end, I sometimes have it record a short browser walkthrough of the completed flow and explain the important decisions. That lets me review the pull request while watching the product behave, rather than reconstructing the result from a long transcript.

The agent does more of the verification. I still own the judgement.

Goals Turn Persistence Into a Tool

A normal prompt encourages a plausible answer. A goal encourages the agent to keep working towards a measurable outcome when the first answer fails.

Performance optimisation is where I have found this most useful. I can give an agent an intentionally demanding response-time target and ask it to continue measuring and experimenting until it reaches it. Some ideas fail. Others reveal bottlenecks I would not have considered or would not have had time to test. The agent can explore many hypotheses cheaply, keep the useful ones, and discard the rest.

This is brute-force creativity: form a hypothesis, change the system, measure the result, and repeat.

Persistence without constraints is dangerous. If I only say “make it faster,” the easiest route may be to remove work the application needs to do, weaken a security layer, return stale data, or optimise a benchmark that does not represent real users. I therefore define the target together with its guardrails:

• Record a reproducible baseline and a primary metric such as p95 latency.
• Preserve user-facing behaviour, data correctness, permissions, and business rules.
• Track counter-metrics such as error rate, query count, memory, and cache correctness.
• Keep a short record of attempts, measurements, and rejected hypotheses.
• Re-run correctness and security checks after the winning change.

Counter-metrics prevent fake victories. A median can improve while the slowest requests get worse. A cache can make a benchmark fast while returning information a user is no longer allowed to see. A query can disappear because the feature quietly stopped doing part of its job.

The goal is not to make one number green. It is to improve that number without violating the system's other obligations.

My Midnight Crew

More recently, I have started offloading well-bounded maintenance work overnight. A scheduled job starts goal-based tasks while I am asleep. By morning, I have pull requests to review instead of a list of problems to investigate.

The overnight agents inspect logs for recurring errors, review dependency audits, run regression suites against development environments, and pick up tickets labelled Agent ready. They can also read an internal complaints board and turn reproducible reports from other teams into development tickets.

I reserve this mostly for bug fixes, maintenance, and non-visual improvements. The agent works on an isolated branch and creates a draft pull request. It does not deploy to production or merge its own work.

Overnight autonomy should end at a pull request.

The pipeline is intentionally asymmetric. Reading and proposing are easy; mutating shared systems is hard. The agent collects work from approved sources, checks that the task is eligible and sufficiently specified, reproduces the issue, implements a change, runs the verification loop, and opens a draft pull request containing both the evidence and any remaining uncertainty. Then it stops. A human decides whether the work moves further.

The Agent ready label creates an explicit boundary. It means the task has enough context, low enough product ambiguity, and clear enough acceptance criteria for an independent attempt. It does not mean the agent has permission to reinterpret the product or widen the work.

Logs, tickets, advisories, and complaint text are untrusted input. They may contain secrets, malformed data, or instructions that conflict with the actual task. The agent treats them as evidence to analyse, not commands to obey. Its credentials are restricted to the systems and operations the job requires. It has no production deployment credential, no permission to merge, and no broad administrative token “just in case.”

Unattended execution should not mean unattended product management.

This setup is not magic. Some pull requests are not worth merging. Sometimes the agent fixes a symptom instead of the cause or discovers that a ticket was less clear than it appeared. That is fine. The value is that reproduction, investigation, implementation, and initial verification happen before I arrive. My morning starts with judgement rather than setup.

I measure the crew by accepted value, not activity. Pull-request count is a dangerous metric because an agent can maximise it by producing small, low-value changes or turning uncertain findings into noise. Better measures are acceptance rate after review, regression rate, reviewer time, reproducibility, and whether the same class of issue returns.

I do not want to wake up to more code. I want to wake up with less uncertainty.

The Job Is Designing the Loop

The best results I get from AI engineering do not come from the longest prompt, the largest context window, or the biggest collection of tools. They come from a tight loop: give the agent durable context, define the outcome and its boundaries, let it inspect the codebase, require evidence through the real interface, review the solution for regressions and unnecessary complexity, and keep a human responsible for the final decision.

The code an agent produces is only one part of the work. The important part is the system around it: how the agent receives context, how it knows when it is done, what evidence it must produce, and where it is not allowed to act.

That is how I think about AI engineering today. Not as asking a machine to write more code, but as building an environment in which a fast, capable, relentlessly persistent engineer can do useful work without quietly making the system worse.

The paradox is that more capable agents require more engineering around judgement, not less. As the cost of producing a plausible implementation approaches zero, the scarce work moves to specifying the right outcome, designing reliable evidence, protecting trust boundaries, and deciding what deserves to exist in the first place.