How I Build Software With a Team of Agents
A plain-English tour of the system I build software with. One agent writes the code, others review and test it, automatic checks guard secrets and documentation, and three decisions always stay with me.
I build software by directing coding agents. An agent is a program that reads files, runs commands and edits code on its own, with an AI model doing the thinking. One agent working alone is fast, and it has three problems. It forgets everything when the session ends. It marks its own homework. And it drifts away from what the project was meant to be.
So I built a system around the agent that fixes those three things. It gives the agent a memory, a second opinion, checks it cannot talk its way past, and a scoreboard. This page explains that system without the jargon. The full technical version, with every part named, is here: Inside the harness: how A://IDE, Pi and Herdr work together.
The four pieces
Two of the pieces are other people’s tools. Two are mine.
- Pi runs one agent. It connects the AI model to the tools the agent needs and keeps a full transcript of everything said and done. I did not build Pi.
- Herdr keeps many agents running side by side, each in its own pane, and lets them send each other messages. If I close the app window the agents carry on. I did not build Herdr either.
- The harness is mine. It is a set of rules, tools and checks that loads into every agent I start. It turns one agent into something closer to a small development team.
- A://IDE is mine too. It is the desktop app I look at all day: every project in one window, a chat with each agent, the files, a to-do list, and a view of what the checks are finding.
The team
The agent that writes the code is never the one that judges it. That one rule shapes the whole team.
How a piece of work goes
- I describe what I want.
- The builder writes a plan: what done looks like, how each part will be tested, and a short numbered list of tasks.
- The plan reviewer checks the plan. The builder then stops and waits for me.
- I say yes. Nothing is written before that.
- For each task, the tester writes failing tests, the builder makes them pass, and the critic reviews that one task.
- At the end the gatekeeper runs every scan. If anything is red, the work is blocked and no AI gets a vote.
- I get one summary card: what was asked, which tests prove it, what the reviewers found, how risky the change is, and what the checks would not have caught. Then I decide.
The checks and balances
These are the rules I trust most, because none of them depend on an agent behaving well.
- The writer never reviews its own work. Reviews are done by a different model, usually from a different company, so the two are less likely to share the same blind spots.
- Reviewers cannot write. They can read and report. They cannot fix things quietly.
- The tester only writes tests. If it changes anything else, the change is undone automatically.
- Tests have to bite. The system removes the new code and runs the tests again. If they still pass, they prove nothing, and the task is marked not done.
- Secrets are scanned on every save. Each commit is checked for passwords and keys before it is recorded. A second, deeper scan runs before anything can be merged.
- Read before you edit. An agent cannot change code in an area until it has read the documentation for that area.
- Document before you finish. An agent cannot end its run with code changed and the documentation untouched.
- Destructive changes wait for me. Deleting a table or wiping data needs my confirmation.
How the project remembers
Agents start every session with no memory, so the project carries the memory for them. Each layer has one job.
- Documentation holds intent: what each part is for and why it was built that way.
- The code graph holds mechanics: which piece of code calls which, and what breaks if one changes.
- The schema file holds the shape of the data.
- Wireframes hold what the screens should look like. I sketch them in A://IDE and the agent edits the same file.
- Git holds history, plus a hand-off note at the end of each session saying what was done and what comes next.
At the start of a session the agent reads all of this before it touches anything. At the end it has to update whatever it changed. That is how a project stays understandable after hundreds of sessions.
What stays with me
Three decisions never leave my hands:
- What to build. I approve every plan.
- What gets merged. Nothing reaches the main branch of a project without my go-ahead.
- What goes live. Production deploys happen only when I say so, and the system keeps the rollback command ready.
It measures itself
Every session is recorded as numbers: which tools were used, how long things took, how much the model read. The content of the work is never stored in that record. Once a day a script turns those numbers into a scorecard, compares it with targets, and files a to-do list of what to improve. Small tuning changes are tried automatically and reversed a week later if the numbers got worse.
Where it falls short
The scorecard is honest, and so I will be too.
- It costs more than I want. A session costs about three and a half times my target, mostly because sessions run long.
- Reviews miss things. About four in ten of the bug fixes I have traced were for bugs that slipped past a review.
- The rituals get skipped. The start-of-session routine runs in just over half of sessions and the end-of-session routine in under four in ten. My target for both is eight in ten.
- I still drive each step. The system tracks the plan, the tests and the reviews, but nothing yet runs a whole job from start to finish without me nudging it along.
The technical version lists all ten weaknesses with the numbers behind them.
The models
The builder currently runs on Claude Opus 5.5 from Anthropic. The reviewers run on GLM-5.3 from Z.ai, with Kimi from Moonshot as the stand-in when GLM is unavailable. I still use GLM-5.3 as a builder as well, and I am about to run the two side by side on the same work to see what difference the model makes once the checks are the same.
If you want the story of how I got here, from pasting ChatGPT answers into an editor to this, that is in Coding with AI: from copy-paste to custom harness.
This case study was written with the assistance of AI.