Six Pull Requests, Twenty Interruptions

On the night of 19 October my local software factory ran for about ten hours on two product repositories and left 6 pull requests ready for review. In the same night its gates rejected 36 runs, and 20 times it sent me a Telegram message because it needed a decision.

Work in progress: this is the state on 20 October 2026. Some of it will be different in two weeks.

The factory takes a task from Linear, plans it, implements it, has a second model check it and opens a pull request on GitHub. During the day I work with cloud agents; at night the factory gets whatever is left, and in the morning I find the pull requests. For weeks the only question was whether it worked. The six pull requests answer that one. The twenty messages raise the next one: how much do I trust it, and where should my attention go?

What is in it

Short version, so the numbers make sense.

  • Two machines, local models. Two NVIDIA DGX Spark serve one model at a time with vLLM. Today it is DeepSeek V4 Flash, split across both. There is no paid API call inside a run: the marginal cost of a night of work is electricity.
  • Code owns the sequence, agents are nodes. A deterministic program runs the phases: plan, build, review, deliver. An agent is one call that returns a structured result. Anything that needs no judgment (running tests, git, opening the PR, updating Linear) is done by code.
  • The gates decide. Tests, lint, type check, plus rules I wrote: the change must not weaken tests (no skip, no removed asserts), must stay inside the files the plan names, must not touch protected paths. A model can attach notes to a run; the gate's verdict is final.
  • Cross-family review. Every pull request is read by a model from a different family than the one that wrote it, against a versioned rubric.
  • Telegram for me. When a decision is mine, a message arrives with buttons. I approve or ask for a redo with one tap, or by voice.
  • Linear as the control panel. One label releases a task, another pauses a repository. When a run fails, an automatic triage rewrites the task, opens a repair task on the repository, or asks me.

The principles live in a project "constitution" that the models read and the gates enforce. The current one is version 4.

The night of 19 October

Two product repositories, about ten hours unattended.

  • Events logged: 79
  • Pull requests ready for me: 6
  • Runs rejected by the gates: 36
  • Rejections handled by the automatic triage: 17
  • Rejections that reached me on Telegram: 20

Six pull requests on real code in one night, at zero marginal cost: a month ago I would not have believed it. The other rows are the ones I keep looking at. Rejections outnumber deliveries 6 to 1 (36 against 6), and the factory needed me 20 times, at night.

Where it broke

I read the reports one by one. The causes, most frequent first (I sorted them while reading; I did not count each one):

  1. A check of mine, wrong. The rule "the plan must name a test file matching this pattern" rejected most of the runs. The repository keeps its tests in places the rule did not expect, so correct work kept bouncing. The fix is a pull request of a few lines; before I wrote it, the rule had already burned hours of model time.
  2. Builder against reviewer. In some runs the reviewer flagged a blocking issue and the builder's output stated, with reasons, that the fix did not fit inside the constraints of the task. Both outputs had a point. Today the system has no path for a reasoned disagreement, so it discards the run.
  3. Baseline tests that change. A test that passed before and fails after is an automatic rejection, and that rule is right. Some of those tests were fragile already (they call git, they write into the repository), so the rejection charged the factory for the repository's own debt.
  4. Badly written tasks. Two tasks had contradictory acceptance criteria. The planner flagged the contradiction and stopped. Here the system did its job: the error was mine.
  5. Operational noise. A 100 MB archive left in the working directory, a missing JSON result. Small things, and each one costs a full run.

What the list has in common: the gates I write age faster than the models. On cause 1 the model did the work, and the check I had written three days earlier was not ready for the repository as it is today.

A conversation that moved me

I described the night to a friend who has been running setups like this for longer than I have. These are the points I took away, close to how they came out of the conversation:

  • Make it reversible. Let the factory merge into a development branch, then fix forward or roll back. A rollback costs almost nothing, and nobody reads all the code before approving, between humans either.
  • Infrastructure outlasts checks. As models improve, checks go stale. Observability, rollback and triage stay useful.
  • My attention in two places. At the start, on the logic: schemas, specs, what must be true. At the release to production, for example a release every two weeks. Autonomy in between.
  • Interrupt only for changes to the spec: something that would make the software different from what was specified. Everything else travels as information attached to the work.
  • Evals should measure the orchestration. Models, runtimes and hardware improve without me. What I have to choose, with data, is the harness and the way the runs are orchestrated.

I do not agree with all of it yet, and I have not decided which parts I will push back on. It gave me the words for the new question.

What I am changing

I am turning the conversation into six decisions, one at a time. All six are still open.

  1. What the development branch promises. My position today: three things only. It builds, the tests pass, it deploys to staging. Everything else, including the rules on weakened tests and allowed files, becomes information attached to the pull request instead of a block. If the branch breaks after a merge, the factory reverts on its own and opens a task to redo the work, without waking me.
  2. Which checks stay hard and which become advice. The test-file rule from 19 October is the first candidate for advice.
  3. What "serious" means, written as concrete criteria. A real database migration, for example, stays mine.
  4. What I look at instead of the code. Maybe the schemas, maybe a demo. I do not know yet.
  5. Releases from development to production: how often, and who presses the button.
  6. Scheduling and routing. Which task runs locally and which in the cloud, on which model and which harness. One number here surprised me: running more jobs in parallel on the same local machine gives only 13 to 28% more throughput, because speculative decoding already saturates the compute. On one box, more parallelism has a price.

What I take from it, for now

What I need before I trust it is an architecture that notices fast and rolls back. A better model would have fixed at most one of the five causes above, the disagreement between builder and reviewer, and I am not sure about that one either.

Every check I write is a bet on the repository as it is that day. It stays as long as it rejects bad work; the first time it rejects correct work, it gets reviewed.

At zero marginal cost the queue moves to me: 20 notifications in one night say I sit in the wrong place in the loop.

The part that paid back this week was the automatic triage: 17 of the 36 rejections were resolved without me, by rewriting the task or opening a repair. It is infrastructure, and it keeps working when the model changes.

I do not know yet where the line between "the machine decides" and "I decide" goes, and I suspect it sits in a different place for each repository.

Next two weeks: decisions 1 and 2.

← Back