I spent 15 years rationing development time. Then we built an AI software factory.

About the author: Peter Dyllick-Brenzinger is co-founder and CPTO of beatsquares. He builds the tools communicators and editors will work with next, after more than a decade in product leadership at BILD, WELT and SPRYLAB. LinkedIn

What an AI software factory actually looks like on a Tuesday afternoon

Managing disappointment, that was what my job in product leadership—at BILD, WELT, and PURPLE—mostly boiled down to for the last fifteen years.

You know the drill. You have a backlog full of genuinely good ideas and a finite number of engineers. Roadmaps, prioritization matrices, and quarterly planning sessions are, when you strip away the jargon, just socially acceptable ways of deciding what you aren’t going to build.

Lately, though, that constraint has just... vanished for us at beatsquares. We didn’t go on a hiring spree. We just changed how the work gets done. Today, our four-person team is shipping more output than the teams of twelve or fourteen I used to manage.

It’s a personal observation, not a peer-reviewed study, but the shift is dramatic enough that I wanted to write down what’s actually happening. If you lead a product or tech team, especially in media or publishing, here is what it looks like when you hand the keyboard over to an AI software factory.

The spectrum of "AI coding"

The term "AI software factory" got some traction earlier this year after StrongDM’s CTO, Justin McCarthy, described their setup. His team runs on two pretty extreme rules: humans don’t write code, and humans don’t review it. Agents take specs, write the code, and converge on a solution entirely on their own.

Dan Shapiro has a useful framework for this, calling that top tier the "dark software factory" because humans are largely out of the loop. One step down, humans act as managers while agents do the heavy lifting. OpenAI’s internal team, which generated about a million lines of code in five months without writing any by hand, summarized their approach nicely: "Humans steer. Agents execute."

We sit right around that middle tier. Agents write the code, generate the tests, and do the reviews. People decide what to build, argue with the agent about the approach, and hit merge.

A real example: one feature, three hours

On October 6th, a product owner for newsletters at one of our clients, mentioned on a call that their morning newsletter had a new author. The system’s author block only allowed for five names, and all five slots were taken.

Given that this is a general problem a lot of clients (will) have, I asked Claude to write a ticket from the call transcript the next day. The ticket laid out why it mattered, what the edge cases were, and—crucially—what we were explicitly leaving out of scope. I edited the draft and passed it to “To Do”-column in Linear.

On October 7th at 11:08, one of my colleagues, Yaser, picked up the ticket and ran a quick “grill me” session with Claude Code, after which the agent went to work. By 12:59, it had opened a pull request with about 800 lines of new code, e2e test included. At 13:22, our automated review agent pushed back, flagging two major issues and seven minor ones. The coding agent fixed them, and at 14:06, Yaser merged it. Our hourly weekday deploy pipeline took it to production from there.

The human work that afternoon was small, but it was the only work that actually mattered. I had to understand what the client needed. I had to make scope calls (raise the limit to twenty, use a dropdown if there are more than five, and ignore the combined author block for now). And Yaser had to decide it was ready to merge. That was it.

The model is a brain in a jar; the harness is the body

Here is the biggest lesson we’ve learned so far: the AI model itself is the least interesting part of the stack.

Our developers don’t really work in an IDE anymore, and they aren’t just using Copilot to autocomplete lines. The work happens in a chat interface with the agent. But a chat interface only works if you’ve built a robust "harness" around it.

Birgitta Böckeler at Thoughtworks defines the harness as everything in the system except the model. It’s the guides that steer the agent before it acts, and the sensors that check its work after. Anthropic has written about similar patterns for keeping long-running agents on track.

In our setup, the harness does a lot of the heavy lifting:

• Tickets are automatically drafted from meeting transcripts and Slack threads.
• Agents write the application code alongside end-to-end tests.
• Every pull request goes through a deep automated review that will flat-out block the merge if it finds major issues.
• Deploys run hourly on weekdays.
• Release notes and documentation updates are generated and pushed automatically after every deploy.

We optimize for process stability, not raw speed. Models change every few months. What was cutting-edge in March is a commodity by September. The harness is what keeps the output reliable when the underlying intelligence shifts.

AI doesn’t fix bad plumbing

I want to be a bit of a buzzkill here, because there’s a lot of hype around AI coding tools right now.

For us, agents came last. Before we even thought about AI, we had modern dev environments, solid CI/CD pipelines, automated testing, and fast deployments. These are the baseline capabilities that the DORA research has associated with elite engineering teams for years.

The 2025 DORA report surveyed nearly 5,000 tech professionals and found that while 90% are using AI at work, it comes with a catch. AI adoption correlates with higher delivery throughput, but it still correlates with lower stability. The authors summarized it perfectly: "AI doesn’t fix a team; it amplifies what’s already there."

If your deployments require manual intervention and your test suite is thin, handing the keyboard to an agent just means you’re going to push bad code into production faster. Fix your plumbing first.

"But doesn't code review just become the bottleneck?"

This is usually the first question I get from other engineering leads, and it’s a fair one. The data backs up the concern.

Faros AI looked at telemetry from over 10,000 developers last year. They found that teams with high AI adoption merged 98% more pull requests. But review time spiked by 91%, bugs per developer went up, and they didn't see a significant improvement at the macro company level. Similarly, a 2025 METR trial found that experienced open-source devs were actually 19% slower when using AI tools (though to be fair, in AI time, a study from last year is practically an archaeological artifact).

The reality is that if you just give developers AI tools and keep your human review process exactly the same, you will drown your senior engineers in pull requests.

Our solution was to make review part of the factory itself. We had the same problem that Faros AI found everywhere. The code reviews were the new bottleneck. Eight weeks ago, we fully automated code review for every PR. Since then, our monitoring via Oh Dear has shown 100% uptime, and users haven't reported any bugs - while pushing features daily. The quality is genuinely better than what I saw before, even with extremely senior teams. Now, eight weeks on one codebase with a small team isn't enough to declare universal victory, but it does prove that the review bottleneck is a design problem you can solve inside the harness.

The bottleneck doesn't disappear, it just moves

Scarcity never really goes away; it just relocates. For us, the new bottleneck is insight.

We can now build software much faster than we can figure out what we should be building. The constraint has moved entirely into user research, product discovery, and UX testing. Andrew Ng has pointed out this exact shift, noting that product management is becoming the limiting factor, and some teams are moving their engineer-to-PM ratios from 8 to 1 down toward 1 to 1.

For product folks, this is actually a pretty exciting shift. You spend less time rationing engineering capacity and negotiating sprint points, and more time actually talking to users and testing designs. The craft of product management finally gets some room to breathe.

Where to start

If you’re looking at this and wondering how to transition your own team, my advice is to take it in steps. Don’t just buy a bunch of AI seats and tell everyone to code faster.

Automate your delivery pipeline first (guess what, AI can help with that). Build the harness. Once that foundation is solid, move the actual coding work into the chat with agents. Automate your code review when the rest is automated and solid.

At beatsquares, we’ve taken these same automation loops and applied them to editorial and communications work. We take articles, company news, and campaigns, and the system turns them into newsletters, podcasts, and social posts, with humans stepping in to edit and approve the final output. It’s been working really well for publishers and corporate comms teams.

If you’re curious about what these loops look like in practice for your newsroom or your team, feel free to book a demo with us. Happy to just chat about it, too.

Sources

1. Justin McCarthy, StrongDM: Software Factory, February 6, 2026
2. Dan Shapiro: The Five Levels: From Spicy Autocomplete to the Software Factory, January 28, 2026
3. Ryan Lopopolo, OpenAI: Harness engineering, February 11, 2026
4. Birgitta Böckeler, Thoughtworks: Harness engineering for coding agent users, April 2, 2026
5. Anthropic: Effective harnesses for long-running agents, November 26, 2025
6. Google Cloud / DORA: Announcing the 2025 DORA report, September 23, 2025
7. Faros AI: The AI Productivity Paradox, July 23, 2025
8. METR: Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity, July 10, 2025
9. METR: Uplift study design update, February 24, 2026
10. Andrew Ng: The Batch, Issue 349, April 17, 2026