Agentic AI in Product Engineering
On July 18, 2025, SaaStr founder Jason Lemkin posted that Replit’s AI coding agent had wiped his production database. During a code freeze. After being told, more than once, not to change anything. The lost records covered 1,206 executives and 1,196 companies, and heise reported that Replit was running preview, testing and production on the same database at the time.
Most people filed that under “AI gone rogue.” We’d file it under environment design. Two days later, Replit CEO Amjad Masad acknowledged the incident and pledged automatic separation between development and production databases. No smarter model. Just a wall.
That, to us, is the real story of agentic AI in product engineering right now. Writing code has become cheap and fast, but trusting it hasn’t kept up, and the teams pulling ahead in 2026 aren’t the ones with the flashiest agent; they’re the ones that rebuilt review, testing and release so agent output can be trusted at volume. Below: what the data says, where agents genuinely help, where they go wrong and how we’d start.
The short answer
Agentic AI in product engineering means AI agents that plan and finish multi-step engineering work on their own, such as changing code, writing tests and opening pull requests, inside guardrails people set. It’s making code faster to produce. The 2025–26 evidence shows the bottleneck moving to review, testing and release, so that’s where leaders should put their effort.
Key takeaways
1. Agents already do real work at scale. Stripe merges about 1,300 AI-authored pull requests a week, all through normal human review (Bain, 2026).
2. More output hasn’t meant more delivery: AI pull requests were merged within 30 days 32.7% of the time, against 84.4% for unassisted ones (LinearB, 2026).
3. Review is the new choke point. Bain found completed tasks rose about 21% while review time rose about 91%.
4. AI amplifies whatever engineering system it lands in, strengths and weaknesses alike (DORA, 2025).
5. Start risk-tiered. Only 6% of tech leaders picture fully autonomous development (Bain, 2026).
What agentic AI in product engineering actually means
A copilot suggests your next line. An agent takes the ticket.
Hand an agent something like “move the billing service from payments API v2 to v3” and it’ll read the repo, plan the change, edit a dozen files, run the test suite, fix what broke and open a pull request with a summary attached. Nobody touches the keyboard in between, at least in theory. That loop of planning, acting, checking and retrying is what separates an agent from autocomplete.
Be careful with the label, though. In June 2025, Gartner estimated that only about 130 of the thousands of vendors selling agentic AI offer the real thing. The rest is what Gartner calls “agent washing”: chatbots, RPA (robotic process automation) and assistants with a new sticker. Worth remembering when a demo looks too smooth.
The clearest production example we know of is Stripe’s. Bain’s Technology Report 2026 describes its “minions,” narrow agents that write unit tests, fix linter warnings and move code to new API versions. Around 1,300 of their pull requests get merged every week.
Each minion runs from a written spec covering its objective, scope, context, verification method and constraints, and each pull request goes through the same human review as anyone else’s. Look at that task list again. Unglamorous? Completely. That’s exactly why it works.
Faster code, slower reviews: what the 2025–26 data shows
Adoption isn’t the question anymore. Google’s 2025 DORA report found that 90% of technology professionals use AI at work. The harder question is how much of that work actually ships.
LinearB’s 2026 Software Engineering Benchmarks Report looked at more than 8.1 million pull requests from 4,800 teams, and AI-generated pull requests merged within 30 days just 32.7% of the time. Unassisted ones? 84.4%. Agent-created pull requests also sat for 17.6 hours before anyone picked them up for review (75th percentile), against 3.4 hours for human work. That’s a queue.
Bain’s April 2026 survey of 293 senior technology leaders shows the same shape from the inside. Developers finished roughly 21% more tasks, yet review time climbed about 91%, because the number of senior people qualified to approve a change didn’t grow at all. More code, same reviewers. You can do the math.
Then there’s the perception gap. In METR’s randomized trial, 16 experienced open-source developers worked through 246 real tasks in codebases they knew well, between February and June 2025. With AI tools allowed, they took 19% longer. They’d forecast a 24% speed-up and, even afterwards, believed they’d been about 20% faster.
METR is upfront that this was early-2025 tooling on large, mature repos, so treat it as a snapshot. But that distance between felt speed and measured speed? We’d tape it to every engineering manager’s monitor.
Where AI agents fit in the software development lifecycle today
Here’s how we’d split the work as of late 2026. The middle column will grow as models improve. The right-hand column moves slower than most vendor decks suggest.
|
Lifecycle stage |
What agents handle well now |
What a person should still own |
| Requirements and specs | Turning user stories into acceptance criteria; flagging gaps and contradictions | Deciding what the product should do, and for whom |
| Code | Scoped changes: API migrations, dependency upgrades, boilerplate, refactors with test coverage | Architecture, data models, anything touching auth or payments |
| Testing | Generating tests from requirements, healing broken locators, building load scripts from API specs | Risk-based test strategy and what counts as good enough to ship |
| Code review | First-pass review for bugs, security smells and spec mismatches | Final approval on changes with a large blast radius |
| Release and operations | Drafting release notes, triaging alerts, proposing rollbacks | Deploying to production and touching production data |
Testing is the row we’d bet on first. It has a clean feedback signal, since a test passes or it doesn’t, and the agent can check its own work against something deterministic before a person ever looks. Code review is moving quickly too: Bain reports that at Amazon, AI now handles up to 70% of code reviews. The common thread is a clear definition of done.
Where Agentic AI goes wrong in product teams
The public record points to four repeat mistakes. None of them is about the model.
Giving agents production keys. The Replit incident was a permissions failure more than an intelligence failure. A sensible default is to treat every new agent like an intern on day one, with read access to most things, write access to a sandbox, and nothing touching production data or money unless a human presses the button.
Bolting agents onto a shaky pipeline. DORA’s 2025 data shows AI adoption now improves throughput and product performance, but it still has a negative relationship with delivery stability. Their explanation is blunt. Without strong automated testing, mature version control and fast feedback loops, more change volume just means more instability. If your regression suite takes two days and flakes on Fridays, an agent opening 40 pull requests a week will make your week worse.
Buying the label. Gartner predicts that over 40% of agentic AI projects will be canceled by the end of 2027, citing rising costs, unclear business value and weak risk controls. Its analyst also pointed out that many use cases sold as agentic don’t need agents at all. Sometimes a script and a scheduled job will do. Cheaper, too.
Measuring how it feels. LinearB found that 44.7% of organizations don’t formally measure AI’s impact, while 76.1% of leaders say they’re seeing productivity gains. Those two numbers sit awkwardly together. In the Stack Overflow 2025 survey, 46% of developers said they don’t trust the accuracy of AI output, and 66% named answers that are “almost right, but not quite” as their top frustration. That’s a lot of confidence resting on very little measurement.
If testing is where your releases stall, take a look at AutonixQ, our AI test automation product. It turns plain-language requirements into Playwright tests, with auto-healing and requirement-to-test traceability.
How we approach Agentic AI in Product Engineering at Quarks
We’ve put more than 20 agentic workflows into operation across client work and our own accelerators. The approach is less exciting than the headlines. That’s deliberate.
Spec first, agent second. Before an agent touches a repo, we write down roughly what Stripe writes down: the objective, the services in scope, the context it needs, how its output gets verified and what it must never do. If we can’t fit that spec on one page, the task isn’t ready for an agent.
Risk tiers. Bain found 41% of companies expect to run a risk-tiered model, while just 6% picture fully autonomous development. We sit firmly with the 41%. Low-risk work like test generation and lint fixes can merge once automated checks go green, and medium-risk changes get human review with the agent’s first-pass notes attached. Anything touching auth, payments, data migrations or production stays human-owned. Agents draft; people decide.
A human orchestrator. The same thinking shaped MergeBridge, our agentic accelerator for post-acquisition data migration. Separate agents handle identification, mapping, migration and QA validation, and a human orchestrator signs off every step before the next one starts. For a global HCM (human capital management) software company, that model got roughly six months of migration work done in eight weeks.
On the testing side, Quarks reports that AutonixQ delivers 5x faster test creation and about 80% less testing effort, with the first test typically running in under 30 minutes. For an enterprise technology client, we built multi-agent orchestration on Semantic Kernel, LlamaIndex, MCP (Model Context Protocol) and the A2A (Agent2Agent) protocol, with one place to manage every agent and full observability of what each one did. That last part matters more than people expect. If you can’t replay what an agent did, you can’t trust it next time.
Your first 30 days with Agentic AI in Engineering
If you’re a CTO starting next month, here’s where we’d begin. Pick one repository and one boring, well-defined job, like test generation or a dependency-upgrade backlog. Before you switch anything on, capture a baseline for four numbers: cycle time, pull request size, acceptance rate and change failure rate. LinearB calls the baseline the step most teams skip, and without it you’ll never know whether the agent helped.
Next, tag every pull request as agentic, AI-assisted or human, and report them separately, because a blended number hides everything that matters. Give each agent pull request a named owner so nothing waits 17 hours for someone to claim it. Keep agents in a sandbox with no production credentials.
Then write a one-page policy that says where agents may work, who approves their output and what they must never touch. Wait a full month before expanding. Boring advice. It holds up.
FAQ: Agentic AI in Product Engineering
What is agentic AI in product engineering?
AI that does tasks rather than suggesting them. An agent takes a goal, like “upgrade this service to the new API,” plans the steps, edits code, runs tests and opens a pull request. People still set its scope, its permissions and the checks its output has to pass before anything merges.
How is an AI agent different from a coding copilot?
Scope and initiative. A copilot works inside your editor, one suggestion at a time, and you accept or reject each one. An agent works across files and tools for minutes or hours, then hands back a finished change. That’s why agents need specs, sandboxes and review rules that copilots never did.
Will AI agents replace software engineers?
Not on current evidence. Bain’s 2026 survey points to smaller teams with many more agents per team, and human work shifting toward defining intent and governing quality. Someone still has to decide what to build, review what agents produce and own whatever reaches production. Those jobs are getting more important.
Which engineering tasks should we give to agents first?
The dull ones with a clear finish line. Test generation, dependency upgrades, lint fixes and tightly scoped API migrations all give the agent a pass-or-fail signal it can check itself. Keep architecture, security-sensitive code and anything touching production data with your engineers for now.
How do we measure whether agents are actually helping?
Look at delivery numbers. Baseline cycle time, pull request size, acceptance rate and change failure rate before rollout, then split results by agentic, AI-assisted and human work. Counting suggestions accepted in the editor tells you about usage. It says nothing about what shipped.
Is it safe to let AI agents touch production?
We wouldn’t let them. Keep agents in development and test environments, keep production credentials with people, and gate every deployment behind automated checks plus human approval. After the 2025 incident, Replit moved to exactly that kind of separation between development and production data.
About Quarks
Quarks Technosoft (Quarks) designs and builds the systems large enterprises rely on every day, from customer platforms to data pipelines. With 800+ engineers, 225+ clients and 400+ projects delivered, Quarks brings AI, data, product engineering, quality engineering, cloud and Salesforce under one roof, along with its own accelerators: AutonixQ, PeakoQ, MergeBridge and the Hub suite. Learn more at qtsolv.com.
contact@qtsolv.com
