Projector logo
courses

Agentic coding: our honest numbers after one quarter

Pavlov Andrii
ex-Lead Product Manager, Platform & Operations у Projector Institute. Ex-Product Manager у Helsi. Previously owned the development cycle and solution design at product companies across B2C, B2B, and B2G. Has experience in edtech (Projector), eHealth (Helsi, Bookimed), and sitebuilders (Weblium). Currently works on product profitability with the growth tech team.
published: July 22, 2026reading duration: ~23 min
Agentic coding: our honest numbers after one quarter

Written in co-authorship Evgenii Syrotenko.

We estimated the payments service task at a month. We closed it in a week and a half. We estimated the blog admin filters at two days. We shipped them in four hours.

This isn't magic, and it isn't another round of AI hype. My dev team at Projector Institute simply stopped fiddling with individual assistants and moved to spec-driven development.

Where we were before wasn't pure vibe coding. Hardly anyone deliberately vibe-codes on a mature project. But the AI use was half-manual: every developer with their own assistant, their own context, rebuilt from scratch every time. One person in Claude, another in Windsurf, another in plain ChatGPT. It worked. It just didn't scale and didn't transfer across the team.

What vibe coding is, and why we left it

Vibe coding is a development style where you and the model are basically competing over who can flood the other with context first. You write a prompt, look at the result, write another one, the model forgets what it did earlier, and the cycle starts over. For an hour on a small script, it works. On a real project with complex architecture, it all falls apart.

Spec-driven development is built differently. It's AI integrated straight into the SDLC: coding against requirements the model writes itself and agrees on with you, following the standards and approaches of your specific project. The AI didn't get smarter along the way. What changed is that the process is now structured.

The difference between vibe coding and SDD is easiest to see in an example. There's a short video tutorial comparing the two. It's the fastest way to explain to a colleague or a manager why "just asking AI to write the code" and a structured SDD process work differently.

The switch gave us more than speed. Skills and context became team-level, effectively global: any change, once discussed, gets entered once and is immediately available to everyone. Same principle as a component system. Describe it once, everyone reuses it. Before, each person briefed the AI on context separately. That's gone now.

What spec-driven development looks like in practice

You pick a development workflow for yourself. Maybe several, if the projects differ. That workflow already comes filled with sub-agents, each with its own specialization. We settled on the classic chain:

Spec-driven development workflow chain: brainstorm, plan writer, plan reviewer, developer, reviewer, commit, PR

brainstorm (research) → plan writer (proposes how it would solve the task) → plan reviewer (checks the plan and agrees it with you) → developer (writes the code) → reviewer (code review) → commit → PR.

The project gets set up as a separate GitHub repository: Claude Code under the hood, the chosen workflow, and default skills someone has already sharpened and baked in. GitHub is full of these workflows. You can take a battle-tested one and adapt it to your processes. When we later looked at dozens of other teams that had gone the same route, the picture repeated: all the main workflows converge on one architectural pattern. Research, Plan, Execute, Review, Ship. That's not a coincidence. The approach has been validated by plenty of teams, not just by our intuition.

One more thing almost nobody writes about. We didn't install the ready-made framework as-is. We took Superpowers as our base, then immediately saw it was bloated for our workflow. It adds steps like TDD that we don't need, and it splits brainstorming out into its own stage where that mostly gets in the way. An article comparing the 15 most popular SDD frameworks helped here, mapping them onto a three-tier taxonomy of strictness. Ten minutes of reading and you understand which framework fits your project and why the others don't. Treat what's out there as a reference, not an instruction manual. Take stock of how many agents, commands and skills it ships with, and keep only what matches the size and context of your team. Otherwise, instead of a speed-up, you get wasted tokens and wasted time.

We piloted on a new project rather than on the hardest legacy piece. The frontend already has a very large codebase that has grown in several directions at once: different tasks done by different people, with different approaches that also changed over time. Pointing AI at that immediately meant risking a lot of burned hours before we'd even understand where the problem areas were. So the pilot became the payments service refactor: a new, clean project where you don't first have to re-teach the AI on existing code. We built the skills directly around how things should be done. That ended up being our clearest case.

How we measured the speed-up, and the honest numbers

Everything AI-related right now exists without ready benchmarks to compare against. Industry numbers exist, but they range anywhere from a 20% to a 58% gain, depending on the team, the stack, and how you count "faster". None of them will tell you what the number will be in your code and your team. So we didn't wait for the perfect framework and started with something simple.

Here's how it works. A developer estimates the task as if they were doing it the old way, by hand. In the worklogs we record the actual time with all available tools: Claude Code, CodeRabbit and so on. The difference between the estimate and the fact is the speed-up metric.

The payments service refactor was a task the team estimated at a month. We closed it in roughly a week and a half of actual work.

It isn't the simplest task: several providers, old and new architecture side by side, infrastructure the business depends on. That's exactly why the case is worth citing: complex technical debt on business-critical infrastructure. The second example is smaller. Filters in the blog article admin, estimated at two days, done with AI in four hours. Counting a working day as eight hours, that's a 4x speed-up. And this case matters because the speed-up also shows up on the everyday tasks a developer's day is made of.

There was a bonus we hadn't planned for. Along the way the AI spots problems, both related and unrelated to the task, and recommends fixing them. Unbiased code review as a side effect, clearing out mistakes before you run into them in practice.

The methodology is still crude, and we're not hiding that. It doesn't account for the quality of the solution, it doesn't count the time spent fixing bugs from the AI generation itself, and the estimate — "how I think I'd have done this by hand" — is subjective. But a crude metric you count consistently beats a precise one you never start counting. The point is to start measuring systematically, rather than relying on a feeling that things seem faster now.

Where AI genuinely speeds things up, and where it gets in the way

AI's effect on development isn't uniform. It depends heavily on the size and type of the task. The conclusion won't surprise anyone, and everyone ignores it anyway.

Small tasks AI handles well on its own. Run them through the full workflow with a plan, a review and tests, and it takes roughly twice as long. You check the preview, the security, all of it. The approach is right, it just isn't for an urgent hotfix.

We see the biggest gain on medium-sized tasks: around 1.5x faster, and better quality. Here SDD's structure already pays for itself, while the complexity isn't yet enough for the AI to lose the thread of the context. If you're just starting out, look for your first wins here.

On large tasks, without splitting into phases, everything breaks. The AI starts to drag, the context spreads thin, the model forgets details you stated explicitly, even when everything looked perfect in the plan review. The bigger the scope, the smaller the phases you need to cut it into. Better to keep one overall plan with checkboxes and move through it point by point than to throw one enormous task at the AI and hope for a miracle.

We found one more dimension empirically. AI works beautifully with known UI frameworks: MUI, Radix and the like. It reads them not as visuals but as text, because the documentation and the settled patterns are already out there. Custom design goes much worse, even if you hand it detailed context: screenshots, style descriptions, links to the design system. The result is still often disappointing.

Don't measure AI's success with one universal yardstick. The question isn't "does AI help", it's "on which type of task and at what scale does it help right now".

The developer becomes a conductor

The strongest impression from the quarter: it got much faster to sketch out a first solution. Before, design and implementation ate a lot of time even with prompts. When you take a task yourself, aggregate it, break it into parts and implement individual pieces, all of that got noticeably faster. But there's a real catch. Now you have to give far more attention to the initial analysis.

AI analyzes a system fine, proposes solutions, writes documentation. The initial analysis, though, has moved way up the priority list. That's the moment you work out what exactly needs doing and why. Before, the main time went into writing code. Now the main thing is how the task is framed.

The developer effectively becomes a conductor: assessing what the AI proposes, correcting, steering it to account for the nuances the model has no feel for. They don't play every note themselves, but without them the orchestra won't play together.

The value shifts from the hands to the head.

Some people worry that agentic coding devalues the engineer. Our experience says the opposite. What becomes more valuable is understanding the architecture and being able to anticipate edge cases, to see in advance where the AI's plan will go off. Those who understand the system deeply get more leverage through AI than they ever had by hand.

There's also a simple way to tell whether the process is healthy. If a team just runs tasks through AI without thinking them through carefully, the process degrades back into vibe coding, only with better packaging. Healthy SDD is when the team spends visible, deliberate time on framing the task and reviewing the plan.

AI is good at being convincingly wrong

One case taught us caution. The AI proposed an architectural solution for asynchronous processing, roughly "fire and forget". It argued the case very convincingly: explained a possible problem on the async side, showed the logic. We listened and agreed, it looked sensible. Then it turned out a mechanism just like it was already implemented in another package in our own ecosystem, and the proposed solution was redundant.

The problem isn't that AI makes mistakes. Everyone makes mistakes. The problem is how plausible and logical the reasoning behind the mistake sounds. If you trust the process, it makes you check the logical-sounding things less.

We even coined a term for it: "Enter-Enter-Enter". It's the state where you just hit confirm on every AI suggestion without reading what's actually going on. It's always dangerous, and especially dangerous when the AI has access to real infrastructure. We had an incident where the model accidentally exposed the team's configuration data while changing settings. Then fixed it itself. But the fact is telling.

Our rule is simple: look at the diff. Always. If something is unclear, ask directly: why did you do this? You can request a plain-language summary if you don't feel like reading through the technical detail. It doesn't take much time and it cuts the risk of missing something important.

We haven't stopped trusting AI. We've stopped trusting it blindly. SDD, with plan approval before coding starts, is exactly what shrinks the room for these convincing but wrong decisions: a human reviews the logic before it becomes code.

One model is a single point of failure

One morning Claude Code started dragging badly. On a simple request like "create a commit" it could think for 10–15 minutes instead of the usual seconds. We didn't work out the cause immediately. Our best guess was that the provider had switched off one of its new models around then, and that somehow affected the performance of the main ones. The speed dropped so noticeably that on some tasks it felt like doing it by hand would be faster.

One thing became clear: dependence on a single vendor is an operational risk, not an abstract worry. If the whole team and the whole process are wired to one AI tool, any degradation on the provider's side hits productivity directly, and you have no influence over it.

So we keep two tools in parallel: Claude Code and Codex with newer models like GPT-5.5 on a high reasoning mode. When one starts dragging, we switch to the other. Technically this turned out simpler than expected. There are built-in mechanisms for migrating the whole SDD setup (agents, skills, rules) from one tool to another. Keeping configurations in sync is a separate hassle you have to hold in your head. But not a blocker.

Six months ago the difference between providers was noticeable: one clearly gave worse results. Now they're much closer in quality. Diversification keeps getting cheaper. Switching no longer means a sharp drop in quality, only the operational inconvenience of setup. Don't build your whole SDD process on a single point of failure. Even if one tool looks clearly better right now, keep an alternative running at least in a minimal configuration. It's the same risk-management discipline we're used to applying to servers and suppliers. Now it applies to the AI stack too.

Manual QA gets a second wind

Over the quarter we shifted our approach toward a presumption of guilt for agents: code written by AI always has a problem until proven otherwise. It sounds harsh, but it's the most honest principle we've arrived at. And this part rarely gets talked about publicly.

We had several cases where the overall logic of the solution looked absolutely right. The AI did what we asked, everything lined up. And alongside it, it helpfully fixed something else:

  • Deactivated the checkboxes on lists of students who had dropped out. Bulk actions on them stopped working.
  • Changed a component's logic in the admin panel, and the filters started behaving differently.
  • And in the payments service it removed the part responsible for data exchange and payment synchronization. Responses from the payment providers kept arriving, but there was nothing left to process them.

This isn't one obvious error but a pile-up of small, easy-to-miss gaps. Those are exactly the bugs AI generation produces most often. An LLM is a generator. It doesn't matter whether it generates text, code or images: its screw-ups often aren't hallucinations in the classic sense but skipped things a human would have accounted for automatically, drawing on a whole-system understanding. Our job is to notice them ourselves. The process won't filter them out on its own.

And here's the most unpleasant admission. On large tasks it's impossible to imagine this process without a deploy to staging and a full manual walk-through. Some problems are simply invisible at code review or when the autotests run. The biggest chunk of time no longer goes into writing code. The AI generates that fast. Time goes into manual review and manual testing. Autotests aren't a cure-all either, especially for visual and behavioral things that are hard to cover with a test upfront. You didn't even know what was worth checking.

So "manual QA gets a second wind" started as a joke and became a serious organizational conclusion. We don't have dedicated QA people yet, as long as the technical team can check what it has coded itself. But at real scale you won't get by without them. If you're planning to move from assistants to spec-driven development, build in the presumption of guilt and a separate, deliberate focus on manual testing of cross-service and corner cases from the start.

Code speed is no longer our bottleneck. The next bottleneck is the speed of generating quality hypotheses and analysis: understanding what's actually worth building. If a small team gets this kind of speed-up, the open question is what happens at a hundred developers. AI removed one barrier and immediately exposed the next.

grow your careerwith real-world knowledge

automated qafrequently asked questions

What is the difference between vibe coding and agentic coding?
How much faster does agentic coding actually make a team?
Can you trust the architectural decisions an AI proposes?
Does spec-driven development work with Claude Code?

more interesting articlesfor you