The enterprise world is currently obsessed with “Agentic AI”—systems that don’t just summarize text, but actually act. Yet, most organizations are making a fundamental mistake: attempting to “bolt on” these sophisticated agents onto fragmented, decade-old legacy architectures.
Today we discussed with Kapil Verma, a Software Engineer at Google, why the industry must stop “patching” the past and start architecting a future where AI is the primary operator.
– You’ve argued that the current approach to AI adoption in large enterprises is “doomed to fail.” Why is the “bolt-on” strategy so dangerous?
– It comes down to what I call the “Hallucination of Action.” For years, the industry standard has been incrementalism — wrapping new tech in a middleware layer and plugging it into a legacy CRM or ERP. While that worked for basic chatbots, an autonomous agent is only as good as the environment it operates in.
When you layer an agent on top of fragmented data silos, the agent might understand the user’s intent, but it becomes paralyzed by the “tool-switching” friction of legacy systems. It knows what to do, but it can’t execute because the underlying pipes are too rigid. To move from simple assistance to true agentic autonomy, we must stop layering and start rebuilding from the ground up.
– In your experience at Google, have you seen a specific project where this “patching” approach backfired?
– Absolutely. I saw this firsthand during our work on Telephony project, a critical internal system at Google. I initially fell into the “Incrementalism Trap,” attempting to “bolt on” AI capabilities to save time.
However, I quickly hit a wall of exponential complexity. To get the new AI to communicate with twenty different legacy modules, I had to build an endless series of “adapters.” My team was spending 80% of its energy on “plumbing” rather than intelligence. I called this the “Complexity Tax.” As a result I went back to the design and reengineered the solution with creating ground-up native AI rebuild and that was really the game changer. This decision reduced development complexity, enabled the successful deployment of AI features that resulted hours of productivity gains. The system has since become a critical component of Google’s internal operations.
– How did you pivot from that “Complexity Tax”?
– I challenged the status quo. I ran a comparative analysis: continuing the patchwork versus a ground-up rebuild. The results were startling. By choosing to architect a native AI feature from scratch, I estimated I could complete the project in 10% less time than the “patch” would have taken.
By ignoring the sunk cost of the legacy system, I delivered a system that was native and fluid. That decision ultimately led to the delivery of AI features like Smart Replies and Summarization that save our agents an estimated four hours of manual work every single day. It proved that the “risky” choice — starting over — is often the most efficient one.
– The Reporter: What are the non-negotiable engineering pillars for this new “AI-First” architecture?
There are three core pillars:
- Unified Omnichannel State-Machines:I must stop building separate “channels” for email, chat, and voice. An agent needs to be “omnipresent,” maintaining total context in a single state-machine regardless of where the interaction started.
- Prescriptive RAG (Retrieval-Augmented Generation):Standard RAG is too passive—it just finds info. While building AI features for Google’s CRM agent platform, I shifted to a “Prescriptive” engine. It doesn’t just retrieve a help article; it interprets account status and policy to prescribe and execute the next logical step.
- Deterministic Boundaries for Non-Deterministic Agents:Since AI is non-deterministic, the architecture must be the guardrail. I built Model Performance Tracking systems that monitor confidence levels in real-time. If an agent hits a 70% confidence threshold, the system automatically hands over a “state-log” to a human, ensuring the transition is seamless rather than a “fail and restart.”
– You often speak about a “Vision Gap” among leadership. How can big companies bridge the gap between “adding AI” and “architecting for AI”?
– The question shouldn’t be “How do I add AI to my workflow?” but “How would I build this workflow if AI were the starting point?”
At Google, we found success by building domain-agnostic cores. This allows us to scale from Payments to Autonomous Hardware using the same routing and escalation logic. Startups do this naturally because they have no baggage, but enterprise leaders must have the courage to treat their legacy stacks as the technical debt they truly are.
Even in our specialized projects, like ML analysis for audio call recordings, we didn’t just use one giant LLM. We trained smaller, specialized models for voice quality and intent detection to act as “helpers.” This hybrid approach allowed us to detect subtle customer sentiments with surgical precision — something you can’t do if you’re just “layering” a generic API onto an old system.
– What is the “Bottom Line” for the industry as we move toward 2026?
– Agentic AI is not a plugin; it is the next evolution of the operating system. The competitiveness of the global economy will be decided by who moves beyond “patching the past” and starts building the foundation for true autonomy. If you want agents that save tens of millions of dollars, you have to build a house they can actually live in.
