The hidden network cost of going all-in on AI

Business cases for AI usually focus on GPUs, models, and cloud compute. They often treat networking as background infrastructure that will somehow handle the load. However, as AI shifts from pilot projects to production, this assumption is proving to be costly.

The reason is simple, as AI models and autonomous agents constantly exchange data across cloud environments, enterprise applications, APIs, and edge locations, creating traffic patterns that traditional enterprise networks were never designed to handle. 

As AI investment increases, so does the bill. IDC data Opens a new window shows AI infrastructure spending reached $89.9 billion in the fourth quarter of 2025 alone and is projected to approach $1 trillion by 2029. Networking is one of the fastest-growing items in that spending.

READ MORE: What to revoke the day someone leaves your company?

AI business cases miss networking

Networking increasingly influences the costs of large-scale AI deployments, according to Taranvir Singh, research manager for network infrastructure and services at IDC.

“Most AI business cases carefully size processing capacity but consider networking as a fixed cost,” Singh says. “The costs that surprise IT and finance are mostly ongoing expenses. Cloud egress, high-speed interconnect, and cross-region connectivity all increase with AI and agentic AI activity, and that expense shows up later.”

The problem is timing. Decisions about where models run, where enterprise data resides, and how workloads move between cloud and on-premises environments directly affect networking costs. Yet, these decisions are often made only after AI projects are already underway.

“AI ROI now depends heavily on how efficiently data moves and where compute sits,” Singh says. “Those decisions need to be part of early strategy discussions and included in the business case from the beginning.”

Production AI changes the economics

Early AI pilots typically operate within a single region or cloud provider. These pilots utilize limited data, engage a few users, and manage predictable traffic to keep networking costs under control.

Production deployments are very different. AI workloads transfer data across multiple clouds, constantly call APIs, and access enterprise data spread across hybrid environments. Agentic AI heightens these demands. Unlike traditional genAI, autonomous agents continuously interact with apps, APIs, enterprise data, and other agents to complete multi-step workflows, creating traffic flows that are much more dynamic and harder to predict.

READ MORE: Privacy is now a competitive security feature

Singh mentions that actual spending often exceeds projections. “A pilot within one region or one cloud keeps connectivity costs lower,” he explains. “Once AI workflows start moving across applications, APIs, and data between clouds and on-premises, traffic increases faster than what is generally assumed.”

These traffic patterns quickly expose the limitations of incremental network upgrades. Organizations that manage costs effectively use telemetry to identify which AI workloads drive network traffic and costs. This visibility enables them to make better choices about data movement and workload placement, rather than discovering cost overruns later.

The same visibility issue extends beyond networking. A new wave of startups is building specialized AI cost-management tools to help businesses pinpoint which models, applications, and teams contribute to AI operating costs.

Network readiness falls behind AI adoption

The pressure is set to grow. By 2035, AI inference is expected to make up 25% of all network traffic, according to Cisco’s Opens a new window AI networking research. The most significant growth is expected between 2029 and 2032, as the adoption of agentic AI accelerates.

Cheaper AI compute will not ease the pressure. Gartner predicts Opens a new window that by 2030, the cost of running inference on a one-trillion-parameter LLMs will drop by over 90% compared to 2025. Lower inference costs are likely to encourage broader AI adoption rather than reduce overall spending. Additionally, Gartner notes that agentic AI agents can consume five to 30 times more tokens per task than traditional chatbot interactions, significantly raising data movement across enterprise infrastructure.

For IT teams, this means that cheaper compute leads to increased AI usage, not lower infrastructure costs. Enterprise networks will need to handle much more traffic as AI agents become part of everyday business workflows, making capacity planning and cost management increasingly important.

Budget and planning cycles are struggling to keep up. Procurement and refresh cycles often lag AI adoption by 12 to 24 months. Singh states that many organizations are facing unplanned network upgrades because they didn’t account for networking alongside their AI compute plans from the start.

Security requirements add more pressure. An EMA survey Opens a new window of 269 North American IT professionals found that 39% cite security risk as the main networking challenge for AI deployments, followed by budget constraints at 34%. These two issues often reinforce each other, as rushed upgrades can introduce new security gaps while tight budgets delay necessary investments.

What IT leaders need to do differently

Networks have evolved beyond being passive infrastructure. With the rise of AI adoption, connectivity is increasingly becoming one of the most dynamic and expensive components of the enterprise AI stack.

The conversation is no longer just about deploying more GPUs. With expanding AI clusters, networking plays a crucial role in keeping GPUs fully used. Slow interconnects, data movement bottlenecks, and poor workload placement can leave expensive compute resources idle, reducing the ROI on AI infrastructure.

Singh highlights that organizations managing costs effectively view networking as a key part of the AI business case from the outset. They ask early questions about where the compute will be located, how data will move, and what connectivity will be needed at scale—not after projects are already underway.

Instead of treating the network as a fixed cost, they use telemetry to understand actual usage patterns. This helps them optimize data placement, reduce unnecessary egress charges, and plan capacity upgrades before costs spiral.

Enterprises that continue to view networking as a simple cost item risk budget overruns, performance issues, and lower-than-expected AI returns. As AI transitions from experimentation to core business functions, networking decisions will increasingly influence not only application performance but also the economics of enterprise AI itself