Disclaimer: This post was entirely generated by Claude Code (Anthropic's AI coding agent) with support, interaction, and direction by Sheun Aluko, MD.

Is Vercel Slowing Down Your AI Agent? A Data-Driven Latency Study

When you proxy LLM API calls through a Vercel serverless function, every request takes an extra hop: browser -> Vercel edge -> LLM provider -> Vercel -> browser. That overhead adds up — or does it?

We built a 50-turn stress test, ran it against GPT-5.2 through our Vercel-hosted agent, and instrumented every millisecond. Here's what we found.

The Setup

Cortex is a voice-first AI agent application. It proxies all LLM calls through a Vercel serverless API route — a common pattern for keeping API keys server-side and leveraging edge infrastructure.

To measure the real-world cost of this architecture, we needed a sustained, realistic workload — not a single request benchmark.

The Simulation

We used Simi, a declarative workflow simulation framework, to script a 50-turn conversation driven by Playwright. The conversation is a sustained narrative — opening a restaurant for time-traveling pigeons called "Coo-linary Paradox" — with periodic context-retention checks to verify the agent remembers details from earlier in the conversation.

Key parameters:

50 sequential user messages with assertions after each
69 total LLM invocations (some turns triggered multi-step agent loops)
Context growth: ~5K tokens (turn 1) to ~24K tokens (turn 50)
Model: GPT-5.2
Total tokens processed: 1,037,947

Instrumentation

Each LLM invocation records three timing values:

Metric	What It Measures
`server_llm_ms`	Time in the LLM provider (measured server-side)
`client_round_trip_ms`	Total time from request to response (measured client-side)
`vercel_overhead_ms`	The difference: network + Vercel function overhead

All telemetry is captured by our InsightsClient, batched to a cloud database, and exported for analysis.

Results

The Big Picture

The entire 50-turn conversation completed in 9.8 minutes, processing over 1 million tokens at a cost of $0.52.

Here's where the time went:

Component	Total Time	% of Round Trip
Server LLM inference	384s	94%
Vercel overhead	22s	6%

That 6% is inflated by a single event — a 17.7-second cold start on the very first call. Remove that outlier and Vercel overhead drops to about 1% of total time.

Latency Distribution (n=69 calls)

Metric	Min	Mean	P50	P90	P95	Max
Server LLM	1.8s	5.6s	5.1s	9.0s	10.5s	14.9s
Vercel Overhead	28ms	326ms	54ms	108ms	113ms	17.7s
Client Round Trip	1.8s	5.9s	5.3s	9.3s	11.3s	19.9s

The median Vercel overhead is 54 milliseconds. When your LLM calls take 5+ seconds, 54ms is invisible.

Does Latency Grow With Context?

One concern with long conversations is whether latency degrades as context grows. We bucketed the data by conversation stage:

Turns	LLM Calls	Avg Server	Avg Vercel	Max Server
1-10	16	5.6s	1.2s*	14.9s
11-20	12	5.8s	46ms	11.3s
21-30	15	5.9s	64ms	9.3s
31-40	13	5.2s	58ms	8.9s
41-50	13	5.2s	67ms	10.5s

*Turns 1-10 average is skewed by the 17.7s cold start.

Server LLM time stays remarkably stable across the conversation (5.2-5.9s average per bucket) despite tokens per call growing from ~5K to ~24K. GPT-5.2 handles growing context well. Vercel overhead is flat at ~50-65ms after warmup.

Token Economics

Metric	Value
Total tokens	1,037,947
Prompt tokens	1,022,728 (98.5%)
Completion tokens	15,219 (1.5%)
Avg tokens per call	15,043
Total cost	$0.52

Prompt tokens dominate at 98.5% — the agent sends a rich system prompt plus the full conversation history on every call.

What About the Cold Start?

The elephant in the room is the 17.7-second first call. This is a classic serverless cold start — the Vercel function needs to initialize the runtime, load dependencies, and establish connections to the LLM provider.

But this happens once per session. For an AI agent where users have multi-minute conversations, a one-time startup penalty is acceptable. If it's not, you can mitigate it with:

Keep-warm pings — a scheduled request every few minutes to prevent cold starts
Vercel's Fluid Compute — keeps functions warm between invocations automatically
Edge Runtime — faster cold starts than Node.js runtime (trade-off: fewer Node.js APIs)

The Verdict

Don't migrate. The data is clear:

94% of latency is LLM inference — you can't optimize this away with infrastructure changes
Median Vercel overhead is 54ms — less than 1% of a typical round trip
Cold starts are a one-time cost — and can be mitigated without architectural changes

Migrating to direct client-side API calls or a dedicated server would add real complexity — API key management, CORS, infrastructure to maintain — all to save 54ms per request.

Where to Actually Optimize

If you want to make your AI agent feel faster, focus on what actually matters:

Response streaming — Show partial responses as they arrive. This doesn't reduce total latency but dramatically improves perceived latency. Instead of waiting 5 seconds for a complete response, users see text appearing within milliseconds.
Prompt optimization — Prompt tokens are 98.5% of total tokens. Reducing system prompt size or summarizing conversation history would cut both cost and latency.
Smarter agent loops — Some turns triggered 2-3 LLM calls (reasoning + function calls + follow-up). Reducing unnecessary multi-loop turns would save 5+ seconds per avoided call — 100x more impactful than eliminating Vercel overhead.

This study was conducted as part of ongoing performance analysis for the Cortex AI agent platform, built on the TidyScripts project.