What are common performance pitfalls when an MCP server exposes tools that call slow external APIs?
Quick Answer
The most common pitfall is blocking the whole server on one slow call — if a tool handler synchronously waits on a slow upstream API with no timeout, a single hung request can stall the server for every other concurrent session (especially on a naively-implemented remote/HTTP server). Mitigations: set explicit timeouts on every outbound call a tool makes, use async/non-blocking I/O so one slow tool call doesn't block others sharing the same process, and where an operation is genuinely long-running, use MCP's progress notifications (covered under transports) so the client/user gets feedback instead of appearing frozen. Other common pitfalls include not caching genuinely cacheable, slow-to-fetch data (re-fetching a rarely-changing resource from scratch on every read), and returning an unbounded amount of data from a tool/resource (a "list everything" tool with no pagination or size cap), which can both slow the response and blow past the model's context budget once the result is fed back into the conversation.
Detailed Answer
An MCP server's performance problems usually trace back to treating an external dependency as if it were always fast and always available. It's the same class of mistake that shows up in any backend integration, just with an LLM on the other end feeling the latency directly.
The blocking-call pitfall
# Risky: no timeout, blocks the event loop/thread indefinitely if the API hangs
@mcp.tool()
def get_stock_price(symbol: str) -> str:
response = requests.get(f"https://slow-api.example/price/{symbol}")
return response.text
# Better: explicit timeout, async so other requests aren't blocked
import httpx
@mcp.tool()
async def get_stock_price(symbol: str) -> str:
async with httpx.AsyncClient(timeout=5.0) as client:
response = await client.get(f"https://slow-api.example/price/{symbol}")
return response.text
A hung external dependency with no timeout doesn't just fail that one call slowly. On a naive synchronous implementation, it can tie up the whole process, stalling unrelated concurrent tool calls, or for a stdio server, the one session it's serving, indefinitely.
Other common pitfalls
- No caching for slow, rarely-changing data. A tool that re-fetches a company's entire product catalog from a slow upstream API on every single call, when that catalog only changes once a day, adds unnecessary latency to every invocation.
- Unbounded result sizes. A
list_customerstool with no pagination or limit can return thousands of rows. That's slow to produce, expensive to transmit, and, once fed back into the model's context as a tool result, can consume a large fraction of the available context window for a single call. - No progress feedback on genuinely long operations. A tool that takes 30+ seconds with no
notifications/progress(covered in the transports topic) leaves the user staring at what looks like a frozen or crashed assistant.
A practical checklist for any tool wrapping an external API
- Set an explicit timeout, shorter than whatever default the HTTP client library uses.
- Use async I/O (or a thread/worker pool) so a slow call doesn't block unrelated requests.
- Cache anything genuinely cacheable, with a sensible TTL.
- Cap result size, and offer pagination for anything that could plausibly return a lot of data.
- Emit progress notifications for anything expected to take more than a couple of seconds.