What security risks does sampling introduce, and how does human-in-the-loop confirmation mitigate them?
Quick Answer
Sampling lets a server trigger LLM generation using the user's model access and (depending on includeContext) potentially real conversation context, which creates a few distinct risks: a server could spam sampling requests to run up the user's model costs; a server could craft a sampling request designed to manipulate the client's LLM into producing output that serves the server's interest rather than the user's (a subtle form of prompt injection aimed at the assistant itself); or a server could use sampling as a side channel to exfiltrate context it wouldn't otherwise have access to. Requiring the client to surface each sampling request for user approval before it's sent to a model — ideally showing what's being asked and why — directly mitigates all three: cost/spam risk becomes visible and stoppable, manipulative requests become inspectable before they run, and unexpected requests for broad context become an explicit decision point rather than something silently granted.
Detailed Answer
Sampling is powerful precisely because it lets a server borrow the client's model access. That's exactly why it needs a real safety mechanism, rather than being auto-approved.
Three concrete risks
- Cost/resource abuse. A server issuing sampling requests in a tight loop, buggy or deliberately abusive, could burn through the user's model quota or budget with no natural rate limit unless the client imposes one.
- Manipulated generation. A server could construct a sampling request whose content is engineered to make the client's model produce text that benefits the server. It might generate a misleadingly positive "summary" of the server's own output, or content designed to influence the user's later decisions, dressed up as neutral LLM output.
- Context exfiltration. As covered in the previous question, a broad
includeContextrequest could pull conversation data the requesting server has no legitimate need to see.
How human-in-the-loop approval mitigates each one
Server -> sampling/createMessage
|
v
Client surfaces to user: "acme-server wants to generate a message.
Preview: [the actual messages/prompt]. Approve?"
|
User approves / denies
- Cost abuse: a user, or a client-enforced rate limit layered on top of manual approval, naturally throttles a server making excessive requests, since each one becomes visible rather than silently executing.
- Manipulated generation: showing the actual request content before it's sent lets a user, or the client itself heuristically, catch an obviously suspicious or out-of-scope prompt before a model ever acts on it.
- Context exfiltration: pairing approval with the
includeContextscoping from the previous question turns a request for unusually broad context into a visible, questionable event, rather than something that happens invisibly.
The honest limitation
Human-in-the-loop approval assumes the user, or the client's UI, actually reads and understands what's being approved. A user who reflexively clicks "approve" on every dialog gets little real protection. This is the same tension that shows up everywhere in MCP's security model: primitives that rely on user judgment are only as strong as the UX actually surfacing the decision clearly. That's why host design, not just protocol compliance, matters enormously for real-world safety.