今日已更新 367 条资讯 | 累计 38644 条内容
关于我们

Threat modeling LLM tool-calling

Bellamer 2026年09月02日 17:36 0 次阅读 来源:Dev.to

Every field above is part of the contract enforced by publisher validate . This post exists so the pipeline always has something real to plan against. Why tool-calling changes the threat model A language model that only emits text has one output channel: the reply. A model that can call tools has as many output channels as it has tools, and each of them is reachable by anything that can get text into the context window — a fetched page, a code comment, a file name, a CI log. The useful framing is that the context window is an untrusted input, and every tool is a sink . Prompt injection is not a new vulnerability class so much as a confused-deputy problem wearing a new hat. Three questions per tool For each tool exposed to a model, write down: What does it read? A tool that reads secrets turns any injection into an exfiltration primitive, whether or not the model "intends" it. What does it change? Distinguish reversible writes from irreversible ones. Deleting a branch and deleting a production table sit in different buckets. Who authorised it? Authority should ride with the request, not with the agent process. An agent running as a service account is an agent with the union of every user's permissions. A minimal mitigation set Scope credentials per invocation, not per deployment. Make irreversible tools require an out-of-band confirmation that the model cannot itself produce. Log the full tool-call payload, not a summary. The summary is written by the thing you are investigating. None of this is exotic. It is ordinary least-privilege design applied to a component that happens to take instructions from strangers.

本文内容来源于互联网,版权归原作者所有
查看原文