今日已更新 339 条资讯 | 累计 19899 条内容
关于我们

Can LLMs save themselves from verbosity?

Benjamin Savoy 2026年06月10日 02:19 4 次阅读 来源:Dev.to

« Je n'ai fait celle-ci plus longue que parce que je n'ai pas eu le loisir de la faire plus courte. » — Blaise Pascal, Lettres provinciales , Lettre XVI (1656) "I have made this one longer only because I have not had the leisure to make it shorter." Pascal's joke is the whole problem: the short version is the expensive one. LLMs lean the other way, they pad. So the question is whether a model can rein in its own verbosity, and what the trimming costs when the deciding clause is buried: "…shall not disclose, except to affiliates who…" Drop the "except," and the answer flips. The test We use ContractNLI: real NDAs, each with expert Entailment / Contradiction / NotMentioned labels. The clauses that decide a label, the buried "negation-by-exception" conditions, we tag as traps . The metric is decision-survival , and it's judge-free: answer from the full document (the ceiling), compress, answer again, score by exact match against the expert label. Survival is the fraction of full-document-correct answers that stay correct after compression. Compression is blind to the question and computed once per document. Three compressors on Groq ( llama-3.1-8b , qwen3-32b , gpt-oss-120b ), one fixed reader ( llama-3.3-70b ), 400 items across 61 NDAs, two prompts: naive ("Summarise this") and effortful (a careful lossless instruction). The raw ceiling is 66%, but 87% on traps, an artifact of the label mix, which is exactly why we report survival rather than accuracy. Finding 1: Prompt engineering is still alive Decision-survival on trap clauses: Compressor naive effortful llama-3.1-8b 57% 74% qwen3-32b 88% 93% gpt-oss-120b 91% 95% The weak model jumps +16 points on traps; the capable ones improve slightly. The payoff from a better prompt is largest exactly where capacity is scarce. Finding 2: The traps catch out simpler models Decision-survival on ordinary (non-trap) clauses: Compressor naive effortful llama-3.1-8b 87% 87% qwen3-32b 88% 94% gpt-oss-120b 94% 91% The small model isn't

本文内容来源于互联网,版权归原作者所有
查看原文