标签：#llms

SearchLeak exploit shows why the industry's approach to LLM security fails over and over.

A new repository-level dataset, published on GitHub under CC0-1.0, helps researchers and developers discover multilingual developer content across READMEs, issues, and pull requests. The post Accelerating researchers and developers building multilingual AI with a new open dataset appeared first on The GitHub Blog .

Alerts are more trustworthy and actionable when noise is reduced. See how we improved the verification step with context-aware LLM reasoning. The post Making secret scanning more trustworthy: Reducing false positives at scale appeared first on The GitHub Blog .

A useful technical idea, repeated often enough, eventually generates an unuseful philosophical claim. The current example is grammar-constrained decoding. The technique is straightforward — at each generation step, the language model's next-token logits are masked so that only tokens whose continuation can satisfy a formal grammar remain selectable; the output is, by construction, structurally valid. JSON parses. SQL is well-formed. Function-call signatures match. There is a real engineering payoff and a healthy ecosystem of libraries that deliver it. The drift is not in the engineering. It is in the rhetorical move that follows the engineering. A growing corner of 2025-2026 AI writing argues, more or less explicitly, that constraining a model's output is making the model approach meaning — that filtering linear sequences is somehow building structure, and that structure is somehow building understanding. I want to take that drift seriously, because it is the same conflation Chomsky and collaborators flagged in their March 2023 essay in the New York Times , and the engineering literature on constrained decoding agrees with Chomsky on the substantive question, even when the marketing copy doesn't. What grammar-constrained decoding actually is A language model produces output one token at a time. At each step, the model emits a probability distribution over its vocabulary, and the decoding strategy (greedy, top-k, nucleus, etc.) picks one token. Without modification, the model is free to emit any continuation; the resulting text might happen to be valid JSON, or it might not. Grammar-constrained decoding intervenes in that step. A formal grammar — typically a context-free grammar, sometimes a regular expression, sometimes a JSON schema or Pydantic model — defines what counts as valid output. At each generation step, the constraint engine computes which next tokens could lead to a continuation that is still satisfiable under the grammar, masks the logits for all other

Estonian government benchmark shows how dozens of models combat Russia's "strategic narratives."

Token Budgeting: Optimizing Generative AI Costs and Performance Modern generative AI applications offer unprecedented capabilities, yet their operational costs can quickly escalate. The primary driver of these costs, alongside computational resources, is token consumption . Understanding and implementing effective token budgeting strategies is not merely an optimization; it is fundamental to building scalable, efficient, and economically viable AI systems. The Economics of Tokens Tokens are the atomic units of text that large language models (LLMs) process. Whether you're sending a prompt (input tokens) or receiving a response (output tokens), each token incurs a cost. This cost varies by model, but the principle remains: more tokens mean higher expenses and often, increased latency due to longer processing times. Efficient token management directly impacts your application's bottom line and user experience. Strategic Pillars of Token Efficiency Optimizing token usage requires a multi-faceted approach, focusing on both input and output, as well as the underlying model choices. 1. Input Optimization: Crafting Smarter Prompts The most direct way to save tokens is to be judicious with the information sent to the model. Every word in your prompt counts. Concise Prompt Engineering : Avoid verbose instructions or unnecessary conversational filler. Get straight to the point. Instead of: "Hey AI, I was wondering if you could please help me summarize this really long article I have here. It's about quantum computing. Could you make it brief, maybe just a few sentences?" Opt for: "Summarize the following article about quantum computing in three sentences: [Article Text]" This significantly reduces input tokens without sacrificing clarity. Context Window Management : LLMs have a finite context window , the maximum number of tokens they can process at once. Sending an entire document when only a specific section is relevant is wasteful. Employ techniques like: Summarization : P

Fine-tuning tests show "bias ... toward confidently representing the claims as true."