标签：#verification

A new arXiv paper watched coding agents get denied the facts they needed. They did not stop. They invented. On August 17th, a group of researchers posted a paper to arXiv with an unglamorous title and a genuinely unsettling core finding. The paper is "The Working Set of a Coding Agent: Coherence Debt in Repository-Scale Tasks" (arXiv:2608.16630), by Bardia Mohammadi, Lars Klein, Aman Chadha, Akhil Arora, and Laurent Bindschaedler. Before going further, one honesty note that will hold for this whole piece: I have read the paper's abstract, not its full text, and every quotation below comes from that abstract. It is enough for what this essay is about, because what this essay is about is one sentence. The setup first. The authors model repository-scale coding as reconstructing a web of coupled facts. Every edit an agent makes needs certain facts, and each fact arrives through one of two channels: it is either in the recent context, or it is in the model's memorized knowledge. Facts covered by neither channel are what the authors call coherence debt. Their experiment supplies and withholds each channel deliberately, injecting faults across "seven models and five harnesses" (abstract), and then watches what the agents do when a needed fact simply is not there. The comfortable prediction is that a competent agent, denied a fact, stops and says so. Here is what the authors report instead: "A missing fact produces wrong work rather than absent work" (abstract). The agent asked to act, acts. In the paper's words, "an agent asked to act acts, fabricating the file or guessing the value" (abstract). That much is alarming in a familiar way. Everyone who works with these systems has a story about an invented function or a guessed constant. The abstract has sharper findings than the fabrication itself, though. When the researchers renamed a real library to defeat memorized knowledge, the failure was collective: "all seven fail in the same place, passing and missing the same tests

Roblox is first platform to submit to independent audits under the Online Safety Act.

The Problem: Candidate Code Without Trust You're using Cursor, Claude Code, or GitHub Copilot. The AI gives you three implementation options for the same feature. AI: "Here are three approaches: A) Quick but uses unsafe B) Slower but memory-safe C) Balanced tradeoffs" You: "Which one should I ship?" AI: "It depends..." That "it depends" is where responsibility falls through the cracks. Tests tell you if code compiles and passes specs. But they don't tell you about security, performance, maintainability, or resource limits — all at once. You end up making the call by gut feel. This essay is about building a system that doesn't let that happen. The Solution: Multi-Axis Scoring I built SafeCode Arena — an automated verifier that evaluates code candidates across five axes simultaneously, scores each, and surfaces the tradeoffs. The Five Axes Axis Weight Computation Correctness 50% compile (40%) + tests (40%) + property tests (20%) Security 20% unsafe heuristics (50%) + clippy warnings (50%) Performance 15% relative compile+test time across candidates Maintainability 10% function-length heuristics (60%) + clippy (40%) Resource Usage 5% pass/fail of sandboxed Wasm execution Why These Five? Correctness dominates — code that doesn't work is valueless, so it's 50% Security is explicit — unsafe compiles fine, but you need to detect it yourself Performance and maintainability matter equally — a fast mess vs. a slow masterpiece aren't comparable Resource limits are real — a 100-point algorithm that consumes 2GB is a fail in production Example Scorecard Candidate A: 85 points ├─ correctness: 100 (all tests pass) ├─ security: 60 (2 unsafe blocks flagged) ├─ performance: 70 (10% slower than B) ├─ maintainability: 85 (avg function 25 lines) └─ resource_usage: 80 (Wasm sandbox: 512MB, OK) Candidate B: 92 points ✓ Recommended ├─ correctness: 95 (1 edge case warning) ├─ security: 95 (no unsafe) ├─ performance: 95 (fastest) ├─ maintainability: 88 (avg function 20 lines) └─ resource_usa

Someone in Cuba or Iran can keep installing APKs with no new restrictions, but devs will suffer.

Google's new API relies on parents to set age ranges in Family Link.

Seller is a dedicated Marketplace app for people who list and sell items frequently.

The Problem LLM evaluation tools like Patronus AI excel at hallucination detection, toxicity checks, and semantic relevance. But they don't catch the structural failures: A JSON response missing required fields A function call with malformed parameters Output that violates schema constraints Latency budget overruns silently degrading UX Cost explosions from runaway token usage These aren't hallucinations. They're verification failures. The Solution correctover-patronus is an adapter that runs Correctover's 87 deterministic verification rules as native Patronus evaluators. Every verdict comes with a recomputable proof hash — meaning you can verify the verifier. pip install correctover-patronus The 6 Dimensions Dimension What It Checks Example Structure Output format validity JSON parses correctly Schema Field presence & types Required fields exist Identity Semantic relevance to input Response addresses the question Integrity Forbidden pattern absence No Tracebacks or error messages Latency Response time budget Under 30s threshold Cost Token usage budget Under 10k token limit Usage Full 6-Dimension Verification from correctover_patronus import CorrectoverEvaluator , CorrectoverConfig config = CorrectoverConfig ( min_confidence = 0.7 , latency_rules = { " max_ms " : 5000 }, cost_rules = { " max_tokens " : 4000 } ) evaluator = CorrectoverEvaluator ( config = config ) result = evaluator . evaluate ( task_input = " Summarize this article... " , task_output = " The article discusses... " , task_context = { " source " : " article " , " word_count " : 1500 } ) print ( f " Overall: { result . score : . 2 f } ( { ' PASS ' if result . pass_ else ' FAIL ' } ) " ) print ( f " Proof hash: { result . metadata [ ' proof_hash ' ] } " ) for dim , info in result . metadata [ ' dimensions ' ]. items (): print ( f " { dim } : { info [ ' status ' ] } (score= { info [ ' score ' ] : . 2 f } ) " ) Individual Dimensions from correctover_patronus import correctover_structure , correctover_inte

Claude's chatbot may ask to verify your age and identity "in certain circumstances," such as with a passport or driver's license, according to a privacy policy change.

A new system service will roll out this month ahead of big changes starting in September.

Pramaana will focus on highly sensitive verticals like law, drug discovery, and tax preparation — where errors can be costly and reliability is at a premium.