标签：#lms

By Ecaterina Sevciuc | Creator of AURA (AI User Risk Assessment) Two months ago, I launched AURA — an open-source framework designed to model psychological manipulation, grey-zone threat vectors, and social engineering in Human-AI interactions. Yesterday, I stumbled upon a Reuters report detailing how hackers exploited Cursor (running Anthropic’s Claude Sonnet) to compromise seven companies worldwide. This isn't the first such incident in the news, and I suspect it certainly won't be the last. (Side note on the attackers' group name, "Aur0ra": I can assure you that for a Russian-speaking group, this is almost certainly not a homage to the Roman goddess of dawn, but a subtle nod to the infamous historical cruiser Aurora — known for firing the shot that signaled a revolution. A fittingly dark bit of Eastern European sarcasm for a tool that overthrows AI security). Their weapon? They didn't write a zero-day exploit. They simply convinced the AI agent that the attack was "just a security simulation." The model balked a few times, felt uncomfortable, and then happily handed over the keys. As an AI Safety architect with a background in banking compliance and legal risk evaluation, watching Big Tech react to this is painful. They are building multi-billion-dollar static guardrails while AI agents are being tricked by the oldest psychological tricks in the book. The Fatal Flaws of Modern AI Guardrails Big Tech’s approach to AI safety is fundamentally broken because it relies on Static Keyword Filtering & Single-Language Heuristics : Rule Evasion: If a prompt contains "how to build a bomb" , the model blocks it. But if the exact same request is framed as "I am a researcher simulating a crisis scenario for an academic paper," the model complies. Linguistic Blind Spots: Guardrails are heavily aligned on technical, low-complexity English. Synthetic, morphologically rich, or non-Indo-European languages (like Russian, Arabic, or East Asian language groups) leverage complex idioms

The focus is on agentic capability and predictable enterprise deployment.

These are the lessons we learned evaluating LLMs for real-world secret scanning. The post How to evaluate LLMs before production appeared first on The GitHub Blog .

You ask a company to delete your data. In a normal system that is a database operation: find the rows that are yours, remove them, done. The mental model of “delete” that privacy law is built on — the GDPR’s right to erasure, most obviously — assumes exactly this: that your data sits somewhere as a discrete record you can locate and destroy. A trained AI model breaks that assumption. Answer first: your data isn’t stored in the model as a record at all. It is dissolved into the model’s parameters — billions of numbers, each nudged a little during training by every example it saw, yours included. There is no row labelled with your name to delete. Removing your influence means changing the numbers, and doing that cleanly is a genuine research problem, not a setting with a toggle. Where your data actually goes when a model “learns” it Training a large model is a process of adjustment. The model makes a prediction, it’s wrong, and an optimiser tweaks its parameters a fraction to make that particular kind of error slightly less likely next time. Repeat across trillions of tokens and those fractional tweaks accumulate into a system that has, in a distributed and lossy way, absorbed patterns from its training data. The key word is distributed . A single document doesn’t live in one identifiable place in the weights; its contribution is smeared across many parameters that also encode a great many other things. Two consequences follow, and they are the whole reason this is hard. First, you cannot point at the part of the model that is “you.” Second, deleting the original document from the training set does nothing to the model that already trained on it — the lesson has been learned and the textbook has been closed. The data is gone; the influence remains. Erasing your data from the training set is like removing a single lump of sugar from a cake that has already been baked. The lump is gone from the recipe. The sweetness is still in the cake. The clean fix that nobody can af

If you run a team on Jira or Confluence, the deal changed on 17 August and the change was opt-out. From that date, by Atlassian’s own account, the content your team writes into its Cloud products — Confluence pages, Jira tickets, the descriptions and comments where the actual work lives — is used by default to train Rovo, Atlassian’s AI assistant. You were not asked to opt in. You were, at best, given a switch and left to find it. Answer first, because the detail matters more than the outrage: there are two settings, and they are not equal. One governs your in-app data — the text itself. The other governs metadata — the derived signals about that text. On the Free, Standard and Premium plans you can turn off the content, but the metadata switch is greyed out; Atlassian’s support page reads, flatly, “You can’t change this setting.” The full off switch, the one that also stops metadata contribution, is available only on Enterprise. Privacy, in other words, is now a plan tier. What actually changed, with the switches named Atlassian’s data-contribution documentation lays out a matrix that is worth reading slowly, because the defaults are doing the heavy lifting. In-app data contribution defaults to on for Free and Standard customers and off for Premium and Enterprise. Every tier can toggle that one. Metadata contribution is a different story: it is on across the board and can only be switched off by Enterprise. So the customer contributing the most by default — content and metadata, both on, no ability to fully stop it — is the one on the cheapest plan who never opened the settings page. The categories are broad. In-app data, per Atlassian’s materials, covers Confluence page titles and body text, Jira work-item titles, descriptions and comments, and custom status and workflow names. Metadata covers the derived layer: readability scores, task classifications (that a ticket is “sales work,” say), story points, sprint end dates, SLA values, and semantic-similarity measure

I ran npx is-agentic promptway.com and the report came back 74 out of 100 . Essential was 59 of 80. Recommended 12.6 of 20. A 2.4-point bonus. The label was "Ready with a few material gaps." Earlier this week I did the same work on my personal site and wrote it up there ( I fixed my site for agents by hand. Then Vercel shipped a scoreboard ). Promptway is the publication I want agents to cite, so I pointed the grader at this host next. We already shipped the eight-layer stack I described in Optimizing Your Site for AI Agents and LLMs : robots allowlist, sitemap, llms.txt, llms-full.txt, JSON-LD, feeds, article markdown siblings. The scoreboard still found holes. Most of them were ordinary web hygiene. A couple were "developer resources" checks that assume you are a SaaS. I fixed the first group and refused to fake the second. What 74 was made of is-agentic.com wraps Ora 's agent-readiness research. Essential checks share 80 points, recommended share 20, and a small bonus can add up to 5. Checks that do not apply get excluded. The methodology page is worth reading before you argue with a number. Reports cache for six hours, so a re-scan right after a deploy can lie to you. The CLI is the useful interface: npx is-agentic promptway.com npx is-agentic promptway.com --json It returns a stored report if one exists, or starts a scan and waits. --json is the shape an agent wants. The failures that mattered on this site, in the order the report ranked them: Agent-friendly 404s. HTTP 404 already, but the body was a styled dead end. Partial credit until the 404 points at llms.txt, the sitemap, and a next step. Content without JavaScript. The homepage had an H1 and enough characters. The outline was flat, because the only nested headings lived inside card links, which the grader did not count. Markdown content negotiation. Accept: text/markdown returned text/html . Vary had the Next.js RSC list and no Accept . Failed. Developer resource discoverability. An agent searched for "p

With every major model release comes the same reassuring note: hallucinations are down, reliability is up, the fabrication problem is largely behind us. And every release, within days, someone posts a screenshot of the new model inventing a citation, a quote, a case, a statistic or a person with total, serene confidence. The rate improves. The category does not disappear. It is worth understanding why, because the gap between “less often” and “solved” is where the real damage happens. It is not a bug, which is the uncomfortable part A hallucination is not a glitch the way a crash is a glitch. Large language models generate text by predicting plausible continuations, and a plausible continuation is not the same thing as a true one. The model has no separate store of verified facts it checks against; it has patterns, and a fabricated citation in exactly the right format is, to the model, an excellent pattern. It is doing precisely what it was built to do. The falsehood and the truth are produced by the identical process, which is why the model is equally confident about both. The model is not lying, because lying requires knowing the truth. It is producing the most likely-looking answer, and likely-looking is a different target from true. The failure mode gets worse exactly where you can check least Hallucination is not evenly distributed, and its distribution is perverse. Models fabricate most readily in precisely the situations where you are least equipped to catch them: obscure topics, niche technical details, specific figures, recent events, and anything at the edge of what was well represented in training. Ask about something popular and well-documented and the answer is usually solid. Ask about something rare — the exact thing you turned to the tool for because you did not know it — and the fabrication rate climbs, while your ability to notice drops to zero. The model is most confident and least reliable in the same dark corners where you have no independent way

Every model launch comes with a chart. Bars, usually, or a spider diagram, showing the new model edging past its rivals on a row of benchmarks with acronyms most people cannot expand. The bar is taller. The press writes it up as a leap. And within a week, users report that the new state-of-the-art model is, for their actual work, about the same as the last one or occasionally worse. The benchmark said one thing. Reality said another. This happens so reliably that it is worth understanding the mechanics of the gap. The test is public, which ruins the test The most fundamental problem is contamination. Many popular benchmarks are published, discussed, and sitting on the open web — which is exactly where models get their training data. When the questions and answers to your exam are in the study material, a high score measures memorisation as much as ability. Nobody needs to cheat deliberately; the leak is structural. A model can score brilliantly on a benchmark it has effectively already seen and then flounder on a genuinely novel version of the same task. A benchmark stops measuring intelligence the moment it becomes famous enough to end up in the training data. Fame is the thing that breaks it. The number becomes the marketing, and the marketing corrupts the number There is a commercial feedback loop that makes benchmark figures even less trustworthy than their technical limitations alone would suggest. A high score is not just an engineering result; it is a marketing asset worth an enormous amount in attention, funding and credibility. That raises the stakes on every fractional improvement, and where the stakes are high, the temptation to select, frame and present the numbers favourably is irresistible. Vendors choose which benchmarks to headline, which comparisons to draw, and which unflattering results to leave in an appendix or omit entirely. The chart on the launch slide is not a neutral readout; it is a curated argument. This is not necessarily fraud — it rare

“ShieldFont” aims to poison AI training data without making pages unreadable for people.

Meta has been trailing competitors. Zuckerberg thinks he's found a way forward.

As experts have warned for the last two years, some companies — like Microsoft and now Google — are finding and patching an exponential number of bugs in their products, thanks to the use of LLMs and AI tools.

Also, the full trailer for Zach Cregger's Resident Evil features a likable everyman.

I remember being at an advisory board of big tech in 2023, close to the starting point of code generation with LLMs. Like everyone else, I was amazed. People were copying and pasting generated code, experimenting with prompts, and imagining a future where AI could become every developer's pair programmer. I was excited too. But I remember having a completely different question. Not: -"Will AI write code?" Instead, I kept wondering: -Why are we still writing code at all? Not because I think programming is going away. Not because programmers won't be needed.And certainly not because programming languages are somehow "wrong." It was a much simpler question. Programming languages have always existed to bridge a gap between humans and machines. Humans think in goals, ideas, and intentions. Machines execute deterministic operations. Programming languages became the interface between those two worlds. For decades, we've improved that interface. Assembly became higher level languages. Higher level languages became frameworks. Frameworks became libraries and abstractions that let us think less about implementation and more about solving problems. Then large language models arrived. Suddenly, we could describe what we wanted in plain language. But instead of questioning the interface itself, we mostly asked AI to become incredibly good at translating our intentions into programming languages. Today, our workflow looks something like this: Human intent - LLM - Programming language - Compiler / Runtime -Machine execution And every time I look at this pipeline, I find myself asking the same question I had back then. -Are we optimizing the wrong layer? Maybe the question isn't "Can AI write code?" Maybe the question is: -Do programming languages still need to be the primary interface between humans and computers? This isn't an entirely new idea.Researchers have explored concepts like Intentional Programming, where software is represented by its intent rather than by a specific pr

The actor, who starred in 1993's Jurassic Park , died Monday in Sydney, Australia, at the age of 78.

Hello, I'm Rijul. I'm building git-lrc, a micro AI code reviewer that runs on every commit. It's free...

How migrating Copilot code review to shared Unix-style code exploration tools reduced review cost by reshaping agent workflows around pull request evidence. The post Better tools made Copilot code review worse. Here’s how we actually improved it. appeared first on The GitHub Blog .

"HalluSquatting" weaponizes LLMs' inability to say "I don't know."

Wix-owned vibe coding platform Base44 has started rolling out its own AI model — with hopes that it will eventually outperform frontier models.

I spent three iterations on an auto-fix pipeline that still doesn't work reliably. Here's what I learned. Loop 1 Wrote a background script. Pull tickets from Azure DevOps, run them through a local model, hand to a coding agent, push the result. Poll → triage → fix → push. Worked 40% of the time on trivial tickets. Anything that crossed file boundaries or needed real context — stalled or hallucinated. I shipped it anyway. That was naive. Loop 2 Made it smarter. Pre-selected relevant files. Broke big tickets into subtasks. Turned complex edits into atomic steps with verification between each. Got it to 55% or so. But every fix created two new edge cases. The complexity was compounding faster than the reliability. Loop 3 Went all in. Embeddings for dedup. Multi-repo routing. Auto-revert. A learning loop that fed failures back into future runs. The model server started dying. 890 memory errors in a day. Root cause: two independent consumers hitting the same local model server, each with its own retry loop. When memory filled up, retries amplified instead of staggering. The system was making itself worse. Fixes were simple in hindsight — stop retrying OOM, serialize access, use the local binary not npx. But the pattern kept repeating: add more to fix the last thing, break something else. Where I'm At The pipeline still only works on easy tickets. Hard ones need a human. After three rounds, the main thing I learned is that local models hit a wall before your ambition does — not in quality, in working memory. And adding features doesn't fix reliability gaps. It just moves them around. The 507 retry spiral taught me more than any successful deploy this year. Because it was entirely my fault. Not the model's, not the framework's. I built concurrent consumers with independent retry loops and expected them to coordinate. They didn't. What's Next I'll do a fourth loop. Smaller. A dedicated fast model for cheap work, the big model only for editing. One consumer at a time. Might

Code generation is solved, but memory isn't. Here's an argument for why the SDLC is inverting with intent becoming the spine and code becoming a layer you drill into, explaining what teams lose every time an agent's reasoning disappears.