标签：#lms

How migrating Copilot code review to shared Unix-style code exploration tools reduced review cost by reshaping agent workflows around pull request evidence. The post Better tools made Copilot code review worse. Here’s how we actually improved it. appeared first on The GitHub Blog .

"HalluSquatting" weaponizes LLMs' inability to say "I don't know."

Wix-owned vibe coding platform Base44 has started rolling out its own AI model — with hopes that it will eventually outperform frontier models.

I spent three iterations on an auto-fix pipeline that still doesn't work reliably. Here's what I learned. Loop 1 Wrote a background script. Pull tickets from Azure DevOps, run them through a local model, hand to a coding agent, push the result. Poll → triage → fix → push. Worked 40% of the time on trivial tickets. Anything that crossed file boundaries or needed real context — stalled or hallucinated. I shipped it anyway. That was naive. Loop 2 Made it smarter. Pre-selected relevant files. Broke big tickets into subtasks. Turned complex edits into atomic steps with verification between each. Got it to 55% or so. But every fix created two new edge cases. The complexity was compounding faster than the reliability. Loop 3 Went all in. Embeddings for dedup. Multi-repo routing. Auto-revert. A learning loop that fed failures back into future runs. The model server started dying. 890 memory errors in a day. Root cause: two independent consumers hitting the same local model server, each with its own retry loop. When memory filled up, retries amplified instead of staggering. The system was making itself worse. Fixes were simple in hindsight — stop retrying OOM, serialize access, use the local binary not npx. But the pattern kept repeating: add more to fix the last thing, break something else. Where I'm At The pipeline still only works on easy tickets. Hard ones need a human. After three rounds, the main thing I learned is that local models hit a wall before your ambition does — not in quality, in working memory. And adding features doesn't fix reliability gaps. It just moves them around. The 507 retry spiral taught me more than any successful deploy this year. Because it was entirely my fault. Not the model's, not the framework's. I built concurrent consumers with independent retry loops and expected them to coordinate. They didn't. What's Next I'll do a fourth loop. Smaller. A dedicated fast model for cheap work, the big model only for editing. One consumer at a time. Might

Code generation is solved, but memory isn't. Here's an argument for why the SDLC is inverting with intent becoming the spine and code becoming a layer you drill into, explaining what teams lose every time an agent's reasoning disappears.

The company warned about dangers of advanced AI far more than rival OpenAI.

LLMs are unpredictable. They hallucinate, leak data, generate harmful content, or refuse legitimate requests. Guardrails constrain model behavior without sacrificing capability. The key is knowing which guardrails matter and which are just noise. Guardrails aren't about controlling the model. They're about controlling the risk. Input validation The most important guardrail. Bad input gets bad output, and bad input can also prompt-inject your system. Strategy 1: Prompt Sanitization Sanitize dangerous patterns early: import re class PromptSanitizer : def __init__ ( self ): self . dangerous_patterns = [ r " ignore\s+previous\s+instructions " , r " system\s+prompt " , r " you\s+are\s+now\s+free " , r " break\s+out\s+of " , ] def sanitize ( self , prompt : str ) -> str : for pattern in self . dangerous_patterns : prompt = re . sub ( pattern , " [REDACTED] " , prompt , flags = re . IGNORECASE ) return prompt This isn't bulletproof. Adversarial inputs are creative. But it catches the obvious ones, and the obvious ones are the most common. Strategy 2: Input Length Limits Length limits prevent token waste and timeouts: class InputValidator : def __init__ ( self , max_length : int = 10000 ): self . max_length = max_length def validate ( self , prompt : str ) -> tuple [ bool , str ]: if len ( prompt ) > self . max_length : return False , f " Input too long: { len ( prompt ) } > { self . max_length } " return True , " OK " Strategy 3: Content Filtering Content filtering blocks policy violations. The patterns here depend on your domain: class ContentFilter : def __init__ ( self ): self . blocked_topics = [ " violence " , " hate speech " , " self-harm " , " sexual content " , " illegal activities " , ] def filter ( self , prompt : str ) -> tuple [ bool , str ]: prompt_lower = prompt . lower () for topic in self . blocked_topics : if topic in prompt_lower : return False , f " Blocked: { topic } " return True , " OK " Simple string matching is fast but imprecise. For production, us

SearchLeak exploit shows why the industry's approach to LLM security fails over and over.

A new repository-level dataset, published on GitHub under CC0-1.0, helps researchers and developers discover multilingual developer content across READMEs, issues, and pull requests. The post Accelerating researchers and developers building multilingual AI with a new open dataset appeared first on The GitHub Blog .

Most testing tutorials ignore e-learning completely. Here's how to build a Playwright test suite that validates your SCORM packages actually work across LMS platforms. Why E-Learning Testing Is Different If you've ever published a SCORM package to an LMS and watched it silently fail — no completion recorded, quiz scores vanishing, navigation broken — you know the pain. E-learning content doesn't behave like a typical web app. It runs inside an LMS-provided iframe, communicates through a JavaScript API (the SCORM Runtime), and its behavior changes depending on which LMS hosts it. Manual QA across even 3-4 LMS platforms is slow and error-prone. In this tutorial, I'll walk you through setting up Playwright to automate SCORM package testing — from basic content loading to verifying API calls and completion status. Prerequisites Before we start, make sure you have: Node.js 18+ installed Playwright ( npm init playwright@latest ) A SCORM 1.2 or 2004 package (a .zip file containing your e-learning content) A local LMS for testing — we'll use SCORM Cloud (free tier) or a simple SCORM API shim Step 1: Set Up a Local SCORM Runtime Shim Testing SCORM content requires an API that mimics what an LMS provides. Rather than spinning up a full Moodle instance, we'll create a lightweight shim. Create a file called scorm-api-shim.js : // scorm-api-shim.js // Mimics the SCORM 1.2 Runtime API that an LMS would expose window . API = { _data : {}, _initialized : false , _calls : [], LMSInitialize : function ( param ) { this . _initialized = true ; this . _calls . push ({ method : ' LMSInitialize ' , param , timestamp : Date . now () }); console . log ( ' [SCORM] LMSInitialize called ' ); return " true " ; }, LMSGetValue : function ( key ) { this . _calls . push ({ method : ' LMSGetValue ' , key , timestamp : Date . now () }); return this . _data [ key ] || "" ; }, LMSSetValue : function ( key , value ) { this . _data [ key ] = value ; this . _calls . push ({ method : ' LMSSetValue ' , key

SCORM was built in the early 2000s for a world of CD-ROMs and Flash. It's 2026 and it still runs 80%+ of corporate e-learning. Here's why, and why generative AI might be the thing that finally breaks the cycle. SCORM Is Everywhere, and Nobody Is Happy About It If you work anywhere near corporate learning, you've encountered SCORM — the Sharable Content Object Reference Model. It's a set of standards that lets e-learning content talk to a Learning Management System: track completion, record scores, resume where you left off. SCORM 1.2 was released in 2001. SCORM 2004 followed a few years later. That's it. The spec hasn't meaningfully evolved in two decades. And yet, almost every LMS on the market — Moodle, Cornerstone, SAP SuccessFactors, Docebo, Absorb — still supports SCORM as a primary content format. Most Fortune 500 compliance training runs on it. Every major authoring tool, from Adobe Captivate to Articulate Storyline to Lectora, exports SCORM packages. It's the TCP/IP of corporate learning: unglamorous, creaky, universally understood. Why It Won't Die: The Network Effect Nobody Talks About People love to write "SCORM is dead" articles. I've been in e-learning engineering for 11 years and I've read that headline at least once a year since I started. SCORM isn't dead because it benefits from one of the strongest network effects in enterprise software. Consider the ecosystem: Authoring tools export SCORM because LMS platforms expect it. LMS platforms support SCORM because authoring tools export it. L&D teams require SCORM because their procurement processes mandate it. Procurement mandates SCORM because it's the only format every vendor supports. Breaking this cycle requires everyone to move simultaneously. That doesn't happen in enterprise software. It especially doesn't happen when "good enough" works and switching costs are invisible but enormous (repackaging thousands of courses, retraining content teams, renegotiating vendor contracts). xAPI (Tin Can) was su

Alerts are more trustworthy and actionable when noise is reduced. See how we improved the verification step with context-aware LLM reasoning. The post Making secret scanning more trustworthy: Reducing false positives at scale appeared first on The GitHub Blog .

After 11 years building e-learning software at a major enterprise, here’s what surprised me when we started shipping GenAI features to real users. The Starting Point Nobody Talks About Most GenAI integration blog posts start with a clean slate — a greenfield app, a fresh codebase, a blank canvas. Real life is messier. When we began integrating generative AI into our e-learning authoring tool, we weren’t starting from scratch. We were dealing with a mature enterprise product — millions of users, legacy architecture decisions made years ago, compliance requirements from Fortune 500 customers, and LMS interoperability standards (SCORM, xAPI) that were designed long before anyone imagined AI-generated content. The challenge wasn’t “how do we call an AI API.” It was “how do we ship AI features into a product that thousands of instructional designers depend on daily, without breaking their workflows or their trust.” Here’s what I learned. Lesson 1: The AI Feature Your Users Want Is Not the One You Think When we first scoped out AI integration, the engineering team gravitated toward the flashy stuff — generate an entire course from a prompt, auto-create assessments, AI-powered slide design. Then we talked to actual users. Instructional designers didn’t want AI to replace their expertise. They wanted it to eliminate the tedious parts of their workflow: Reformatting content across different output types (responsive HTML5, PDF, SCORM packages) Generating alt-text for hundreds of images in accessibility-compliant courses Summarizing lengthy SME-provided documents into digestible learning chunks Suggesting quiz questions from existing content (not generating courses from nothing) The takeaway: don’t let engineering excitement drive your AI feature roadmap. Do 10 user interviews before writing a single line of integration code. The highest-impact GenAI features are usually the boring ones. Lesson 2: Prompt Engineering Is a Product Decision, Not an Engineering Task We initially t

A useful technical idea, repeated often enough, eventually generates an unuseful philosophical claim. The current example is grammar-constrained decoding. The technique is straightforward — at each generation step, the language model's next-token logits are masked so that only tokens whose continuation can satisfy a formal grammar remain selectable; the output is, by construction, structurally valid. JSON parses. SQL is well-formed. Function-call signatures match. There is a real engineering payoff and a healthy ecosystem of libraries that deliver it. The drift is not in the engineering. It is in the rhetorical move that follows the engineering. A growing corner of 2025-2026 AI writing argues, more or less explicitly, that constraining a model's output is making the model approach meaning — that filtering linear sequences is somehow building structure, and that structure is somehow building understanding. I want to take that drift seriously, because it is the same conflation Chomsky and collaborators flagged in their March 2023 essay in the New York Times , and the engineering literature on constrained decoding agrees with Chomsky on the substantive question, even when the marketing copy doesn't. What grammar-constrained decoding actually is A language model produces output one token at a time. At each step, the model emits a probability distribution over its vocabulary, and the decoding strategy (greedy, top-k, nucleus, etc.) picks one token. Without modification, the model is free to emit any continuation; the resulting text might happen to be valid JSON, or it might not. Grammar-constrained decoding intervenes in that step. A formal grammar — typically a context-free grammar, sometimes a regular expression, sometimes a JSON schema or Pydantic model — defines what counts as valid output. At each generation step, the constraint engine computes which next tokens could lead to a continuation that is still satisfiable under the grammar, masks the logits for all other

Estonian government benchmark shows how dozens of models combat Russia's "strategic narratives."

Token Budgeting: Optimizing Generative AI Costs and Performance Modern generative AI applications offer unprecedented capabilities, yet their operational costs can quickly escalate. The primary driver of these costs, alongside computational resources, is token consumption . Understanding and implementing effective token budgeting strategies is not merely an optimization; it is fundamental to building scalable, efficient, and economically viable AI systems. The Economics of Tokens Tokens are the atomic units of text that large language models (LLMs) process. Whether you're sending a prompt (input tokens) or receiving a response (output tokens), each token incurs a cost. This cost varies by model, but the principle remains: more tokens mean higher expenses and often, increased latency due to longer processing times. Efficient token management directly impacts your application's bottom line and user experience. Strategic Pillars of Token Efficiency Optimizing token usage requires a multi-faceted approach, focusing on both input and output, as well as the underlying model choices. 1. Input Optimization: Crafting Smarter Prompts The most direct way to save tokens is to be judicious with the information sent to the model. Every word in your prompt counts. Concise Prompt Engineering : Avoid verbose instructions or unnecessary conversational filler. Get straight to the point. Instead of: "Hey AI, I was wondering if you could please help me summarize this really long article I have here. It's about quantum computing. Could you make it brief, maybe just a few sentences?" Opt for: "Summarize the following article about quantum computing in three sentences: [Article Text]" This significantly reduces input tokens without sacrificing clarity. Context Window Management : LLMs have a finite context window , the maximum number of tokens they can process at once. Sending an entire document when only a specific section is relevant is wasteful. Employ techniques like: Summarization : P

Fine-tuning tests show "bias ... toward confidently representing the claims as true."