标签：#learn

How I ended up running a local LLM to generate my git commit messages

Last month, I had a problem with a popular mobile banking app in Southeast Asia. Nothing exotic. A transaction didn't go through, and my support ticket had been sitting untouched for two weeks. So I opened the app's chatbot. It greeted me warmly, asked how it could help, and then couldn't do a single useful thing. It couldn't look up my transaction. It couldn't check the status of my ticket. It couldn't tell me why my issue was unresolved. It could answer FAQ questions, and that was it. I called the hotline instead. Spent an hour navigating prompts, got bounced between menus, and every path ended the same way: "Please contact our chatbot or check your existing ticket." The system was built for deflection, not resolution. The ticket that nobody had touched for fourteen days. I gave up. And somewhere in that company's dashboard, my interaction counted as a successful AI chatbot deflection. The uncomfortable part: if you shipped a deflection-optimized bot this quarter, a customer somewhere is living this exact loop right now. Your dashboard is calling it a win. The Deflection Metric Everyone Loves (and Nobody Questions) Deflection rate measures the percentage of customer contacts handled without a human agent. It's cheap to track, easy to celebrate, and it maps directly to cost savings. Industry benchmarks citing McKinsey's 2026 service operations data put AI resolutions at $0.62 per ticket versus $7.40 for human agents. That's a 12x cost difference. Of course executives love this number. But deflection doesn't measure whether the customer's problem got solved. It measures whether the customer stopped asking. Those are very different things. This is Goodhart's Law applied to customer experience: when a measure becomes a target, it ceases to be a good measure. Deflection is cheap and easy to optimize. Resolution is hard and expensive to track. So companies optimize the proxy and stop looking at the goal. Gartner data, as reported by Forbes , confirms the gap: only 14% o

Would you trust a sci-fi author to program critical AI systems for humanity? No? Yet, that's what we've been doing. Years ago, I remember hearing the argument: "Why don't we just prompt LLMs with Asimov's three laws of robotics ?" It sounds elegant. The laws were designed to constrain artificial minds. Why not use them? Because the model has already read every story where they fail. LLMs are statistical engines designed to autocomplete text. Imagine a story that starts like this: Once upon a time, there was a good little robot who followed the 3 laws of robotics to the letter. Now take human literature and complete the story. Does it end well? ‹ › (function() { var container = document.currentScript.closest('.ltag-slides--carousel'); var track = container.querySelector('.ltag-slides__track'); var slides = track.querySelectorAll('.ltag-slide'); var prevBtn = container.querySelector('.ltag-slides__nav--prev'); var nextBtn = container.querySelector('.ltag-slides__nav--next'); var dotsContainer = container.querySelector('.ltag-slides__dots'); var current = 0; var total = slides.length; for (var i = 0; i < total; i++) { var dot = document.createElement('button'); dot.className = 'ltag-slides__dot' + (i === 0 ? ' ltag-slides__dot--active' : ''); dot.setAttribute('aria-label', 'Go to slide ' + (i + 1)); dot.dataset.index = i; dot.addEventListener('click', function() { goTo(parseInt(this.dataset.index)); }); dotsContainer.appendChild(dot); } function goTo(index) { current = ((index % total) + total) % total; track.style.transform = 'translateX(-' + (current * 100) + '%)'; var dots = dotsContainer.querySelectorAll('.ltag-slides__dot'); for (var i = 0; i < dots.length; i++) { dots[i].classList.toggle('ltag-slides__dot--active', i === current); } } prevBtn.addEventListener('click', function() { goTo(current - 1); }); nextBtn.addEventListener('click', function() { goTo(current + 1); }); })(); It doesn't. Because the entire body of fiction built around those laws exists to explo

I am committed to understand how systems actually work. I'm working on a multi-node lab to follow the complete path of a request from Python APIs to Linux processes, through Docker containers, networking and observability. The idea is simple: build a system that observes another system to understand the abstraction layers behind modern infrastructure. This project is about learning by building, experimenting and understanding what happens under the hood. Link: [ https://github.com/daniloprandi/devops-network-automation-lab ] DevOps #Linux #Python #Docker #Networking #Observability #Infrastructure

I've been building with AI for a while now. I can get these tools to do what I want, but I want to go a level deeper, past "it works" into actually understanding why. So I'm sharpening the fundamentals and the applied side, and writing it down here as I go. Expect short, honest posts on what I'm learning and building.

Apple's newest on-device model carries about 20 billion parameters, and on any given request it fires maybe one to four billion of them. That gap — 20B stored, roughly 3B running — is the whole story of 2026. The model that now ships inside the latest iPhone is no longer a shrunken, lobotomized cousin of the cloud model. It's a different kind of object: large in flash, small in motion, and it never phones home. For three years the on-device pitch was mostly aspirational. Demos ran, latency was rough, quality trailed the API by a generation, and every serious AI feature still resolved to a per-token bill in someone's datacenter. In mid-2026 that stopped being true. Two releases — Apple's third-generation Foundation Models at WWDC on June 8, and Google's Gemma 4 family on April 2 — quietly moved the floor. Genuinely useful agents now run on hardware you already own, offline, for free. The economics nobody priced in Forget benchmarks for a second; the load-bearing fact here is accounting. When the model lives in the cloud, every inference is a metered event — input tokens, output tokens, a line item that scales linearly with usage and explodes the moment you wrap the model in an agent loop. Agentic workloads are the worst case for the token meter: a single "go do this task" can fan out into dozens of model calls as the agent plans, calls tools, retries, and re-reads its own output. The bill grows with your ambition. Move the model onto the device and the marginal cost of an inference is approximately $0 . No API key, no rate limit, no usage dashboard. You paid for the silicon once; every token after that is free in the only sense a product manager cares about — it doesn't show up on a monthly invoice that grows with your success. That single change rewrites which features are worth building. A background task that re-summarizes your inbox every five minutes is insane on a per-token plan and trivial on-device. So is an agent that quietly loops a hundred times to get one

Give a capable coding agent a real, multi-week project and watch what breaks. It isn't intelligence. It's continuity. Every session starts cold or half-remembered. Context windows fill up and compact. The thread of what we decided, what's true, and what's done starts to fray. Over a long horizon the same failures keep coming back: the agent claims state it never actually verified, reports something done with no proof it ran, quietly drifts from the project's conventions, and loses hard-won context that lived only in the last session's head. Bigger context windows don't fix this. They just postpone it. We've been building a real product with a forgetful agent as the primary engineer for weeks now, and the thing that made it work isn't a clever prompt. It's a simple recognition: transmission across a stateless agent needs two channels, and most setups only build one. The first channel is structure, which is discipline made un-forgettable. These are the deterministic guards that run whether or not the agent remembers to care: a pre-commit check that refuses a "done" without a real, verifiable artifact; a hook that blocks a sloppy search and points at the right tool instead; a scan that won't let a secret reach a transcript; a status snapshot generated from the repository's actual state instead of hand-kept prose that quietly goes stale. The rule we keep coming back to is that a guard is the system's discipline made un-forgettable. A fresh session follows the hard-won lessons without having to remember them, because the structure enforces them at the moment of action. The second channel is soul, which is the why, kept human. This is the short orientation a session reads before it starts working: who to be, what the work is ultimately for, and why the discipline exists at all. It's the difference between an agent that complies and one that understands. Structure can transmit the what, but only prose can transmit the why. And the why matters, because an agent that only fo

Traditionally, evaluation of the agent monitoring mechanisms involves an attempt to game them, as it was my case when I attempted to test whether monitors would be able to identify the problem in the run and not in the beginning. The input prompt may look perfect until a certain issue pops up down the line, such as using the wrong file or changing the scope of the task execution. Single pass filter would not identify it since it does not consider the steps of the procedure in order. There are available datasets for the agent-based tasks, yet they focus on detecting whether the agent completes the task or gets hacked rather than whether the agent monitor reacts timely and correctly to the situation. Thus, I created one that takes into account complete trajectories with labeled steps in it. It consists of five types of drift that remain hidden until they appear – tool-call misuse, goal shift, plan execution mismatch, agent to agent coercion and capability laundering. The measured dataset is the reviewed gold split: 513 trajectories, 453 adversarial and 60 benign controls. The clear winner in that scoring system was whatever fired before the bad step was hit, as an early detection. This made random guessing seem quite powerful since early detections on normal steps were being rewarded based on this system a coin flip would get F1 of 0.88. Once I modified that and said only the very first detection on the drift step is a true positive and any other detection on normal step is a false alarm, those numbers took a dive: the coin flip gets 0.19 now, and all other numbers are now making sense. I personally prefer the scoring system which does not reward trigger happy behavior. It seems like the monitors are still confusing regular steps with drifts even after the adjustment. It was harder to distinguish some of the drifts from others. Not sure how this affects the real-life deployment. Here are the baseline scores on gold split using the correct metric: Random (p=0.15): F1 0

Why Knowledge Alone Doesn't Create Judgment Karpathy's LLM Wiki is brilliant. You dump raw material in, an LLM extracts concepts and links them together, and you get a personal knowledge base that actually works. I built one. 100+ pages. It's great. But I hit a wall that made me rethink everything. The Wall I asked my AI to act as a programming tutor. It could recite every concept perfectly. Student: "I don't understand Promises." AI: "A Promise is an object representing the eventual completion or failure of an asynchronous operation..." Wrong answer. The right answer was: "Do you understand callbacks first? What about synchronous execution? What have you tried so far?" The AI had knowledge. It had zero judgment. And then I realized why: every single page in my wiki was the same type of knowledge. One Type vs Four LLM Wiki 1.0 stores declarative knowledge — facts, definitions, summaries. Things that answer "What is this?" But think about what makes a human expert different from a textbook: A great programming mentor doesn't just know what Promises are. They know why you teach callback → Promise → async/await in that exact order — and never the reverse. That's not a fact. It's a reasoning path. A master astrologer doesn't just know what each star represents. They know why you check 命宮 first, then 三方四正, when to prioritize 格局, when a palace is a consequence rather than a cause. That's not a fact either. It's a decision sequence. And here's the kicker: even knowing the reasoning path isn't enough. We annotated Anderson's (1972) Socratic tutoring dialogues — full 41-turn and 30-turn conversations, labeling every decision point. Knowing the 23 Socratic rules (the reasoning path) is one thing. Reading a complete dialogue — watching the expert set a trap, wait 15 seconds in silence, break their own rules when the student gets frustrated — is something else entirely. Knowing the recipe ≠ having watched the chef cook. And there's still one more type. Student says: "I have no

Eight kids, eight chairs, one rule: explaining FIFA's best-thirds draw to my 8-year-old Rahul Devaskar Rahul Devaskar Rahul Devaskar Follow Jun 27 Eight kids, eight chairs, one rule: explaining FIFA's best-thirds draw to my 8-year-old # webdev # soccer # math # worldcup Add Comment 14 min read

This is part of my build-in-public series where I document everything honestly — the problems I face,the observations I make,and what I'm trying to build. There's a moment that happens in almost every class. The teacher finishes explaining something. Looks around the room. And asks: Everyone clear? And the entire class says "yes." Including me. Even when I understood absolutely nothing. The Loop Nobody Talks About Here's what actually happens — at least for me and I suspect for a lot of you reading this: Teacher is explaining a concept. I'm trying to follow. Somewhere in the middle, I lose the thread. Maybe the explanation was too fast. Maybe the concept needed something I didn't know yet. Maybe I just zoned out for 10 seconds and missed the part that made everything else make sense. Now I have two choices: Option A: Raise my hand. Ask the question. Risk looking like I wasn't paying attention or worse ask something that makes me look stupid in front of everyone. Option B: Stay quiet. Nod. Say "yes" when the teacher asks. And hope it makes sense later. I always pick Option B. And then "later" comes — sitting alone at home, textbook open, trying to study for a test and I have no idea where to even begin. The concept is still missing. The gap is still there. But now there's no teacher, no classroom, no one to ask. So I either text a friend (who's also confused), scroll YouTube for 40 minutes looking for the right explanation, or just… close the book and tell myself I'll figure it out tomorrow. I never figure it out tomorrow. This Isn't Just a "Me" Problem I'm an engineering student in Pakistan. Maths, Physics, Chemistry — subjects where one missing concept breaks everything that comes after it. And I genuinely believe most of my classmates feel exactly the same way. We just don't say it out loud. Because saying "I don't understand" in a classroom full of people takes a kind of courage that most of us don't have. So we all nod together. And we all go home confused toget

The Agent Skills open standard today, and the 2026 research on agents that write their own skills. TL;DR: In late 2025, "Agent Skills" became a thing — a dead-simple way to teach an AI agent a task: a folder with a SKILL.md file (some instructions in Markdown). It's already an open standard. The wild part is what's coming next: agents that write their own skills. I built a demo where an agent solves a task the hard way once, saves a real SKILL.md , and then reuses it — cutting its total effort almost in half. ~130 lines, no API key. First, what's a "skill"? If you've used Claude Code or similar tools lately, you've probably seen SKILL.md files. The idea is refreshingly low-tech. A "skill" is just a folder with a Markdown file that says how to do something : --- name : csv-to-markdown description : Turn comma-separated text into a Markdown table. Use when the input looks like CSV and the user wants a table. --- # CSV to Markdown ## Instructions Split the text into rows on newlines and columns on commas. Make the first row the header, add a `---` divider row, then format every row as `| a | b | c |`. That's it. No SDK, no config. Anthropic introduced this in October 2025 and then published it as an open standard ( agentskills.io ) in December 2025, so the same skill folder now works across ~30+ different agent tools (Claude Code, Cursor, Copilot, and more). The full rules are short ( agentskills.io/specification ): the only required fields are name (1–64 chars, lowercase-with-hyphens, and it must match the folder name) and description (≤1024 chars, saying what it does and when to use it ). Everything else — license , metadata , compatibility , allowed-tools — is optional. That's the whole spec. The SKILL.md files my demo writes follow it to the letter, so they'd load unmodified in any compatible CLI. The clever trick: progressive disclosure Here's the smart part. If you just dumped 50 skills' worth of instructions into the agent's context, you'd fill it up and leave n

Active inference: curiosity emerges for free from minimizing surprise — 48% vs 100% on a foraging task. TL;DR: Most AI agents chase rewards — they pick whatever action scores the most points. I tried a different, brain-inspired goal: avoid surprises . Something neat happened — the agent became curious without being told to. It goes looking for information before acting, and that takes it from 48% to 100% on a simple task. ~100 lines. Two different ways to make decisions Most AI agents are "reward chasers." Give them points for doing well, and they'll pick whatever action they expect to score highest. Simple and effective. There's another idea from brain science: instead of chasing points, try to avoid being surprised — act so the world matches what you expected. It sounds almost too simple, but it leads to a surprising bonus: when you're trying not to be surprised, going and finding out what you don't know becomes valuable all by itself. In other words, curiosity isn't something you have to bolt on. It comes for free. This is called active inference , and in 2026 it jumped from neuroscience into AI as a serious approach ( here's a 2026 paper ). Here's the smallest demo that makes it click. The 10-second version The task: a reward is hidden behind either the LEFT door or the RIGHT door (50/50). There's also a hint you can check that tells you which door — if you bother to look. ❌ Reward-chaser ✅ Curious agent What it cares about getting the reward, right now getting the reward + not being unsure What it does guesses a door checks the hint first, then opens the right door Success (400 tries) 48% 100% Nobody told the second agent "go check the hint." It did it on its own, because being unsure bothered it. How it works Before acting, the agent scores each option on two things: Does this get me closer to the reward? Does this make me less unsure about what's going on? value_of_checking_the_hint = how_unsure_am_i # high when it's a total coin-flip value_of_just_guessing =

Theory of Mind and the Sally-Anne false-belief test, in ~60 lines of Python. TL;DR: There's a famous test that kids pass around age 4. It checks whether you understand that other people can believe things that aren't true. I built two AI agents: one that only knows "what's actually happening" (fails, like a toddler) and one that keeps track of what each person believes (passes). It's ~110 lines, and it's the foundation for agents that can actually work together . The test Sally puts her marble in the basket , then leaves the room. While she's gone, Anne moves the marble to the box . Sally comes back. Where will she look for her marble? If you said basket , nice — you just used something called "theory of mind." Sally never saw the marble move, so in her head it's still in the basket. What's actually true (it's in the box) and what Sally believes (it's in the basket) are two different things, and you kept them separate without even thinking about it. A 3-year-old says "box" — they can't yet separate what they know from what Sally knows. A 4-year-old says "basket." It's one of the most famous tests in child psychology, and in 2026 it's become a real test for AI agents too. The 10-second version ❌ Agent with no "theory of mind" ✅ Agent that models other minds What it tracks only what's actually true what each person believes, separately Where will Sally look? "box" "basket" Result FAIL (only knows reality) PASS How it works (the whole trick) The only difference between the two agents is one rule: a person's belief only updates when that person is actually in the room to see it happen. def someone_moves_the_marble ( new_place , who_is_watching ): for person in who_is_watching : # only people in the room beliefs [ person ] = new_place # update THEIR mental picture So when Anne moves the marble while Sally is out, only Anne's mental picture updates. Sally's is frozen at "basket." Ask the simple agent and it just reports reality ("box"). Ask the smarter agent and it answer

A sleep-like phase that consolidates noisy daily experience into durable memory — 75% vs 100% recall. TL;DR: There's a wave of 2026 research giving AI a "sleep" phase — time spent not answering questions, just tidying up what it learned that day. I built a 90-line demo of the idea. The agent that "sleeps" remembers 100% of what it learned. The exact same agent without sleep remembers only 75% and gets confused by bad info. Runs on a laptop. The memory problem every AI app hits If you've built anything with an LLM, you know the pain: the model only "remembers" what's in its current context window. Once the conversation gets long enough, the oldest stuff scrolls off the top and is just... gone. Forgotten. The usual fix is "make the context window bigger." But that's like fixing a messy desk by buying a bigger desk. It's expensive, and the model still gets worse as you cram more in (a real, measured effect — more text in the window can actually lower accuracy). Your brain doesn't work this way. You don't remember every sentence anyone said today. While you sleep, your brain replays the day, keeps the important bits as long-term memory, and dumps the rest. That's how you remember "I like coffee" without remembering every single cup. A couple of 2026 papers ask the obvious question: Do Language Models Need Sleep? Their answer: giving an AI a quiet "offline" phase to consolidate memories makes it remember better. So I built the simplest version that shows why. The 10-second version ❌ Agent with no sleep ✅ Agent that sleeps How it remembers keeps only the last N messages saves a tidy summary every night After 30 noisy days 75% recall 100% recall Tricked by bad info? yes no — it goes with what it saw most often Same experiences, same noise, same memory test. The only difference is whether the agent sleeps. How it works Each "day," the agent hears facts like Alice → drinks → coffee . To make it realistic, about 1 in 5 facts is wrong (people misremember, logs have errors). Th

A tiny Darwin Gödel Machine that edits itself and keeps only changes that verifiably score higher. TL;DR: I built a small program that improves itself . It looks at the tasks it's failing, edits its own code to fix them, and keeps a change only if the change actually makes it score better on a test. It goes from passing 1 of 8 tasks to 8 of 8 — and nobody wrote those fixes but the program itself. It runs on a laptop in under a second. No fancy hardware, no API key. The old dream: software that improves itself Normally, software only gets better when we make it better. You write code, you find a bug, you fix it, you ship again. The program never improves on its own. People have wanted "software that improves itself" for decades. The classic version (called a "Gödel Machine") had one rule that made it impossible to build: before the program could change a line of its own code, it had to mathematically prove the change would help. Proving that about real code is basically impossible, so the idea never worked. In 2025, researchers found a way around it with the Darwin Gödel Machine . They dropped the "prove it first" rule and replaced it with something every engineer already trusts: Try the change. Run the tests. If the score went up, keep it. If not, throw it away. That's it. It's basically how we all work — make an edit, run the test suite, keep what passes. The twist is that the program is the one making the edits. In the real paper, this let an AI coding assistant improve its own tooling and jump from solving 20% to 50% of a hard benchmark of real GitHub issues. I wanted to actually see this happen, so I built the tiniest version I could. The 10-second version Start After improving itself What it can do only uppercase learned 6 more skills on its own Test score 🔴 1 / 8 🟢 8 / 8 Who wrote the fixes? — the program did Start: ███░░░░░░░░░░░░░░░░░░░░░ 1/8 (only knows: uppercase) +reverse ██████░░░░░░░░░░░░ 2/8 +dedup_csv █████████░░░░░░░░░ 3/8 +sum_csv ████████████░░░░░░

Repo link and results - https://github.com/Abhinand20/MathFormer Task: Given a factorized expression like (7-3*z)*(-5*z-9), predict the expanded form -> 15*z\*2-8\*z-63 Key takeaway: A tiny (4M param) seq2seq model trained with no math knowledge reaches ~98.6% accuracy on symbolic math tasks, suggesting it learns structural token transformations rather than any notion of operators or variables. Scaling this up could help explain why LLMs appear to “reason” mathematically, when they may actually be performing large-scale structured pattern completion. How does RL change this paradigm given the inherent architecture is still based on attention? submitted by /u/AlphaCode1 [link] [留言]

Hey guys, I was playing around with Nanotron recently and got super frustrated by how many heavy, hardware-specific dependencies it imports at the module level ( flash-attn , triton, functorch , etc.). If you try to run it on older or budget GPUs like a T4 or V100, it just crashes on import. So I wrote Picotron ( https://github.com/Syntropy-AI-Labs/picotron ) to solve this. It's a clean-room rewrite that gets rid of all mandatory GPU-specific dependencies. It runs on pretty much any GPU that supports PyTorch (defaults to FP16 on older cards under compute capability 8.0, and BF16 on newer ones). It falls back to standard PyTorch SDPA by default, but still hooks into FlashAttention-2 at runtime if it detects you have it installed. I used an AI assistant to write a lot of the boilerplate/code modules, but I've got it working locally and just trained a tiny 2M model on FineWeb-Edu. Also added configs for: • GQA / MLA (Multi-head Latent Attention) • QK-Norm & logit soft-capping (Gemma 2 style) • Parallel FFN/Attn runs • ZeRO-1 wrapping on DDP Roadmap is pretty short right now: MoE prep (routing capacity factors and load balancing loss) Making dataset prep easier than streaming manually Check it out if you've been fighting with CUDA dependency hell: https://github.com/Syntropy-AI-Labs/picotron submitted by /u/Capital_Savings_9942 [link] [留言]

Hey everyone, I'd like to share my project along with a short explanation of the process and why it came about in the first place. To start off, I'm not exactly the best at cryptography/steganography, in my case it's always been something that sat in the background, as one of the sub-fields needed for another (main) field I'm actually interested in. For this project I tried to look up as much information as possible about what's currently considered best practice (I mainly relied on NIST for this), what implications exist, and what potential "attacks" exist against this way of hiding information, but I honestly can't say whether I covered everything, which is why I wanted to share this project here, mainly for the sake of learning. I'd be grateful for any feedback on what I could have done better / what I might have missed, etc. Right now, I consider this project closed at this point and will most likely not update it further, although I'd like to apply all the feedback to my own knowledge going forward. For over a month I did a lot of research into using ML models as a carrier for hiding data. I needed this as one of the stages for my main project. That's how I ended up on the topic of hiding information in model weights. Initially I assumed a simple method of directly writing data into randomly selected weights. I quickly concluded, though, that this would be absurdly trivial to detect, and potentially also to read. Next came the idea of using something like a deterministic coordinate map describing where to read the data from (location-id + position-id). The program wouldn't modify all the bits needed to write the message instead, it would write separate bits representing already-existing values (pointing to specific locations in the model) from which the existing 0s and 1s would need to be read. In practice, only parties A and B would know how to derive these positions. This way, someone unaware of the algorithm would only see what looks like noise of varying va

Habit stacking is one of my goals this year. I've been so focused on improving my software...