标签：#r

Local LLM Deployment, Agent Handbook, & LLM Cost Reduction: Applied AI Workflows Today's Highlights This week's highlights cover practical guides for running state-of-the-art LLMs locally and building AI agents, alongside an innovative technique to significantly cut LLM API costs for code processing. These resources focus on actionable insights and frameworks for real-world AI application development. Jamesob's guide to running SOTA LLMs locally (Hacker News) Source: https://github.com/jamesob/local-llm This GitHub repository provides a comprehensive, hands-on guide for setting up and running state-of-the-art Large Language Models (LLMs) on local hardware. It meticulously covers the necessary tooling, dependencies, and configuration steps required to get various open-source LLMs operational without relying on cloud APIs. The guide emphasizes practical considerations for local inference, including hardware requirements, model quantization techniques, and performance optimization for different architectures, directly addressing production deployment patterns. It serves as an invaluable resource for developers and researchers looking to experiment with LLMs, develop applications offline, or reduce costs associated with cloud-based inference by leveraging local compute. The guide offers concrete details and actionable steps, making it an essential resource for anyone aiming to implement LLMs in a controlled, private, or cost-effective environment. Comment: This guide is fantastic for anyone wanting to get serious about local LLM development. It covers the nitty-gritty details of setting up your environment and getting models like Llama-3 running efficiently on consumer hardware, which is crucial for privacy and cost savings. 60% Fable cost cut by converting code to images and having the model OCR it (Hacker News) Source: https://github.com/teamchong/pxpipe The pxpipe project introduces an innovative technique to drastically reduce API costs when processing code with Lar

One quiet handwritten page a day. No feed, no AI. Discussion | Link

I build and run one of the tools on this list (AGenO — full disclosure below), and I use every other tool here regularly. This is what "free" actually gets you on each one, including the catches. The AI tool landscape has a dirty secret: almost nothing labeled "free" is free. Most tools give you a taste — ten messages, three images, one song — and then the paywall lands. So instead of another list of forty tools nobody has tried, here are nine that give you real value at $0, organized by what you're trying to make, with the actual limits spelled out. Quick comparison Tool Best for What's actually free The catch ChatGPT General chat & writing ~10 msgs/5h on the flagship model Silently switches you to a weaker model after the limit Claude Long documents, nuanced writing 10–25 msgs/5h, varies with demand Limits shrink when servers are busy Gemini Image generation & editing Generous with a Google account Best features drift to the paid tier Perplexity Research with citations Unlimited basic searches Pro searches are capped Suno AI music ~10 songs/day No commercial use on free; failed generations can eat credits Leonardo AI Stylized art & game assets Daily token allowance Confusing token system; images are public on free Character.AI Roleplay & AI characters Unlimited chat Heavy filters; your chats train their models AGenO All of it in one place Images, songs with vocals, chat, characters, stories, coding problems — daily free allowance One-person project — busy hours can mean a short queue Canva Magic tools Quick social graphics 50 text-to-image uses Design-tool add-on, not a real generator Chat and writing ChatGPT is still the default for a reason — the free tier includes the flagship model and it's good at nearly everything. The catch nobody tells you about: after roughly ten messages in five hours, it quietly downgrades you to a mini model without making it obvious. If your answers suddenly get dumber mid-conversation, that's why. Claude writes the most natural prose

Google Trends has no official API, and most wrapper libraries rot within months. But the Trends site itself runs on a keyless JSON API that anyone can call, and it serves the exact numbers you see in the UI. Here is the full recipe, including one gotcha where Google quietly labels your session a scraper. The two step flow Trends works in two steps. First you call explore , which returns a list of widgets, one per chart on the page, each with a signed token: GET https://trends.google.com/trends/api/explore ?hl=en-US&tz=0 &req={"comparisonItem":[{"keyword":"web scraping","geo":"US","time":"today 12-m"}],"category":0,"property":""} Then you call a widget data endpoint with that widget's request and token : GET https://trends.google.com/trends/api/widgetdata/multiline?hl=en-US&tz=0&req=<widget.request>&token=<widget.token> The widget kinds map to endpoints: TIMESERIES uses multiline , GEO_MAP uses comparedgeo , and both RELATED_QUERIES and RELATED_TOPICS use relatedsearches . The cookie trick Call explore cold and you get a 429. The API wants a NID cookie, and here is the counterintuitive part: you get it by requesting the public explore page first, and that page may itself respond 429 while still setting the cookie you need. const res = await fetch ( ' https://trends.google.com/trends/explore?geo=US&q=test ' ); // res.status may be 429. The Set-Cookie header is still there. const cookies = res . headers . getSetCookie (); Grab the cookie from the 429 response, retry explore , and everything works. Strip the anti JSON prefix Every Trends response starts with a junk line like )]}' to break naive JSON.parse calls. Drop everything up to the first newline: const body = await res . text (); const data = JSON . parse ( body . slice ( body . indexOf ( ' \n ' ) + 1 )); The scraper flag Here is the part I have not seen documented. Look inside the widget request object that explore returns to a keyless session: "userConfig" : { "userType" : "USER_TYPE_SCRAPER" } Google knows. And

It happened during a quiet, solemn moment in a community prayer hall. I was sitting in the third row, reflecting, when suddenly, a high-pitched ringtone shattered the silence. It wasn't my phone, but the ripple effect of embarrassment was immediate. Everyone looked around, shifting uncomfortably. That collective tension is something we have all felt—a moment of human error that technology should have intercepted. I looked at my own device, feeling the familiar anxiety of whether I had remembered to flip the physical silent switch. It was then that I decided to stop relying on my own memory. We live in an era of hyper-connected devices, yet the most basic context-awareness—knowing where we are and how our phone should behave—remains manual. I found myself constantly toggling between 'Normal' and 'Silent' modes at the library, the office, and the gym. If I forgot, I was the person disrupting a meeting. If I remembered to mute it, I inevitably forgot to unmute it, missing urgent calls from family for hours. The existing solutions were either too bloated, requiring invasive cloud permissions, or they simply failed to trigger reliably when the screen was off. I needed a solution that was local, predictable, and battery-conscious. Building Muffle started with the realization that I had to master the GeofencingClient API without draining the user's battery. The primary challenge wasn't just triggering an event; it was doing so while the device was in a deep sleep state. I initially experimented with a standard LocationManager approach, polling GPS coordinates at set intervals. That was a disaster. It kept the radio active, pinged the GPS satellites constantly, and decimated the battery life in under four hours. It was an immediate non-starter for a production-ready application. I pivoted to the Geofencing API provided by Google Play Services, which leverages the fused location provider. This is significantly more efficient because the system handles the batching and hardwa

Hi everyone, I want to share textparser, a high-performance, lightweight text parsing and AST generation library written in pure C. 💡 The Core Idea & Why It's DifferentTraditional parser generators (like Flex/Bison) come with a steep learning curve and rigid code generation. On the other end, hand-writing recursive descent parsers or state machines becomes an unmaintainable mess as your language grows.textparser bridges this gap using a hybrid JSON + Python + C workflow:You define your tokens, syntax patterns, and color styles in a clean, human-readable JSON grammar file. A lightweight Python compiler tool processes the JSON, runs optimizations, and emits a dense, static C header array.The C runtime engine loads these pre-compiled arrays instantly. It processes raw text strings into an abstract token tree (textparser_token_item) using a highly optimized regular expression engine (crpe2).By offloading the grammar overhead and heavy state-machine parsing logic to the Python build step, the actual runtime C library stays incredibly lean, memory-efficient, and fast. 🚀 Features At A Glance30+ Languages Out-of-the-Box: Includes ready-to-use JSON grammars for C, C++, Rust, Python, JavaScript, HTML, SQL, and dozens more.Rich Token Metadata: Every parsed token tracks exact code coordinates, structural flags, and custom syntax styling options.Zero Bloat: Ideal for terminal text editors, syntax highlighters, custom linters, and lightweight static analysis tools where bringing in a massive compiler front-end is overkill.

The rise of AI has brought an avalanche of new terms and slang. Here is a glossary with definitions of some of the most important words and phrases you might encounter.

Hi everyone, I built H64LM, a research project to better understand modern LLMs by implementing one from scratch in PyTorch. Instead of relying on high-level training frameworks, I implemented the core components myself attention, MoE routing, normalization, and the training loop. Features 249M-parameter Transformer Grouped Query Attention (GQA) Sparse Mixture-of-Experts (8 experts, Top-2 routing) with 3 auxiliary routing losses SwiGLU, RoPE, RMSNorm Sliding-window attention Mixed-precision training, gradient accumulation Custom training loop (no Trainer abstractions) Checkpointing and resume support The included checkpoint was trained on a subset of WikiText-103 to validate the pipeline end-to-end, not to be a strong model it's visibly overfit past epoch 10 (best val PPL ~40.5). Known limitations are documented in the README, including batch-size-1-only generation and no true DDP (falls back to DataParallel). GitHub: https://github.com/Haiderkhan64/H64LM Feedback on the implementation or architecture is very welcome. submitted by /u/Loose_Literature6090 [link] [留言]

There's a phase almost every developer gets stuck in. You're consuming tutorials, bookmarking articles, finishing courses, and buying books you'll read "eventually." You're learning constantly — but you're not producing anything. You're just... absorbing. That's the learning vacuum. And if you've been there, you know how easy it is to confuse staying busy with making progress. At some point, the shift has to happen. You stop being a sponge and start being a signal. Here's how I started making that turn. Start a Daily or Weekly Code Journal You don't need a blog, a brand, or an audience for this. Just a file. A note. Anything. Write down what you built, what broke, and what you figured out. Even one sentence counts. I like to write a quick sentence and how many hours, just like if you were filling in an invoice for contract work. The act of putting it into words forces you to actually process what you learned instead of letting it blur into the background noise of your brain. Over time, those entries start to look like a roadmap — and you realize you've come further than you thought. Code Something You Actually Want to Build Pick something dumb. Pick something fun. A browser game, a weird UI experiment, a tool that solves exactly one tiny problem in your life. I signed up for DEV Challenges , Summer Bug Challenge and upcoming Weekend Challenge to get my ball rolling. The best projects I've ever worked on had no real-world utility. They were just interesting to me. And that interest kept me showing up even when things got hard. A tutorial can't give you that. Only a project you actually care about can. Find Your People Whether it's here or a Discord server, a local meetup, a dev community on Farcaster or Lens, or just a forum thread you keep coming back to — find somewhere to show up regularly. Lurking is fine at first. But eventually, drop a comment. Answer a question you know the answer to. Share something you built. Community is where isolated learning becomes shar

Every business that has ever typed a web address into a browser owes a small debt to a company most people have never heard of. On March 15, 1985, a computer maker called Symbolics registered Symbolics.com and, in doing so, became the first ever holder of a .com domain name. More than forty years later that address is still registered and still resolves - making it the oldest .com domain on the internet. Who was Symbolics? Symbolics Inc. was a Massachusetts company that built specialized computers called Lisp machines - workstations designed from the silicon up to run the Lisp programming language, then the darling of artificial intelligence research. These were serious, expensive machines aimed at labs and universities, and the company sat right at the cutting edge of 1980s computing. So it was fitting, if a little accidental, that they were first in line when commercial domains became available. The domain name system itself was brand new. DNS had only been introduced in 1983 to replace the unwieldy HOSTS.TXT file that every machine on the early internet had to keep in sync. The now-familiar top-level domains - .com , .org , .net , .edu , .gov - were defined in 1984. When registration opened, .com was meant for commercial entities, and Symbolics grabbed theirs before anyone else did. A slow start for the web's most valuable real estate What is striking today is how little demand there was. In the whole of 1985, only a handful of .com domains were registered - names like BBN, Think, and a few other technology companies trickled in over the following months. There was no gold rush, because there was no web yet. Tim Berners-Lee would not propose the World Wide Web until 1989, and the first website would not appear until 1991. A domain name in 1985 was a technical convenience for reaching a machine, not a brand or a piece of property. That makes Symbolics.com a kind of time capsule. It was registered before the web, before browsers, before e-commerce, and before anyon

Watch your AI coding agents build, live in 3D Discussion | Link