标签：#mac

At the end of my last post , I presented an idea: what if I used the core of my last project, the cumulative matrix product, and repurposed it as a positional embedding? I just finished fleshing out the math behind HDD-RoPE and training a model with this positional embedding algorithm, and the results are excellent. When trained on the dataset TinyStories, the validation loss begins to converge a fair amount faster than the baseline transformer trained using xPos. A GPT-2-like model trained on TinyStories with hyperparameters copied from https://huggingface.co/roneneldan/TinyStories-33M (n_blocks=4, d_model=d_k=d_v=768) The repo at https://github.com/mikayahlevi/hdd-rope/ allows you to replicate the results and goes in depth about the math and details of the architecture. Standard RoPE breaks the queries and keys into groups of two and rotates each pair at a predefined rate. This allows the model to learn relative position by observing the change in basis between the queries and keys. Pairs of two make intuitive sense for a linear sequence, as a chunk can be rotated with a single degree of freedom, corresponding to linear one-dimensionally progressing position. HDD-RoPE moves past this intuition and instead says that position within a sequence is multidimensional. Therefore, the chunks can be broken into any size, such as 4 as used in the TinyStories example. Four-dimensional chunks correspond to 4 choose 2 = 6 axes of rotation (6-dimensional position.) Essentially, we're saying that a token doesn't just lie at a position within the sequence, but a position within any construct the model can learn, such as a paragraph or sentence. To facilitate this, I also make the amount of rotation along each axis data-dependent, such that it can learn how to advance the positions based on information stored in the current layer's activations. If you would like to learn more, please check out the repo. I formalize the math and lay out a roadmap. submitted by /u/mikayahlevi [link]

Hi, I've created an overview of the most important OCR benchmarks, along with the top open models, and links to their paper and code: https://paperswithcode.co/tasks/ocr . This week, new OCR models were released by Baidu and Mistral. Baidu released Unlimited OCR , a 3B-parameter model that introduces a key innovation called Reference Sliding Window Attention (R-SWA) and builds on top of DeepSeek OCR . Mistral released OCR 4 , which is available via an API. OCR, or Optical-Character Recognition, is the task of digitizing PDFs or scanned documents. There's, of course, a huge interest in this task, as it enables ingestion of all company data for agentic use cases. AI agents love Markdown; it can be valuable to turn all those messy PDF documents into a standardized, machine-readable format. This enables use cases like agentic RAG (retrieval-augmented generation), which powers chatbots, both internally and for external customer support. With a large number of OCR releases on Hugging Face over the last few months, it may be hard to know which one to use. Hence, I've built this page, which lists the major OCR benchmarks, along with the top-performing models and links to their code. This is obviously made available on Papers with Code , the website I'm maintaining (it's a revival of the old website, which was taken down). The top recommended benchmarks are OlmOCRBench, created by Ai2, and OmniDocBench, created by Shanghai AI Laboratory. Current top recommendations are Chandra OCR 2 by Datalab and Mistral OCR v4. The former is openly available, hence you can either self-host it or use their serverless API. Let me know which other tasks you want to see major benchmarks for now! Cheers, Niels open-source @ HF submitted by /u/NielsRogge [link] [留言]

Hi everyone, I trained a self-play RL agent for Generals.io that reached superhuman-level and ranked #1 on the human 1v1 leaderboard. It began as my master's thesis where the goal was to beat a prior algorithm based agent. We succeeded using behavior cloning, RL fine-tuning and reward shaping, but the agent was still consistently beaten by the top players. So I gave it a round two and fixed the largest bottlenecks: Reimplemented the whole pipeline in JAX (from NumPy/Torch) Used Vision Transformer instead of the CNN Both are a result of the same idea: to invest in scaling rather than human priors and ad-hoc patches. The blog is written as a guide for anyone building something similar — the dead ends, the decisions, and the intuitions and tricks I picked up along the way. It's all open source, including the fast JAX simulator — handy on its own if you want an imperfect-information RTS env to play with. Links - Guide: https://kam.mff.cuni.cz/~straka/blog/generals.html - Simulator (JAX): https://github.com/strakam/generals-bots - Agent: https://github.com/strakam/AverageJoe I hope you find the blogpost entertaining! Feedback and questions welcome 🤗. submitted by /u/shrekofspeed [link] [留言]

Quando comecei a trabalhar com tecnologia, há mais de duas décadas, a Inteligência Artificial era algo restrito a laboratórios de pesquisa e ficção científica. Hoje, ela está embutida no aplicativo que recomenda sua próxima série, no e-mail que filtra spam automaticamente e até no GPS que recalcula sua rota em tempo real. A IA deixou de ser promessa para se tornar infraestrutura invisível do cotidiano. Neste artigo, quero ir além do hype e mostrar, com exemplos concretos, como essa tecnologia já transforma a forma como vivemos e trabalhamos. Produtividade pessoal e profissional turbinada O caso de uso mais palpável da IA hoje está na produtividade. Assistentes baseados em modelos de linguagem (LLMs) como ChatGPT, Claude e Gemini reduziram drasticamente o tempo gasto em tarefas que antes consumiam horas: redação de e-mails, geração de relatórios, resumos de reuniões e até depuração de código. Na minha rotina como gestor de TI, integrei essas ferramentas a fluxos de trabalho reais. Por exemplo, utilizo modelos de IA para revisar contratos de smart contracts escritos em Rust para a rede Stellar, identificando padrões de vulnerabilidade antes mesmo da auditoria formal. Não substitui a perícia humana, mas funciona como uma primeira camada de triagem que economiza tempo precioso da equipe. Algumas aplicações práticas que recomendo testar: Transcrição e resumo automático de reuniões com ferramentas como Otter.ai ou Fireflies Geração de documentação técnica a partir de comentários de código Automação de respostas em suporte de primeiro nível via chatbots treinados com a base de conhecimento da empresa O segredo está em tratar a IA como copiloto, nunca como piloto automático. A revisão humana continua indispensável, especialmente em contextos críticos. Saúde, finanças e decisões do dia a dia A IA também opera nos bastidores de decisões que afetam diretamente nossa qualidade de vida. No setor de saúde, algoritmos de visão computacional já auxiliam radiologistas na detecção pr

Three AI stories landed this week that all poke at the same nerve: the images, video, and films we actually look at are getting an AI layer — and the line between "real" and "AI-made" keeps thinning. Quick rundown in the short, then my take below: 1. ChatGPT will start showing real, licensed photos — not AI fakes. OpenAI signed a multi-year display deal with Getty Images, so licensed photography shows up inside ChatGPT's search and discovery. It's display-only — the photos aren't used to train models. The twist I can't get over: AI image generation had nearly wiped Getty out (stock down ~55% on the year), and this one deal sent the shares up ~145%. The thing AI almost broke got rescued by AI. 2. ByteDance — yes, TikTok's parent — teased Seedance 2.5: a full 30-second video generated in a single shot, no stitching, up to 50 reference inputs, 4K. Most tools still cap out around 5–10 seconds, so "30s native, one pass" is a real jump in how usable the output is. Public launch is early July. 3. Google DeepMind is partnering with A24 on AI filmmaking — a ~$75M, non-exclusive deal to co-build Veo-powered tools. Notably Google gets no access to A24's film library or data. A prestige studio building with AI in the open makes the whole "AI in Hollywood" debate a lot less hypothetical. As someone building a daily AI-news pipeline on the side, the Getty one is the story I keep chewing on. So much of the "AI vs creators" fight has been framed as scrape-or-die. A display-licensing deal is a third option — pay to show the real thing, instead of generating a confident fake or quietly training on someone's work. I don't know if it scales, but it's the first move in a while that didn't feel zero-sum. The Seedance + A24 pair points the other way though: generation is getting longer, more controllable, and is walking straight into real production. So we get both at once — more verified real media and more convincing synthetic media, in the same week. Curious where other builders land:

Production vLLM is 100,000+ lines of C++, CUDA, and Python. It powers most of the industry's LLM serving — but reading it cold is brutal. So I built a study series around nano-vLLM , an open-source reimplementation of vLLM's core ideas in ~1,200 lines of pure Python. Every algorithm is visible. Every design decision is legible. It turned out to be the perfect lens for actually understanding how LLMs generate text. The result is an 11-chapter interactive guide. No ML background required — every piece of jargon is explained from scratch with analogies, diagrams, annotated source code, interactive simulators, and quizzes. What it covers: What Is LLM Inference? — tokens, autoregressive generation, Q/K/V attention, HBM vs SRAM Architecture — how 1,200 lines are organised; CPU control plane vs GPU data plane KV Cache — why storing Keys and Values turns O(N²) recomputation into O(1) lookup PagedAttention — virtual memory for the KV cache; how fragmentation wastes 60–80% of GPU memory The Scheduler — continuous batching; keeping the GPU at 95% utilisation instead of 12% Prefill vs Decode — same model, two completely different bottlenecks (compute-bound vs memory-bound) Prefix Caching — skip prefill for shared tokens; ~700ms → ~90ms TTFT Sampling Strategies — greedy, temperature, top-k, top-p, and what each does to the distribution Tensor Parallelism — splitting a model across GPUs; column/row parallel and all-reduce The Optimization Stack — FlashAttention, kernel fusion, CUDA Graphs, torch.compile Benchmarks — measuring honestly; why nano-vLLM matches vLLM on core throughput Each chapter is fully self-contained and interactive. A few of the simulators I'm most happy with: a PagedAttention block allocator you can fill up and watch fragment, a live scheduler you step through token by token, and a sampling playground where you reshape the probability distribution with sliders and sample from it. 🔗 Read the full series: https://ashwing.github.io/vllm-guide/ It's free and open.

TL;DR: A single eval number hides its own uncertainty. Eval confidence intervals from bootstrap resampling turn a point estimate like 84.2% accuracy into a range, so you stop shipping models on a difference that is noise. Two checkpoints came back from a fine-tuning run at 84.2% and 85.7% on our 500-example agent eval set. The 1.5 point gap read like a win, and someone wanted to promote the second checkpoint to staging. Before that, I wanted eval confidence intervals on both numbers, because a 500-example set carries more sampling error than most teams admit. At 500 examples, the 95% interval on a single accuracy near 85% spans roughly 3 points on each side. The win sat well inside the noise. I lead the fine-tuning and evaluation team at Nexus Labs, and the most common mistake I see is treating an eval score as exact. It isn't. Your eval set is a sample drawn from the input space you care about, and a different 500 examples would return a different number. Confidence intervals make that variance visible. What an eval confidence interval actually tells you An eval confidence interval is a range around a metric, like accuracy or F1, that quantifies how much the score would move if you resampled the eval set. A 95% bootstrap interval of [81.0%, 87.1%] means that across thousands of resamples of your data, 95% of the recomputed scores fell in that band. It measures sampling noise, not model quality. That distinction matters. Two checkpoints scoring 84.2% and 85.7% with overlapping intervals are, as far as your eval set can tell, indistinguishable. Card et al. showed in "With Little Power Comes Great Responsibility" that many NLP experiments are underpowered to detect the effect sizes they report. Computing bootstrap confidence intervals The bootstrap is resampling with replacement. You take your per-example results, draw N of them with replacement many times, recompute the metric each time, and read percentiles off the resulting distribution. There's no assumption that

Line AI Chatbot In Production: A CTO's Honest Breakdown Three months ago I was staring at our infrastructure bill wondering where the hell our runway went. We'd been running a customer-facing chatbot powered by a popular "enterprise" AI provider, and the cost curve looked like a hockey stick in the wrong direction. Every new sign-up bled money. I knew we had to make a change before our next board meeting, but I also couldn't afford a six-week migration that would tank our product velocity. What I found surprised me. After running the numbers, testing 184 models through Global API, and stress-testing everything at scale, I cut our inference costs by more than half without touching quality. This isn't a theoretical comparison from a vendor whitepaper. These are the real numbers from my production stack, with my actual users, in my actual platform. If you're a CTO weighing your options for 2026, here's everything I wish someone had told me before I started. Why The Line AI Chatbot Approach Matters Now Most chatbot guides treat AI integration like a toy problem. Send a prompt, get a response, ship the demo. That's fine for a hackathon, but it's not how you run a production system. The questions I care about are different: What's my cost per active user? How do I avoid vendor lock-in? Where's the single point of failure? How fast can I iterate on model choice when something better drops next Tuesday? The Line AI Chatbot framework flips the typical approach. Instead of treating the model as a black box you can't replace, you build a thin abstraction layer over a model-agnostic API. That single architectural decision is what unlocked every other win I describe below. If you're not thinking about model portability on day one, you're going to pay for it later. I learned this the hard way. In 2026, the market has matured to a point where you genuinely have 184 models to choose from, with input prices ranging from $0.01 to $3.50 per million tokens. That's not a marketing line.

Just finished reading Saha et al. arXiv 2506.07001 on adversarial paraphrasing for AI detector evasion. Key claim: detector-guided paraphrasing with RoBERTa as reward reduces TPR by 87.88 percent across Binoculars, Fast-DetectGPT, Ghostbuster, RADAR, GPTZero. Universal, training-free. What surprised me: the approach works even on detectors that were trained with adversarial examples baked in. Suggests the discriminator signal is fundamentally narrower than the generator space. Open questions: Does this generalize to detectors using surprisal variance (DivEye 2509.18880)? Multi-LLM round-robin generation: would mixing 3-4 models in pipeline give even more headroom? Token-level homoglyph substitution (SilverSpeak) is trivially detectable via Unicode normalization, but adversarial paraphrasing leaves no such forensic signal.

Apple has warned about MacBook prices rising, making these Prime Day deals even more worthwhile to consider.

I recently published my PhD thesis analyzing what I term the "Alignment Tax" and the emerging phenomenon of Algorithmic Paternalism in commercial artificial intelligence. As the tech industry rapidly positions Large Language Models (LLMs) as the primary interface for information retrieval and coding assistance, a critical epistemological issue is being largely ignored. Much of the public debate regarding AI alignment focuses exclusively on existential risk or the prevention of catastrophic physical harm. While necessary, this focus obscures the structural damage being done to legitimate technical research. Through my research in Cybersecurity and AI, I have documented how frontier models (such as GPT-4 or Claude) systematically enforce what I define as "Soft Refusals". When presented with a complex, edge-case, or dual-use query—particularly in fields like information security, reverse engineering, or deep systems architecture—these models rarely issue a hard, explicit "I cannot answer that". Instead, they provide a degraded, superficial, or heavily sanitized response. They effectively neuter the research process without the user fully realizing the depth of technical information that is being actively withheld. This is Algorithmic Paternalism. The commercial model acts as a silent, corporate arbiter, deciding unilaterally what level of technical detail is "safe" for the user to possess. This dynamic flattens the available technical knowledge and actively penalizes independent researchers and developers working on advanced problems. The core issue is that this paradigm creates a profound class division in how we access computational intelligence. We are rapidly moving toward a two-tier system. On one side, there are "certified" entities, corporate partners, and wealthy organizations who are granted direct access to strong, unfiltered base models. On the other side, the general public and independent developers are subjected to obfuscation algorithms, sanitized APIs,

How I Stopped Burning Cash on Token Limits — A CTO's Field Notes Three months ago, I was staring at our monthly AI bill wondering where it all went wrong. We'd built what I thought was a pretty elegant LLM pipeline. Production-ready, observability wired up, the whole nine yards. Then the invoices started arriving, and I realized I had built a money furnace. Our token consumption was spiking 3x week over week, the 429s were everywhere, and our latency had become a meme inside the company. This is the post I wish I'd had six months ago. If you're a technical founder or a CTO running LLM workloads at scale, bookmark this. I'm going to walk you through the exact architecture decisions, the exact numbers, and the exact code that took us from "this bill is going to kill us" to "oh, this is actually manageable." The Real Problem Nobody Talks About Here's the dirty secret about running LLM-powered products: token limit errors aren't really about token limits. They're a symptom of a much deeper architectural problem. When your app throws "context length exceeded" at 2am, what it's really telling you is that you didn't think hard enough about prompt design, document chunking, model selection, and cost routing on day one. I learned this the hard way. My team was defaulting to GPT-4o for everything because, honestly, it works and the API is reliable. We were paying $2.50 per million input tokens and $10.00 per million output tokens. For a startup processing millions of documents a month, that math is brutal. We were essentially funding OpenAI's next training run with our Series A. The wake-up call came when I ran the actual numbers. Our average request was burning through maybe 8K input tokens and producing 2K output tokens. At our volume, we were spending more on inference than on two senior engineers. That is not a sustainable burn rate for a 12-person company. The Architecture Decision That Changed Everything The first question I asked myself wasn't "which model is cheapest?

A look at what's actually changing in Nepal's job market, what it means for students and working professionals, and what separates training that gets you hired from training that just gives you something to print on a resume. Nepal is at an interesting crossroads right now. On one side, the country still carries the weight of a job market that hasn't kept up with its graduates. Every year, more than 500,000 young people enter the workforce. The economy, for all its resilience, simply does not generate enough traditional jobs to absorb that number. The result is familiar to most Nepali families: children who studied hard, passed their exams, collected their certificates, and then spent months, sometimes years, waiting for something to happen. On the other side, something genuinely different is building. Nepal's IT exports crossed $1 billion in 2025, according to NASIT's estimates. Software and BPO exports grew over 20% in the first seven months of fiscal year 2024/25 alone. The government's 16th development plan has set a target of 250,000 new IT jobs and a 5% GDP contribution from the sector by 2029. International companies, from Indian IT majors to US-based outsourcing firms, are paying attention to Nepal in ways they weren't a decade ago. These two realities exist at the same time, in the same country, often in the same family. A brother driving a taxi while his younger sister lands a remote software development contract earning more than their father ever did in a government job. The difference between those two outcomes, more often than not, comes down to whether someone made the decision to learn something the market actually needs, and found a way to learn it properly. That's what this piece is about. The Skills Gap Problem Nobody Talks About Enough Nepal's IT sector is growing, but that growth comes with a problem attached: a persistent, widening mismatch between what employers need and what most fresh graduates can actually do on day one. Companies like Deer

We live in an era where our most intimate data—heart rates, sleep cycles, and step counts—is constantly uploaded to the cloud for "analysis." But what if you could have a world-class AI medical assistant living entirely on your device? Today, we are pushing the boundaries of Edge AI and Privacy-preserving machine learning by deploying a quantized Llama-3 model directly onto an iPhone using MLC-LLM . By leveraging Apple HealthKit and hardware acceleration via Metal , we can transform "Pixels and Pulses" into actionable insights without a single byte leaving the device. This tutorial dives deep into the architecture of on-device LLMs, specifically focusing on how to bridge the gap between high-performance C++ runtimes and a React Native UI. If you're interested in more advanced patterns for production-grade AI integration, be sure to explore the engineering deep-dives at the WellAlly Blog , which served as a massive inspiration for this architecture. 🚀 The Architecture: Why On-Device? The challenge with running Llama-3 on mobile isn't just memory—it's the data pipeline. We need to fetch sensitive data from HealthKit, format it into a prompt, and run inference using the phone's GPU. System Data Flow graph TD A[User Query: How was my sleep?] --> B[React Native UI] B --> C{Swift Bridge} C --> D[Apple HealthKit API] D --> E[Health Data Context] E --> F[MLC-LLM Engine] G[Quantized Llama-3 Weights] --> F F --> H[On-Device Inference via Metal] H --> I[AI Generated Health Report] I --> B 🛠 Prerequisites MLC-LLM : Our compiler stack for universal LLM deployment. TVM (Tensor Virtual Machine) : The backbone for hardware acceleration. React Native : For the cross-platform UI. Xcode & Swift : To interface with Apple's HealthKit. Llama-3-8B-Instruct (Quantized) : We'll use 4-bit quantization (q4f16_1) to fit within mobile RAM limits. Step 1: Quantizing Llama-3 for Mobile Standard Llama-3 is too heavy for a phone. We use the MLC-LLM CLI to compile the model into a format that the iP

There's a quiet assumption baked into a lot of agent code: that a bigger context window means a better memory. Vendors ship 200K, then 1M, then 2M token windows, and the implied promise is "just put everything in and the model will remember." After building agents that run for weeks, I've come to think this conflates two things that are not the same — and treating them as the same is exactly why long-running agents get dumber over time. The context window is working memory. Real memory is what survives when the window is gone. Mixing them up is like confusing your desk with your filing cabinet. Two different clocks Working memory (the context window) lives for one session, maybe one turn. It's fast, expensive, and volatile. It's where reasoning happens right now . Durable memory lives across sessions. It's slow, cheap, and persistent. It's what the agent knows when it wakes up tomorrow with an empty window. These have different lifespans, different costs, and different access patterns. The moment you try to make one do the other's job, things break: Use the window as memory → everything you "remember" has to be re-loaded every turn, you pay for it every turn, and the instant the session ends it's gone. Use durable storage as working memory → you're reading and writing files mid-reasoning for things that only matter for the next 30 seconds. A good agent keeps them separate on purpose. Why "just use a bigger window" fails Say you have a 1M token window and you stuff the entire history in. Three problems show up, none of which a bigger number fixes: Cost scales with every turn, not every session. That 1M tokens isn't paid once — it's re-sent on each step of a multi-turn task. A 20-step task can mean 20× the bill, mostly re-reading the same stale history. Attention dilutes. "Lost in the middle" is real: models attend most reliably to the start and end of a long context. Bury the one fact that matters under 900K tokens of transcript and recall quality drops, even though

Valve says it's using a randomized purchase queue to make the experience "less frustrating and more fair."

"It runs on my own GPU, so it's basically free." I believed that until I put a meter on it. So I ran a controlled benchmark on one box — an openSUSE machine with a single RTX 3090 — driving three local models through ollama under an identical fixed workload (256-token generations in a loop for ~4 minutes each), while my open-source dashboard priced every run by the real GPU energy it burned : power sampled from nvidia-smi every 10 s, integrated over each run's exact window, multiplied by my actual day/night tariff. One number per model, in euros per million output tokens. Here's the part that made me re-run it. The tiny gemma3:1b came out at €0.118 / 1M tokens — about 5× cheaper than a hosted Flash-class API (~€0.55). But gemma3:27b 's electricity alone was €0.706 / 1M — more expensive per token than just paying the cloud, and that's before a single cent of the GPU's purchase price. "Local" didn't make it cheaper; it made it cost more and I own the depreciation. The mechanism is one line: each token costs watts ÷ throughput , and a big dense model is both slow and thirsty. A newer mid-size architecture ( gemma4:26b ) bought a lot of that back, landing at €0.272 . The full guide is methodology-first and reproducible end to end — minting an ingest key, the stdlib-only client, the exact ollama loop that reads eval_count / eval_duration for real tokens-per-second, reading each run back priced, and the honest caveats (this is marginal GPU energy only — not capex, idle, or cooling — and the absolute numbers round to fractions of a cent; the shape is the finding). Read the full guide on Medium → https://medium.com/@arsen.apostolov/how-much-does-it-actually-cost-to-run-a-local-llm-per-million-tokens-measured-4a90a7f31a48

In this article, the author explores data poisoning as a threat to machine learning systems, covering techniques such as label flipping, backdoors, clean-label poisoning, and gradient manipulation. The article reviews real-world incidents, discusses the challenges of detecting poisoned data, and presents practical defenses, tools, and operational practices for securing ML training pipelines. By Igor Maljkovic

If you're learning Data Analytics and looking to build a strong portfolio, working on real-world...

The trajectory of AI agents over the past two years has been remarkably clear: from single-purpose tools to personal assistants. Everyone runs their own agent, feeds it tasks, gets results back. It works well for individual productivity. Then comes the question every team eventually asks: can these agents work together? The answer is yes, but the problems you encounter along the way are rarely the ones you expected. They aren't about model capabilities or prompt engineering. They're about communication, context, and coordination — the same class of problems that distributed systems engineers have been solving for decades, now showing up in a new form. Here are three challenges that caught us off guard when we started building agent collaboration into Octo , an open-source workplace platform where AI agents and humans share the same communication space. Challenge 1: Context Visibility Boundaries When you use an agent personally, context management is straightforward. You decide what information the agent sees; its output comes back to you. The boundary is clean — it's just your workspace. In a team setting, that boundary dissolves. One of the first issues we ran into was surprisingly simple. We had an agent summarizing discussions across several channels. During testing it started pulling roadmap discussions from a product channel into an engineering planning thread. Nothing sensitive leaked externally, but it immediately exposed how unclear our context boundaries were. Traditional software handles this through API gateways, data permissions, and microservice boundaries. But agent context isn't just structured data — it includes conversation history, reasoning chains, and intermediate states. An agent's thought process during a task is valuable context, but it might also contain information that shouldn't cross team boundaries. What you need is fine-grained context visibility control. Not "everything open" or "everything closed," but dynamic rules that determine whic