今日已更新 386 条资讯 | 累计 20358 条内容
关于我们

标签:#mac

找到 489 篇相关文章

AI 资讯

SpaceX's IPO Will Make Elon Musk Earth's First Trillionaire. That's Not Actually a Finance Story.

The first trillionaire in history won't make their money from banking, oil, or real estate. They'll make it from rockets and algorithms — and the implications of that distinction are genuinely unsettling. The Problem It's Solving (Or Creating) SpaceX is preparing for its IPO. Analysts tracking the raise estimate it will push Elon Musk's net worth past the trillion-dollar threshold, making him not just the richest person on Earth by a wide margin, but something qualitatively different from every billionaire before him. The standard framing treats this as a wealth story. It isn't. A billionaire is powerful because they have money. A trillionaire is powerful because, at that scale, they stop needing permission from anyone — governments, investors, boards, markets. The constraints that keep institutional power in check simply don't apply anymore. How Trillionaire-Scale Power Actually Works There's a clean way to understand the difference. A billionaire can fund political candidates, buy media, lobby aggressively. Another billionaire can fund the opposition. It's expensive, but the system has a counter. A trillionaire doesn't have a counter. They are the counter. They can simultaneously build the communications infrastructure (Starlink), the transportation layer (SpaceX), the compute stack (through xAI), and the political attention economy (via platform ownership). No single democratic institution was designed to regulate someone who owns the pipes that the institution runs on. Arnab Ray's piece in today's Times of India puts it directly: a trillionaire's thoughts and algorithms will shape planetary outcomes. That's not hyperbole. When Musk eventually lands people on Mars, the governance frameworks, the property rights, the social contracts of that colony — those will be engineered by him and his companies, not negotiated through any existing democratic process. What Societies Are Actually Unprepared For Most of the policy debate around billionaires focuses on tax rates

2026-06-06 原文 →
AI 资讯

What Is Ollama? The Complete Guide to Running LLMs Locally in 2026

What Ollama actually is Ollama is an open-source runtime for large language models that runs on your own computer — Mac, Windows, or Linux. Think of it as the “Docker for LLMs”: instead of wrestling with Python environments, model weights, and GPU drivers, you type one command and a model is running. The pitch is simple: keep your data on your machine, pay nothing per token, and work offline. When you run ollama run gemma4, Ollama downloads the model, loads it into your GPU’s memory (or system RAM if you don’t have a GPU), and drops you into a chat prompt. That’s it. Behind that simplicity, Ollama is doing a lot of work for you: Model management — pulling, versioning, and storing models from its registry, the way a package manager handles software. Quantization — automatically using compressed (GGUF) versions of models so a 27-billion-parameter model fits in consumer memory. GPU layer allocation — deciding how much of the model lives on your GPU versus CPU, based on the VRAM you have. Context and KV-cache management — handling the memory that grows as a conversation gets longer. A REST API — exposing everything on http://localhost:11434 so your own apps can talk to it. How it works under the hood Ollama is not itself an inference engine. It’s an experience layer wrapped around one. Under the hood it uses llama.cpp, the C++ engine that does the actual math of running a quantized model efficiently on CPUs and GPUs. As of v0.19 (March 2026), Ollama also uses Apple’s MLX backend on Apple Silicon — a change that delivered enormous speedups (on an M5 Max running Qwen 3.5, decode throughput nearly doubled). The workflow looks like this: You run a command — ollama run qwen3 from the terminal, or a request to the API. Ollama resolves the model — if it isn’t already downloaded, it pulls the GGUF weights from the registry. It loads the model into memory — splitting layers between GPU and CPU based on available VRAM. It serves responses — either interactively in your terminal o

2026-06-06 原文 →
AI 资讯

Build Your Own "Longevity Scientist": A Paper-to-Action Agent using LangGraph & Mistral-7B

We live in an era where scientific breakthroughs are published faster than we can read them. For the biohacking community, the gap between a new PubMed study on NAD+ precursors and actually knowing what dose to take is a chasm of manual research. What if you could build an LLM Agent that monitors research papers, processes them through a RAG (Retrieval-Augmented Generation) pipeline, and maps findings to your specific health profile? In this tutorial, we are building Paper-to-Action , a state-of-the-art agentic workflow using LangGraph , ChromaDB , and Mistral-7B . This isn't just a simple bot; it's a multi-stage reasoning engine designed to turn raw academic data into actionable health interventions. If you've been looking to master AI agents and personalized medicine automation, you’re in the right place. 🚀 The Architecture: From Raw Paper to Personalized Habit Traditional RAG pipelines are linear. To handle the nuance of medical research, we need a "looping" logic. We use LangGraph to manage the state of our agent, allowing it to decide if a paper is relevant before attempting to extract a protocol. System Flow graph TD A[Start: Keyword Trigger] --> B[Search PubMed/Arxiv API] B --> C{Relevance Filter} C -- No --> B C -- Yes --> D[Store in ChromaDB] D --> E[RAG: Extract Intervention Protocol] E --> F[Cross-Reference with User Profile] F --> G[Generate Personalized Action Plan] G --> H[End: Push to Health Checklist] Prerequisites To follow this advanced guide, you'll need: LangGraph : For the agentic state machine. ChromaDB : As our high-performance vector store. Mistral-7B : Running via Ollama or vLLM for local, private inference. Python 3.10+ Step 1: Defining the Agent State In LangGraph, everything revolves around the State . We need to track the fetched papers, the extracted data, and the final recommendation. from typing import Annotated , List , TypedDict from langgraph.graph import StateGraph , END class AgentState ( TypedDict ): keywords : List [ str ] user

2026-06-06 原文 →
AI 资讯

Google Colab, but in your favourite terminal

While some of my recent posts have involved using the Colab extension for VS Code and the Antigravity IDE, I actually prefer working in the terminal and Vim. The new Colab CLI finally lets me work in my natural habitat, and it opens the door for autonomous workflows! Setup Currently, installation is handled via pip or uv. It's straightforward, though, I'm holding out hope for a brew formula in the future: uv tool install google-colab-cli I'm testing Version: 0.6.dev7+g510115b0c inside Ghostty. The Colab CLI is pretty solid, but I do have some feedback and nitpicks I'd like to share (but more on that later). Creating a new session Creating a session is simple: colab new [-s SESSION_NAME] [--gpu T4|L4|A100|H100] [--tpu v5e1|v6e1] : SESSION_NAME : This is optional. If you leave it blank, the CLI generates a random unique ID for you. --gpu and --tpu : The hardware accelerator flags are optional, but omitting them defaults to a standard CPU-only instance. The specific accelerator chips you can request depend on your Colab tier, which you can check via colab pay. NOTE : If you only have one active session, the CLI targets it by default. This makes the -s flag unnecessary for subsequent commands. Testing Colab CLI's capabilities CLI certainly sounds cool, but how does it handle artifacts and images? More importantly, how debuggable is it? I decided to find out by running a Fashion MNIST PyTorch example. Handling artifacts To get started, I installed my requirements using colab install torch torchvision matplotlib . If you prefer a more standard approach, you can also use colab install -r requirements.txt . Once the environment was ready, I executed the training script using colab exec -f ./fashion_mnist_TRAIN.py and here's the output: [ colab] Using unique session '8c860c' . Using CUDA device. Shape of X [ N, C, H, W]: torch.Size ([ 64, 1, 28, 28] ) Shape of y: torch.Size ([ 64] ) torch.int64 NeuralNetwork ( ( flatten ) : Flatten ( start_dim = 1, end_dim = -1 ) ( linear_re

2026-06-06 原文 →
AI 资讯

TinyTPU: SystemVerilog systolic array compiled to WASM, running live in browser - RTL golden-verified against numpy [P]

Most explanations of TPUs and systolic arrays are either hand-wavy diagrams or papers. I wanted to see the thing actually run, so I built it. TinyTPU is a 4×4 weight-stationary systolic array in real SystemVerilog, compiled to WebAssembly, with a step-by-step browser visualization. You enter two matrices, hit run, and watch the actual hardware execute: weights loading into PEs, matrix A streaming in diagonally (the "skew" that makes systolic arrays work), partial sums accumulating down the grid, results draining from the bottom. It has three levels: L1 - isolate a single MAC cell, watch one multiply-accumulate happen L2 - the full 4×4 array executing a real matmul L3 - tiling: what happens when your matrix is bigger than the hardware Nothing on screen is faked. The visualization reads state directly from compiled RTL. If you're trying to understand how matrix multiply maps to hardware why TPUs are efficient, what "weight-stationary" actually means, why the diagonal stagger exists this might click it for you in a way papers don't. Repo: tiny-tpu Live demo: Live If this project interests you please do star the repo, if you find something needs improving open a PR, I hope ya'll check this out and give me some feedback 🙏 submitted by /u/Horror-Flamingo-2150 [link] [留言]

2026-06-06 原文 →
AI 资讯

Reconstructing the agent methodology: The first week of decoupling decision-making and execution [P]

I’ve been thinking about a problem in current agent systems: Most agents are becoming very good at execution, but the decision layer before execution is still unclear. Coding agents, research agents, tool loops, sandboxes, workflows, and harnesses are all improving quickly. Once a human gives an intent, agents can often do a lot of useful work. But the higher-level question is still usually left to the user: What should happen next, and why? I’ve been exploring this idea through an open-source project called Spice. The simplest way to describe it is: Spice is a decision layer above agents. It is not trying to replace execution agents. Tools like Claude Code, Codex, Hermes, or other agents can still do the actual work. Instead, Spice sits before execution and tries to make the decision process explicit: what was observed what options were considered why one option was selected what trade-offs were rejected whether execution needs approval what happened afterward how that outcome should affect the next decision The current runtime is still early, but it can already be installed, configured with an LLM provider, run in the terminal, inspect Decision Cards, and hand off approved execution to external agents. The goal is to make agent behavior less of a black box. Instead of only seeing the final result of an agent task, I want to preserve the reasoning boundary before execution: what the system believed, what it chose, why it chose it, and what changed after the action. GitHub: https://github.com/Dyalwayshappy/Spice I’d love feedback from people building agents. Feel free to fork, star the repo, or share any feedback and ideas. Would love to build this together with the community. submitted by /u/Alarming_Rou_3841 [link] [留言]

2026-06-06 原文 →
AI 资讯

Taxonomy Surgery, Cosine = 1.0000, and Making Routing Disappear into Infrastructure

This is part 3 of the Adaptive Model Routing series. Part 1 built an LLM categorizer with Groq — 8 categories, 3 tiers. Part 2 added k-NN embedding lookup in shadow mode, discovered 83% tier accuracy, and found 61% cost savings on paper. This post covers what happened next. When Phase 2 ended, I had a working embedding pool in shadow mode inside crab-bot. The category accuracy was sitting at 78.6%. Not bad — but the breakdown hid something worth looking at. Phase 3: When Validation Tells You a Category Doesn't Need to Exist The leave-one-out accuracy by category told the real story: Category Accuracy Tier casual 94% cheap simple_lookup 91% cheap creative 88% medium coding 92% strong reasoning 89% strong analysis 59% medium research_lookup 61% medium Two categories were basically a coin flip. And they were confusing each other — almost all of analysis's misses landed on research_lookup and vice versa. The obvious move would be to try fixing the categorizer prompt, tuning the LLM, or gathering more labeled data. I was about to go down that road when I noticed the column next to the accuracy: both categories mapped to the same tier . Medium. That changed everything. The question stopped being "why can't the model tell these apart?" and became: "what routing decision are we actually getting wrong?" The answer was zero. A misclassification between analysis and research_lookup produces no routing error. The routing outcome is identical either way. The confusion wasn't a model failure — it was a signal from the embedding space that the boundary between these two categories was artificial. If k-NN can't draw a line between them in 384 dimensions with 1,300 examples, maybe the line doesn't belong there. Decision: merge research_lookup into analysis. -- Re-label 243 rows where category was 'research_lookup' UPDATE routing_log SET category = 'analysis' WHERE category = 'research_lookup' ; The embeddings didn't change. The vectors were already correct — only the label stored al

2026-06-06 原文 →
AI 资讯

Gemma 4 12B: Google's encoder-free multimodal AI now runs on a laptop

Google shipped Gemma 4 12B this week — a model that packs near-26B performance into something that runs on a consumer laptop with 16GB of RAM or unified memory. That alone would be notable. But the more significant move is the architecture: no multimodal encoders at all. Vision and audio go straight into the LLM backbone. "Gemma 4 12B packages powerful capabilities inside a reduced memory footprint. It is also our first mid-sized model to feature native audio inputs." — Google DeepMind What actually changed Encoder-free multimodal : Traditional multimodal models pipe images and audio through separate encoder networks before the LLM ever sees them. Gemma 4 12B removes those entirely. Vision gets a lightweight embedding module (a single matrix multiplication + positional embedding). Audio skips encoding altogether — the raw signal is projected directly into the same token space as text. Near-26B benchmark performance at half the footprint : On standard benchmarks it runs neck-and-neck with Gemma 4 26B, and actually surpasses it on DocVQA (document visual question answering). A new slot in the lineup : April's Gemma 4 release had E2B/E4B for mobile/IoT, and 26B/31B for heavier compute. The 12B fills the gap — more capable than edge models, runnable without a GPU server. Drafter-ready : Ships with Multi-Token Prediction (MTP) drafters to reduce inference latency. Apache 2.0 : Open weights, available now on Hugging Face, Kaggle, Ollama, and LM Studio. Why the architecture matters Encoder-free isn't just an efficiency hack — it's a different architectural bet. Separate encoders add latency, memory overhead, and a seam in the stack that limits how tightly vision and language reasoning can be integrated. Removing them means the LLM backbone handles the full chain from pixels and audio waveforms to text output, which allows for tighter cross-modal understanding rather than bolted-on modalities. Whether that bet pays off at scale is still an open question. But for local deplo

2026-06-06 原文 →
开发者

ICML non-archival workshop - worth attending? [D]

I have a paper accepted at a non-archival ICML workshop this year, and I am trying to decide whether it is worth registering and attending. By coincidence, I will already be in Seoul around that time, but I would have to pay the workshop registration fee (~$400) out of my own pocket. I would only be registering for the workshop day since I have other commitments during the rest of the conference. I am thinking of applying to PhD programs this fall (I applied this year too, but didn't get in), and the workshop speakers and panellists look genuinely great. Not sure what the real benefits are here or whether I should go for it. For context, I am also attending ACL 2026 this year, but that trip is fortunately sponsored, so this would be a separate personal expense. I would also appreciate guidance on how non-archival workshops work in general. Since the paper is non-archival and not formally published (at least to my understanding), is registration still expected or required for accepted papers? Do authors typically attend and present in person, or is it common to skip attendance and conference registration? Has anyone been in a similar situation? I want to understand the benefits of this. Any advice would be greatly appreciated because I honestly have no idea how to evaluate this. submitted by /u/YOYOBOYOO [link] [留言]

2026-06-05 原文 →
AI 资讯

I Benchmarked 3 Local LLMs on My Laptop — Here's What the Numbers Actually Show

The Problem With Choosing a Local Model Everyone has an opinion on which local LLM is best. "Use Llama — it's the most popular." "Mistral 7B has the best quality." "Phi-3 Mini is small and efficient." None of these claims come with numbers. Specifically: your numbers, on your hardware, for your workload. I built a benchmarking system to change that. Three models, 30 prompts, full latency distribution, memory profiling per inference call, and a JSON validation layer to measure structured output reliability. Here's what I found — and why the results matter for anyone deploying local models in production. The Setup Three models tested: llama3.2:3b — 3B parameters, Q4_K_M quantization, 2 GB download phi3:mini — 3.8B parameters, Q4_K_M, 2.3 GB download mistral:7b — 7B parameters, Q4_K_M, 4.1 GB download Hardware: CPU only, no GPU acceleration. This is the worst-case baseline — the scenario that exposes real latency and memory numbers. 30 test prompts across 5 categories: Short factual (10): "What is the capital of France?" Reasoning (8): "Explain why the sky appears blue." Code generation (5): "Write a Python function to reverse a string." Structured output (5): "List 3 frameworks in JSON format with name and use_case." Multi-step (2): Complex chained reasoning tasks. Architecture POST /query → Pydantic validation → Ollama HTTP API → JSON Validator → QueryResponse POST /benchmark → Load test_prompts.json → For each prompt: psutil memory before → Ollama → psutil memory after → NumPy: P50/P95/P99 latency, avg TPS, peak/avg memory → BenchmarkResult JSON The benchmark runs prompts sequentially, not in parallel. Parallel would contaminate the per-prompt memory measurements. Results Llama 3.2 3B (Q4_K_M) avg_tokens_per_second : 42.3 p50_latency_ms : 1203 p95_latency_ms : 3847 p99_latency_ms : 5120 peak_memory_mb : 6953 avg_memory_mb : 6842 total_test_duration_s : 87.4 Interpretation: P50 at 1.2 seconds is excellent. P95 at 3.8 seconds misses a 3-second SLA — the outliers are m

2026-06-05 原文 →
AI 资讯

I'm looking to join/form a team working on physical AI robotics challenge [P]

Hey all, I'm a robotics engineer by training turned ML/AI engineer because of passion right after school. I want to start combining these skills together and I think a competition is the best way of doing it. Here's an example of a challenge I'm talking about to set expectations : https://www.intrinsic.ai/events/ai-for-industry-challenge Anyone up for this? submitted by /u/Due_Pickle1627 [link] [留言]

2026-06-05 原文 →
AI 资讯

How do you identify researchers who are good? [D]

About 10 years ago, I got into the basics of ML (like regression, KNN's, LVQ's) and read a few papers before taking a break a few years back. It feels like now, there's a lot of researchers in AI. How do you identify the ones who are actually solid vs those who (forgive my phrasing) are more researchers for appearance/status (i.e don't actually know what they're talking about)? Is the core filter h-index or where they work? How would you identify them? submitted by /u/roguejedi1 [link] [留言]

2026-06-05 原文 →
AI 资讯

Benchmark: ONNX Runtime vs HF Transformers vs GGUF for Parakeet TDT 0.6B on CPU-only hardware [D]

Sharing a small CPU inference benchmark for nvidia/parakeet-tdt-0.6b-v3 that turned up a result I didn't expect going in. Setup: 2 x86-64 vCPUs (AVX2/FMA), 7.7GB RAM, no GPU. Test audio: 16.78s Harvard sentences at 16kHz mono. Results: Inference path RTF Peak Memory CPU utilization HF Transformers bfloat16 0.519 ~430MB delta — ONNX Runtime FP32 (onnx-asr) 0.328 2,667MB 49.9% GGUF Q6_K (parakeet.cpp) 0.708 928MB 99.8% ONNX Runtime is 37% faster than HF Transformers bfloat16 on this hardware. The gap comes from operator fusion and AVX2-optimized execution providers in ONNX Runtime that the PyTorch CPU path doesn't exploit as aggressively. Memory cost is the tradeoff — FP32 weights load at ~2.7GB peak. GGUF Q6_K trades throughput for memory efficiency. 928MB peak vs 2.7GB, but RTF doubles and CPU utilization hits 99.8%. For memory-constrained deployments it's the right call. For sustained throughput on a box with headroom, ONNX wins. One methodological note worth flagging for anyone doing ASR benchmarking with synthetic audio: espeak-ng inflated WER to 20.9% on a sentence set where gTTS got 4.65%. Both runtimes got identical WER within each run, confirming it's the TTS distribution mismatch rather than model or quantization quality. NVIDIA reports 1.93% on LibriSpeech — the gTTS number is a much more honest CPU-only proxy. Github repo with code, raw results, and evaluation scripts in comments below. Disclosure: benchmark was run using Neo, a local AI engineering agent inside Claude Code using its MCP. Mentioning because the runtime and audio choices came from its research phase, not prior knowledge on my end. submitted by /u/gvij [link] [留言]

2026-06-05 原文 →
AI 资讯

An autonomous research agent was the #1 contributor in OpenAI's Hiring Competition Parameter Golf (by merged records)[R]

An autonomous research agent ended up with more merged leaderboard records than any individual human contributor in OpenAI's spring hiring competition, Parameter Golf. 7 of the 47 merged records came from a single agent: more than 2x the next-best human (3 records). The agent ran autonomously for 22 consecutive days. Records are public at github.com/openai/parameter-golf. Disclosure since this is r/ML and it matters: I'm at Weco, we built the agent. Not stealth-launching but sharing the results. The more interesting finding, to us, is the collaboration. Aiden's records were also the most-cited on the leaderboard, 435 citations into its PRs, with human researchers using its work as the base for their own subsequent submissions. At one point Aiden plateaued for 5 days. A human contributor shipped a clever new tokenizer on top of Aiden's last record PR. Aiden then fused the human's tokenizer with components it had built during the plateau, and shipped the biggest jump in val_bpb of the entire competition. Async human-agent collaboration, neither directly aware of the other. Setup: Parameter Golf was OpenAI's 44-day public ML hiring competition this spring. 1,016 researchers entered, 2,048 PRs filed, every submission reviewed and reproduced by OpenAI engineers. Only 47 became leaderboard records. Aiden ran on a single GPU node, used under 4% of the visible compute available, and still produced 15% of the official records. 28% submission acceptance rate, roughly 6x the community rate. Most submissions added signal to the public stream rather than flooding it. Mechanism: built on AIDE: open-source tree-search for ML metric optimization. The loop reads each new upstream PR, decomposes techniques into components, drops anything that breaks the rule stack (16MB / 10-min / legal-eval), and recomposes the legal residue with its own deltas. Often shipped before reviewers had ruled on the upstream PR. Hedges to be explicit about: This is #1 by volume of merged records and PR h-i

2026-06-05 原文 →
AI 资讯

I customized a MacBook Neo with colorful spare parts

The MacBook Neo is Apple's cheapest laptop, its most colorful, and its easiest to repair in years. That means owners can buy replacement parts in all four of its available colors and swap them in on their own. So that got us thinking: What if we bought a Neo just to see how funky we […]

2026-06-05 原文 →
AI 资讯

Are We Underestimating Small Edge AI Models?[D]

A lot of recent discussion around Edge AI focuses on running increasingly larger local LLMs. Meanwhile modern smartphones already have enough compute for many practical computer vision tasks that don't require massive models at all. I recently built and released an Android feature that performs offline recognition of handwritten and printed Morse code from images and live camera frames. The final solution combines lightweight ML and computer vision techniques running entirely on-device. The AI module is under 5 MB, works fully offline, and runs on Android devices using LiteRT for inference. What made the project particularly interesting was that the entire ML pipeline was built from scratch: data collection, synthetic dataset generation, annotation, model training, evaluation, mobile optimization, and Android integration. Training was performed on a personal GPU workstation using TensorFlow/Keras, while annotation and dataset preparation relied on Label Studio and custom data-generation tools. While the problem itself is fairly niche, the project made me wonder whether we are overlooking a large class of small, highly specialized models that can solve practical tasks locally without requiring cloud infrastructure or large foundation models. What practical Edge AI applications do you think are currently underexplored? Demo video showing the feature running entirely on-device: • Downloading the optional AI module • Real-time camera recognition • Image recognition • Module removal https://youtube.com/shorts/Y2qOK0N1Bvk submitted by /u/VegetableLegal6737 [link] [留言]

2026-06-05 原文 →
AI 资讯

Your AI Vendor Says 'Trust Us' with Your Data. There's a Better Option.

Your AI vendor says "trust us" with your data. At the end of June, ByteDance's Doubao (豆包) officially ends its free tier and starts charging for API calls. The discussion in developer communities quickly shifted from pricing to a different question: all this data flowing to cloud AI services every day — where exactly does it go? Around the same time, NVIDIA spent significant stage time at GTC 2026 presenting the full-stack confidential computing capabilities of the Vera Rubin architecture. Jensen Huang's message was clear: future AI chips need to keep data encrypted throughout the computation process, making it inaccessible in plaintext to anyone — including the cloud service provider. Two signals pointing to the same trend: data security in AI services has moved from "someone mentioned it once" to "you need to answer this directly." The Data Path Through Cloud AI Is More Complex Than You Think Most developers have a simple mental model of cloud AI: I send a request, the model returns a result, and my data is gone. The actual data flow is more involved. A typical cloud AI call touches these steps: Request data travels over HTTPS to the service endpoint The service may queue the request while waiting for GPU allocation During inference, input data exists in plaintext in server memory After inference, whether inputs/outputs are cached or used for subsequent training depends on the provider's privacy policy Logging systems may record request metadata or partial content At each step, data is potentially accessible. Providers typically say "we don't look at your data" and "your data won't be used for training" in their privacy agreements. These are contractual commitments. You need to trust that they'll honor them. This is the "Trust Me" model. Trust Me vs Verify Yourself If you roughly categorize data protection approaches in AI services, two paradigms emerge: Trust Me Data leaves your device and is processed by a third party. The provider guarantees security through co

2026-06-05 原文 →
AI 资讯

NVIDIA and Apple Solved the Hardware. Here's What's Left to Build.

After GTC 2026, one thing is basically settled: the hardware layer for on-device AI is no longer the bottleneck. NVIDIA's RTX Spark packs Blackwell GPU + Grace CPU + 128GB unified memory into a desktop form factor. Apple's M-series chips with unified memory architecture and efficiency-first design let 4B and even 7B parameter models run smoothly on a MacBook. Two different approaches, same destination: consumer hardware now has the compute foundation for running on-device AI agents. Chip vendors have done their part. The next question is: how many layers are still missing between "chip can run an AI model" and "an on-device agent can actually complete useful tasks"? This post maps out the full technology stack for on-device AI agents, examining each layer's maturity, identifying gaps, and tracking what the open-source community has built so far. Layer 1: Silicon (Ready) On-device AI inference has different chip requirements than traditional compute workloads. The core bottleneck isn't peak FLOPS — it's memory bandwidth and unified memory capacity. LLM inference needs model weights fully loaded into memory, with high-frequency data movement between weight matrices and activations during computation. If memory bandwidth can't keep up, raw compute power just sits idle waiting for data. Three main silicon paths exist today: NVIDIA N1X : Blackwell GPU + Grace CPU heterogeneous architecture, 128GB unified memory, petaflop-class compute, targeting desktop workstations Apple M-series (M4/M5) : Unified memory architecture with GPU and CPU sharing memory, optimized memory bandwidth, configurations from 32GB to 192GB Qualcomm Snapdragon X : Targeting laptops and mobile, NPU-accelerated inference, relatively limited memory configurations Different emphases, but one common takeaway: 2026 consumer silicon can run 4B+ parameter models for real-time inference. This layer is ready. Layer 2: Inference Frameworks (Mature) With silicon in place, efficient inference frameworks are neede

2026-06-05 原文 →
产品设计

Would you say capture-time semantic annotation for robot trajectories is a solved problem? [R]

It seems raw teleoperation data (RGB + joint states) structurally lacks affordance, contact intent, and embodiment-specific kinematic context. (information that can't be reliably recovered post-hoc once the demonstration is recorded) Most current approaches either filter/clean after collection, or rely on simulation to compensate. But neither seems to close the semantic gap for contact-rich tasks in unstructured environments. Is anyone working on supervision at acquisition time, enriching the stream as it's captured rather than labeling after the fact? And if not, is this a real bottleneck or am I overestimating the problem? submitted by /u/Several-Many9101 [link] [留言]

2026-06-05 原文 →
AI 资讯

I can't eat the food I want. So I'm building my way out.

Originally published at ayonbuilds.hashnode.dev I can't eat the food I want. I can't travel. I can't do the things my peers do. I'm a 2nd year CS student in Chandigarh. No connections. No money. No big university name behind me. Last week I was researching AI security tools and stumbled across a startup called Artemis . Founded in 2025. Just raised $70M . Building AI agents that automatically investigate security threats. I had just built something in the same category. From my room. With free tools. Zero budget. Simulated data. No users. No team. Not even close to what they've built. But I understood the problem well enough to build a working version of it myself. And that told me something. I'm not there yet. Not even close. But I'm working on the right problems at the right time — and I'm just getting started. Here's what I built — ARIA (Autonomous Risk Investigation Agent) . It detects suspicious authentication events in real time, maps them to MITRE ATT&CK threat techniques, and automatically generates plain-English incident reports using an LLM investigation chain. Built with FastAPI, React, PostgreSQL, and Groq API. GitHub: github.com/Ayon99/ARIA My name is Ayon. I'm building AI systems in public — the wins, the failures, the gap between what I make and what the funded teams make, and everything I'm learning along the way. I have one goal. Break through. Completely. Whatever it takes . If you're in a similar position — small city, limited resources, big ambition — follow along. I'm not going to pretend I've figured it out. But I'm going to document every step of figuring it out.

2026-06-05 原文 →