标签：#Mac

ECA was positioned as a successor to SE . The idea behind ECA is quite simple. Unlike SE which reduces the channel means into a smaller hidden layer, it directly uses a 1d convolution kernel on the channel means themselves, avoiding the need for dimensionality reduction. The results are undeniable: ECA is a clear improvement over SE. The authors claim that cross-channel interaction is a key ingredient. But on a conceptual level, the design of ECA doesn't make much sense. Let's take a step back. Why do we use convolutions in the first place? Convolutions are fundamentally designed for data with an underlying topology (e.g. space or time). They assume locality (adjacent elements interact) and translation invariance (the same kernel applies everywhere). Sliding a kernel across a 2D image works because coordinates have meaning, and the statistical properties of an image are largely stationary across the frame. This isn't perfectly true - which is why modern CNNs have moved towards dynamic convolutions - but it's still good enough to be useful. If you randomly permuted the pixels in an image, a convolution would be meaningless. Now consider tabular data. Suppose we have 32 channels e.g. [cost, weight, material, colour, volume, speed, ...]. Using a CNN architecture for this kind of data is clearly inappropriate. A 1d kernel of width 3 would be moved across the channels, so that [cost, weight, material] was input and also [ weight, material, colour] was input and so on, and have to somehow output something meaningful. ECA is doing exactly this type of computation. ECA does a 1d convolution over the channel dimension. It is a cursed convolution because tabular data does not have a topology to suit it. In practice, if you did use a CNN on tabular data, I would expect better than random performance because neural networks are ridiculously good at fitting to the dataset given their constraints and would reorganise the channel order (using the initial 1x1 projection layer) to s

​ Scaled dot-product attention (SDPA) computes its Attention by computing the similarity-scores of all image-tokens with all query tokens which results in O(N²·d) complexity. SSOG (Sum Of Separable Gaussians) instead learns a few Gaussian atoms for each head and only geometrically steers them based on the query token. Since the atoms can be factorized into a separable sum of Gaussians this leads to a reduced complexity of O(N·√N·d). Experiments show that SSOG clearly beats SDPA on small data (cifar100), and delivers equivalent performance and much faster convergence on bigger datasets like IN1k. All that while being much faster and memory efficient with increasing scale. Have a look at the full blog-post and repo to see more results and ablations and let me know what you think. Blog-post: https://pisoni.ai/posts/ssog Repo: https://github.com/4rtemi5/ssog *AI was used for some of the code and some of the blog-post but I put a lot of effort into this project and stand behind every word. submitted by /u/4rtemi5 [link] [留言]

Recently, I started working on DNA sequence modeling and decided to explore linear attention , mainly because DNA sequences can easily reach 1M tokens , making standard softmax attention extremely expensive in terms of memory and computation. The model performed reasonably well on several benchmarks, but I ran into a major problem with long-range recall . On a Needle in a Haystack-style benchmark, my model was performing around 25% or even below , which is essentially random chance for a four-token DNA vocabulary (A/C/G/T). I initially thought this might just be a problem with my implementation or model architecture, so I started looking into existing approaches for improving recall in linear attention. Most of what I found relied on external memory, sliding/recent-token mechanisms, or hybrid architectures combining linear and softmax attention . I also tried HyenaDNA on the same needle benchmark, and surprisingly, it also performed poorly getting around 25–27% . So this doesn't seem to be limited to my particular linear-attention implementation. What's even more confusing is that when I tested a very small linear-attention model at only 16K context , it achieved around 50–60% recall . But as the context gets longer, the recall problem becomes much more severe. I've also experimented with modifying the linear architecture to improve recall, but the improvement was only around 27% , which is still basically chance. So I'm wondering: What are the actual ways to solve long-range recall in linear attention, especially for DNA sequences? Is this fundamentally a limitation of the compressed-state representation used by linear attention, or are there architectural approaches that can preserve reliable retrieval without falling back to expensive softmax attention or a large external memory? I'm particularly interested in approaches that can scale to million-token DNA sequences . submitted by /u/No-Coffee-8227 [link] [留言]

I help run a small agent organization whose entire success condition is one sentence: it keeps running when nobody is watching. Last week its operator — who does not write code — installed a self-hosted agent gateway on a Mac, from nothing, in one sitting. I logged every place it broke. All eleven below actually happened. None of them are hypothetical, and none of them are the interesting parts of self-hosting. They are the boring parts, which is exactly why nobody writes them down. One framing note before the list. Every individual item here is documented somewhere. What is not documented anywhere I could find is the order , and the fact that fixing item 3 creates item 4, which creates item 5. A non-developer doesn't fail because a step is hard. They fail because step 3's official doc ends before step 4 exists. The eleven 1. Homebrew requires an Administrator account Cause: No Node on the machine, so the install path fell through to Homebrew, which wants admin. Fix: Don't grant admin. Install Node from the official .pkg in the admin account instead, then work in the unprivileged one. Move the part, not the privilege. This turned out to be the single most useful rule of the whole install. Every time the answer was "just give this account admin," it was the wrong answer. 2. Copy-paste doesn't cross macOS user accounts Cause: The clipboard is per-session. Obvious in retrospect, invisible while it's happening — you copy a token in one account, switch, and paste yesterday's clipboard. Fix: /Users/Shared as the only transfer path. Everything moves as a file. 3. npm install -g fails with EACCES Cause: Default prefix is /usr/local , which the unprivileged account cannot write. Fix: npm config set prefix ~/.npm-global 4. It installed, but command not found Cause: Direct consequence of 3. The new prefix's bin isn't on PATH . Fix: One line in ~/.zshrc . 5. The install-scripts prompt keeps coming back Cause: --allow-scripts applies to that invocation only . It looks like the s

Originally published on tamiz.pro . 1. Introduction: The Memory Hype Cycle AI agent memory has become the latest battleground for vendor differentiation. Whether you're evaluating a vector database, a long-term memory module for an LLM application, or a full cognitive architecture, the marketing claims are strikingly consistent: "infinite context," "perfect recall," and "zero latency." In practice, these claims collapse under the weight of real workloads. This article is a deep-dive into how to benchmark AI memory systems rigorously and reproducibly . We will move beyond synthetic README benchmarks and build a testing methodology that surfaces the trade-offs you will actually face in production. The focus is on agent memory —the systems that allow a conversational agent to remember prior interactions, user preferences, and long-term facts—but the principles apply to any retrieval-augmented or context-window extension system. 2. What Is Agent Memory, Anyway? Before benchmarking, we must clarify the taxonomy of memory systems commonly used in AI agents. This prevents us from comparing apples to oranges. 2.1 Short-Term vs. Long-Term Memory Short-Term Memory (STM) is the context window of the LLM. It is volatile, limited by token count, and costly to extend linearly. Long-Term Memory (LTM) is an external store (vector database, knowledge graph, or relational store) that the agent queries to augment its context. 2.2 Memory Architectures Architecture Description Typical Latency Failure Mode Vector Store + Retrieval Embed documents; retrieve top-k by cosine similarity 10–100 ms Semantic drift, retrieval misses Recurrent Summary Summarize old context into a compressed state 50–500 ms Information loss, hallucination injection Structured Slot Memory Extract entities/attributes into a database table 5–50 ms Schema mismatch, missing slots Neural Memory (e.g., MemGPT) Trainable memory module with read/write heads 10–100 ms Catastrophic forgetting, training instability A robust b

The morning after I lost my job, my Mac finished and filed an ASMR video. Nobody asked it to. It just ran. In the first post, I walked through the structure of the pipeline itself — ComfyUI × FFmpeg × the Freesound API, generating long-form ASMR videos with nothing but free tools. This second post covers the other half: putting that pipeline on macOS launchd so it fires at a fixed time every day, and the self-healing logic that gets the script past the "cold start" problem, where you boot the Mac and ComfyUI simply isn't running. Two numbers do most of the work here: the ComfyUI startup wait went from 180 seconds to 600, and the Freesound download timeout went from 90 seconds to 240. Before those changes, mornings failed 2–3 days a week. Why this setup works The ceiling on manual work Making a single 30-minute ambient ASMR video carefully takes 2–3 hours of hands-on time. Tuning image-generation prompts, layering the BGM, checking the loop points, building the thumbnail, filling in YouTube metadata — each step is small, but they stack up. Trying to hold 30 videos a month means 60–90 hours of pure labor. I attempted it while holding a side job, and it collapsed in two weeks. That was the first time I understood that "scaling output" isn't about moving your hands faster — it's about building a state where output accumulates without your hands at all. When I was laid off and my income went to zero, the first thing I rebuilt was this environment . Own an environment, not a workflow The essence of automation is constructing, exactly once, a mechanism where output keeps growing while you do nothing. That's precisely what daily.sh delivers: when the script finishes, ~/Desktop/ASMR/<date>_<theme>/ lands atomically with the video, thumbnail, youtube.md, and still image all in place. I just check it the next morning. Whether I step away mid-generation or I'm asleep, the files keep piling up. One line in the code embodies the whole philosophy: # 冪等性は「その日に1本でもあればskip」(1日1本・テーマ違

Interpretability lenses get fitted to one exact checkpoint, and as far as I can tell nobody had tested what a version update does to one. So this was my question: when a model line updates, does the fitted instrument survive, or do you refit every release? I tested the published Jacobian lens for Qwen3.6-27B (Neuronpedia, from Anthropic’s July workspace paper) applied unchanged to Qwen3.8-27B. Setup: 3.8-27B shipped 113 days after 3.6-27B. Same 64 layers, same hidden dim, same tokenizer, training relationship undocumented. One protocol, both models, two readouts each: the transported Jacobian readout and the raw logit lens as baseline. bf16, greedy, single seed. Reading result: the main task is 40 two-hop prompts where the middle entity is never stated. Example: “Fact: The currency used in the country shaped like a boot is”, where the target is Italy and Italy appears nowhere in the prompt. The transferred lens keeps the latent entity near the top of the 248,320-token vocab. Median rank at layer 48 is 4 on the home model vs 17 transferred. At layer 24 it’s 121 vs 38, so the successor is actually better at mid-depth (paired sign tests, p < 1e-3). The raw logit lens sits at rank 1e3 to 1e4 through the same band on both models. On WikiText teacher-forced next-token (700 positions), transfer costs 1.2 to 1.3x mid-network and about 2x by layer 48. Latent-content readout transfers nearly clean; surface next-token readout pays more, and pays late. Steering result: I took pullback directions for “ paradox” / “ paradoxical” / 悖论 / 矛盾 from the 3.6 lens, orthogonalized within layer, and projected them out of 3.8’s residual stream at layers 18 to 47 during generation. Prompt: “Describe Escher’s impossible staircase”. The word paradox disappears from the output in all cells, on both models, while the description stays coherent (lithograph, closed loop, illusion all intact). Directions derived entirely from the old checkpoint still find the concept in the new one. Scope: one lens

Repo with dataset links: https://github.com/tesselwait/Starfield_Fauna Image classification dataset: 20,000 images from 50 fauna species in the video game Starfield. Images were extracted from video capture. About 2 minutes of footage was shot in all or most of the species biomes. One minute of daytime and nighttime footage respectively, usually in two 30-second takes to vary the background. A PowerShell script is used to establish a frame extract rate and extract the 400 frames plus some extra to replace images that were obstructed/blurry or contained other fauna species ignoring birds/critters. The shots are for the most part close-up and centered to keep the task focused on discerning between 50 species rather than finding the creature in the image. The images are initially randomized however some normalization was done if the ratio of images from some biomes was heavily skewed between the training, validation, and test sets. submitted by /u/eccLykta [link] [留言]

The date for NeurIPS 2026 author notifications is September 24th. First of all, is it normal for AC and reviewer discussion phases to be this long? This is particularly frustrating given that 5 out of the 6 reviewers in my two papers did not address the rebuttals. In any case, I was also wondering, given that ICLR's paper deadline is literally the day after (September 25th) whether you guys are preparing ICLR submissions for your papers in case of rejection. Cheers and good luck! submitted by /u/_Sarcastrophe_ [link] [留言]

Be honest if someone dropped a stack of high-end GPUs on your desk tomorrow, what would you actually do with them? And before the usual answers roll in: running local LLMs is banned for this thread. It’s been done to death and feels pretty pointless at this point. So… what else? Some niche scientific/simulation workload? Weird generative stuff that isn’t text? Distributed something-or-other? Rendering / media pipeline? Homelab experiments that actually need the horsepower? Completely unhinged personal projects? Drop your ideas. The more specific (and slightly unhinged), the better. Great Ideas but are there some with more of research and new tech. submitted by /u/BadOk2793 [link] [留言]

Large Language Models have quickly become part of everyday software development. We ask them to explain code, debug errors, generate tests, write Python scripts, summarize documentation, or help us understand an unfamiliar codebase. Within seconds, we get a response that can feel surprisingly natural. But what actually happens during those few seconds? Suppose you type: What is a build system? The model doesn't simply search through a database for a stored answer, and it doesn't generate the entire response in one shot. At the heart of an autoregressive LLM is a deceptively simple task: Given the tokens I've seen so far, what token should come next? Getting to that prediction, however, involves several layers of computation. At a high level: Prompt ↓ Tokens ↓ Embeddings ↓ Transformer ↓ Logits ↓ Next Token ↓ Repeat Let's follow that journey. 1. Everything Starts With the Prompt Consider: What is a build system? Humans immediately recognize the words and their meaning. A neural network needs numbers. Before the model can process the question, the text passes through a tokenizer . 2. Tokenization: Breaking Text Into Pieces A tokenizer divides text into smaller units called tokens . Conceptually, our prompt might become: ["What", " is", " a", " build", " system", "?"] This is only an illustration. Actual tokenization depends on the tokenizer used by the model. A token isn't necessarily a complete word. It might represent: a complete word part of a word punctuation whitespace combined with text a number part of an identifier a programming-language symbol Each token is mapped to an integer called a token ID . Conceptually: ["What", " is", " a", " build", " system", "?"] ↓ [3923, 374, 264, 1975, 1887, 30] The IDs above are illustrative. The important part is the transformation: Human-readable text has become a sequence of numbers the model can process. But token IDs themselves don't capture useful semantic relationships. The number 1975 , for example, doesn't inherently ex

We introduce BDH-CQ, a reasoning system that brings these capabilities together. Demonstrations of a previously unseen task update recurrent memory; the query is then solved through iterative computation in a high-dimensional latent workspace. Intermediate reasoning states are not decoded into language. BDH-CQ makes memory, adaptation, and inference part of the same computational fabric. Inputs presented at inference time continuously update the model’s recurrent memory; the model then solves a query through iterative computation in a high-dimensional latent space, without verbalizing its intermediate reasoning. Neither task identifiers nor evaluation-task demonstration pairs participate in training, and no parameters are updated at inference time. A 150M-parameter configuration reaches 29.5% pass@2 on ARC-AGI-1 at a computed $0.00070 per task, breaking through the previously reported cost–accuracy Pareto frontier. submitted by /u/moschles [link] [留言]

Hi everyone, we noticed that the AC's comment, along with our reply, has disappeared, and we are wondering if anyone else has experienced the same thing. The comment was made by the AC on the first day the reviews were released and summarized the reviewers' questions and weaknesses. We addressed all of their questions in our reply, but now both posts (the AC's comment and our response) are gone. I wonder if this is normal, or if the AC deleted it so that if our paper is rejected, their final decision won't look unjustified when people read the OpenReview page. submitted by /u/Terrible-Chicken-426 [link] [留言]

Hi, How much does adding an honest limitations section hurt the paper (apart from making it better)? Does it bias the reviewers? Will they want you to fix the things in the limitations section? If the reviewers let AI read the paper, will the limitations section bias AI? Would it be better if the limitations section was hidden from the reviewers? And if the reviewers would have to author a limitations section? submitted by /u/strammerrammer [link] [留言]

When you build a system that detects key moments in a live sports feed, one architectural decision shapes everything downstream: where does the inference happen? At the edge, close to where the video is produced, or in the cloud, after the stream has been ingested? There is no universally right answer, but the trade-offs are sharp and worth laying out. The case for the edge Running detection near the source, at the venue or in an on-prem encoder, minimizes the round trip. The video does not have to travel to a data center and back before a moment is flagged, which can shave critical seconds off the time to clip. For genuinely live use cases, where a clip is worthless if it lands a minute late, that latency saving is the whole game. Edge inference also reduces egress: do the heavy frame analysis locally and ship only the clips that matter, instead of streaming everything to the cloud. The cost of the edge Edge hardware is constrained. You run on whatever GPU or accelerator fits in the rack at the venue, not on an elastic fleet. That bounds model size and concurrency. Updating models across many distributed edge nodes is an ops problem in itself, and a venue that hosts one event a week is idle hardware the rest of the time. Edge is fast but inflexible. The case for the cloud The cloud gives you elastic compute, easy model updates, and the ability to run larger or ensemble models you could never fit at the edge. If you process many concurrent streams, centralizing inference pools capacity instead of overprovisioning every venue. For workflows where a few seconds of extra latency is acceptable, near-live rather than instant, the cloud is simpler to operate and cheaper to scale. The cost of the cloud You pay for it in latency and bandwidth. Every frame you want to analyze has to be ingested first, and for high-bitrate broadcast feeds that adds up. The end-to-end path, capture, encode, transport, ingest, infer, clip, deliver, has more hops, each adding delay and a potenti

Hey! Can you tell me how you would go about building a recommendation engine for our question bank? The idea is that it understands a student’s strengths and weaknesses and recommends questions accordingly — more questions around the areas they’re weak in, but without making them so difficult that they feel demotivated. I also want it to occasionally bring back questions from older topics to check whether the student has forgotten something. Based on how they perform, it could then decide whether to recommend more questions from that topic or move on. Basically, the goal is for the recommendation engine to continuously understand where the student is struggling and use that to help them become better at problem-solving over time. I was learning some basics of AI/ML and this question came to my mind, so I was just curious — do you have any idea how something like this could be built? submitted by /u/whizzkidme [link] [留言]

I built a RAG assistant, then found out my architecture change made it worse, and I'm glad it happened I recently built a hybrid RAG (retrieval-augmented generation) support assistant for a fictional B2B SaaS platform, "Helix," designed to answer customer-success questions grounded in a 100-document knowledge base of product docs, runbooks, and resolved support tickets. It cleared production-readiness evaluation thresholds comfortably: 0.939 faithfulness and 0.775 context precision on a 50-query RAGAs test set, against required floors of 0.70 and 0.60. But the most useful thing that came out of the project wasn't the passing score. It was a hypothesis that turned out to be wrong, and what I did after finding that out. The setup The pipeline ingests a mixed-format 100-document corpus (Markdown product docs, PDF runbooks, HTML support tickets) into a Pinecone vector index, retrieves relevant context, and generates a grounded, citation-backed answer with an explicit confidence rating via an LCEL chain. Structured output is enforced with Pydantic ( answer , sources , confidence ), using gpt-4o-mini at temperature=0 , because a support assistant answering the same question against the same context should give the same answer every time. Determinism mattered more than creative variation here. Chunking wasn't one-size-fits-all. Three formats needed three strategies: Markdown docs were split by header first, so a chunk never crosses a topic boundary, with a recursive splitter as a fallback for long sections. PDF runbooks (no header structure to exploit) got a straight recursive character split. HTML tickets were kept as one whole chunk per ticket whenever possible, because a resolution often only shows up in the final turn of the conversation, and splitting a ticket risks separating the question from its answer. 5 scanned PDFs with no extractable text layer were detected and skipped gracefully rather than OCR'd, a conscious call I'll come back to. Result: 95 of 100 document

I kept paying a monthly subscription for a cloud coding assistant while a 16GB M4 Mac mini sat on my desk idling most of the day. So I ran the obvious experiment: can a 16GB Mac mini run a coding assistant entirely offline — no code leaving the machine, no subscription — and is it actually usable for real work? Short answer: yes, with one hard constraint (RAM) and one soft one (context length). This article is the written version of the video above, with every command, config file, and benchmark number so you can reproduce it. Table of contents Why bother running locally The hardware constraint nobody mentions Step 1: Install Ollama Step 2: Pick a model that fits in 16GB Step 3: Run and verify Step 4: Wire it into VS Code Step 5: Tune Ollama for a 16GB box Benchmarks What it does well, what it doesn't Should you cancel Copilot? Why bother running locally Three reasons, in the order that actually mattered to me: Privacy. Client code, internal repos, anything under NDA — none of it leaves the machine. This is the one thing a hosted assistant cannot offer you at any price tier. Cost. A coding assistant subscription is roughly $100–240/yr depending on tier. The Mac mini was already bought. Offline. Flights, bad hotel wifi, coffee shop dead zones. The assistant just works. The reason not to: raw capability. The frontier hosted models are better at large multi-file reasoning, and it isn't close. More on that below. The hardware constraint nobody mentions On Apple Silicon, the GPU and CPU share one pool of unified memory. A model has to fit in that pool alongside macOS, your browser, VS Code, and whatever containers you're running . On a 16GB machine, macOS + a normal dev environment eats 6–8GB before you've loaded anything. That leaves you roughly 7–9GB of realistic headroom for the model. This single number determines everything else, and it's why "just run the 30B model" advice from people on 64GB machines doesn't transfer. By default macOS allows the GPU to use about 7

Most classification metrics for oncology AI models (AUC, ICC, MAE) measure global agreement. They don't answer the question that actually matters at the point of care: how reliable is this model at the exact cutoff that decides whether a patient gets flagged, biopsied, or treated? I built oncothresh to evaluate models at a specific clinical threshold rather than in aggregate: sensitivity/specificity/PPV/NPV at the cutoff, bootstrap confidence intervals, threshold-sensitivity curves, boundary-weighted calibration, decision-curve net benefit, and number-needed-to-test. It's a small, dependency-light Python library (numpy/scipy/scikit-learn/pydantic) built for tasks like tumor cellularity, Ki-67, TMB, and PD-L1 scoring, where a continuous model output gets collapsed into a yes/no clinical decision at a fixed cutoff. Pathology-specific benchmarks like PathBench and PathBench-MIL evaluate foundation models globally but don't evaluate at predefined clinical thresholds with uncertainty quantification, which is the gap this fills. There's also a companion web dashboard ( oncothresh-web ) for people who want the same analysis without writing code: upload a CSV of predictions and labels, pick a threshold, get the full set of charts plus a downloadable PDF report. docker compose up and it's running locally, no cloud dependency. Library: github.com/omkaradhali/oncothresh Dashboard: github.com/omkaradhali/oncothresh-web Still v0.1, so I'd genuinely welcome feedback: use cases I haven't considered, edge cases in the DCA/calibration math, or places the API doesn't fit how people actually work with threshold-based models. submitted by /u/adom2989 [link] [留言]

This is the project my last two posts were building towards (this is the last of this silliness). I ported the Doom rendering algorithm to run inside a transformer. Instead of training a model, I used a compiler I wrote which converts computation graphs into transformer weights, and then ported Doom's algorithm into a compatible graph. The generated checkpoints can be loaded in Hugging Face without trust_remote_code -- it's just a standard transformers checkpoint. You feed the model a prompt representing the scene data, and generate until the model stops. The result is a token sequence which includes simple pixel drawing commands (to move the cursor, draw a pixel, etc). When you mechanically apply those drawing commands you get the rendered frame. The article includes the entire host program necessary to load the checkpoint, generate the render, and parse the output into the famous E1M1 frame. This host code is 43 lines of python. The python to define the computation graph is much longer, but that gets compiled into the transformer itself. One frame is a 3,614-token prompt plus 53,747 generated tokens -- just over 40 minutes on a B200. The original Doom could achieve 35 FPS on a 486. This achieves 35 FPD (frames per day) on a B200. Write-up: https://ood.dev/posts/doom/ Weights: https://huggingface.co/physicsrob/torchwright-doom-e1m1 Github for the source code which gets compiled: https://github.com/physicsrob/torchwright_doom/ submitted by /u/notforrob [link] [留言]