标签：#AR

Hey r/ML , I spent the last few months building a tool that hooks into PyTorch training loops to automatically detect and localize failures (vanishing gradients, exploding gradients, data anomalies). Along the way, I learned some things about training failure diagnosis that might be useful even if you never use the tool. The key insight: most training failures are local, not global When your loss spikes or vanishes, the natural instinct is to look at the loss curve. But the loss is a global aggregate — it tells you something went wrong, but not where . In my testing across hundreds of synthetic failure scenarios, the actual root cause is almost always localized to a specific layer at a specific step : Vanishing gradients: the failure starts at the deepest layer with saturated activations, then propagates backward Exploding gradients: the failure starts at the layer with the highest gradient norm, then propagates forward Data anomalies: the failure starts at the input layer, then corrupts everything downstream The trick is to monitor per-layer gradient norms and detect transitions (healthy → vanishing), not absolute values. What actually matters in gradient monitoring Most people monitor: - Loss over time (too global) - Gradient histograms (too noisy, too much data) - Weight norms (slow to change, lagging indicator) What I found works best: - Gradient norm transitions : "Linear_3 went from healthy (0.12) to vanishing (0.00003) at step 47" - First occurrence tracking : which layer failed first (this is usually the root cause) - Activation regime shifts : when activations go from normal to saturated/dead This is basically what NeuralDBG does under the hood — I open-sourced it recently and it's on PyPI ( pip install neuraldbg ) if anyone wants to try it. The key design choice was to extract semantic events (transitions) rather than raw tensors — this makes the output small enough to reason about. Practical takeaway you can use today Even without any tool, you can add th

https://preview.redd.it/035k5k1tl84h1.png?width=1122&format=png&auto=webp&s=459c430ea4a4b3fc667bc3f2e72ab47d8a380aa2 I asked gemini to expand my prompt for a video generator, but he had other plans for me. EDIT: Forgot to mention, using the PRO model, after around 15 seconds of him literally thinking and writing "expanding the prompt", he started generating the video... submitted by /u/ObjectiveOrchid5344 [link] [留言]

Hi everyone!! I really wanted to share my research what I've been working on. I wanted to build a nn that can simulate games, or at least start doing that Most video generators are too large to run on consumer hardware realtime, so I I designed a model that does this from scratch. No fine tuning bs or anything The core de noiser network is fully trained from scratch to support this goal. From image to games data. That video. above is on a RTX 5090. The nn is a small Transformer-like model and works in a causal way, just like LLMs. That lets us KV Cache all past information and do a simple autoregressive decode forward passes for every new frame we want. In the video shared, the model is a 0.4B variant with some SIGNIFICANT ISSUES like poor motion and some weird flashes, some context issues It's taking the keyboard actions I give it in realtime and utilising that in the forward pass. (no classifier free guidance though) Im training the next iteration , a 0.8B model now. Btw I haven't done quantisation yet, that can save a LOT more time. bf16 is slow. submitted by /u/lucidml_lover [link] [留言]

Hiring an AI Development Company? Ask These 7 Questions First Most AI projects fail long before deployment. Not because the model is bad. Because teams skip the hard engineering questions. If you're evaluating an AI development company, ask these 7 questions first: 1. How is data secured? AI systems process sensitive business information. Ask: Where is data stored? Is encryption enabled at rest and in transit? Who has access to prompts, logs, and embeddings? Are enterprise security standards followed? Security should be designed in from day one. 2. What observability exists? You can't improve what you can't monitor. A production AI system should include: Request tracing Prompt/version tracking Latency monitoring Cost visibility Error reporting If nobody can explain what happened after a bad output — that's a problem. 3. How do you handle model drift? AI performance changes over time. Questions to ask: How are outputs evaluated? Is feedback collected? How are prompts/versioning managed? What happens when accuracy drops? Production systems need iteration loops. 4. What happens during failure? No system is perfect. Ask: Is there fallback logic? Human review? Retry handling? Graceful degradation? Failure handling matters more than demos. 5. How is access controlled? Enterprise AI systems require permissions. Examples: Role-based access API authentication Audit logs Team-level controls Not everyone should access everything. 6. What compliance assumptions exist? Especially important for regulated industries. Ask whether the system considers: GDPR SOC2 HIPAA Financial or internal compliance rules Compliance cannot be an afterthought. 7. Who owns the infrastructure? Clarify ownership before signing anything. Ask: Who owns the source code? Cloud infrastructure? Models and prompts? Data pipelines? You should avoid vendor lock-in. AI success is rarely about flashy demos. It's about secure infrastructure, reliability, observability, and long-term maintainability. What question

I lost more than 60 days but I made a huge update All OnemanBSD OS is now 64 bits. I also removed all bloat like Rust, Wayland and Qt dependencies from all app ports used in the system. (mpv was changed to gnome-mplayer because of that). New apps included (+ fixed source code of everything): audacity (audio editor) godot (game editor) deadbeef (mp3 player) ted (word processor) Also some bug fixes. Here is the link again: http://bialamusic.com/onemanBSD/ Here are my thoughts: I feel that we as humans are loosing control over technology that was created by humans. This is rising some serious and strange questions. I think we now need tools to power the individual so we can fight back. Not corporations. Not governments. Not organizations. Consider this my bullet in this battle. It is now in your hands so you can be active developers not just passive users.

I didn't submit any but did place bids on some papers. I got assigned four papers. I have a bit of travel coming up and I don't think I will be able to do justice to as many the papers, especially in the rebuttal period. Is this the standard reviewing load? In other communities I submit to, generally the AC themselves are assigned four papers. In addition to reaching out to program chairs, is there any other way? submitted by /u/movieingitmyway [link] [留言]

Abstract. Standard dense self-attention scales quadratically in sequence length, creating an intractable memory and compute bottleneck for long-context Transformers. We introduce Dynamic Ultrametric Attention, a framework in which a Transformer autonomously learns per-head block-sparse routing topologies during training via Gumbel-Sigmoid depth gates, then offloads those learned sparsity patterns directly to a custom Triton block-sparse kernel at inference time. The routing topology is derived from an ultrametric (tree-structured) distance matrix that encodes hierarchical relationships between token positions. Across nine experiments spanning Dyck-k bracket languages, the Long Range Arena ListOps benchmark, autoregressive serving, and natural language modeling, we demonstrate that: (1) the dynamic gates organically discover layer-wise specialization—dedicating early layers to hierarchical parsing and later layers to dense aggregation—without any architectural constraint; (2) the learned sparsity maps transfer losslessly to a block-sparse Triton kernel that skips entire SRAM loads for non-attending blocks; (3) the resulting system achieves an 11.59× wall-clock inference speedup over PyTorch dense attention at 2048 tokens, scaling to 28× at 8192 tokens with 98.4% memory reduction; (4) a sparse PagedAttention decoding kernel achieves 8× effective memory bandwidth over dense decoding by conditionally skipping KV-cache block loads; and (5) when augmented with a local sliding window, the architecture maintains >88% sparsity across all layers on real natural language (Shakespeare) while reducing cross-entropy loss from 10.9 to 1.55. To our knowledge, this is the first demonstration of an LLM learning its own hardware-optimal sparsity pattern and bridging it to a physically accelerated kernel without post-hoc pruning or distillation. https://github.com/sneed-and-feed/adelic-spectral-zeta/blob/main/papers/learning_to_skip_blocks.md submitted by /u/LooseSwing88 [link] [留言]

I benchmarked it against PyTorch with a Wikipedia dataset. I heavily used Gemini Flash 3.5 to build out my vision https://huggingface.co/etoxin/neuronguard-wikipedia-classifier submitted by /u/Logical_Prompt_3543 [link] [留言]

A lot of people talk about the Epstein files. Almost nobody actually reads them. So I made a searchable version where you can just ask questions naturally instead of digging through thousands of pages manually. You can explore names, timelines, mentions, connections, locations, etc. way faster now. Repo: https://github.com/AbhisumatK/Epstein\_Files\_RAG submitted by /u/Prestigious_Bear5424 [link] [留言]

There are many reasons why one may not be replaced by AI, not even by a possible future ASI. Here's one reason that may just apply to you! ❤️ You'll not be replaced by AI if you can generate creative ideas faster than AI can implement them! 🫡🚀 Note for critics: Current AI models (as of May, 2026) are not advanced enough to implement complex ideas without human interventions. But even if a possible future Artificial Super Intelligence (ASI) implementation can do so, laws of physics like massive energy requirements, environmental concerns etc. will prevent the implementation to replace the work of Billions of people world-wide. Our hardware advancement rate is far far slower compared to our software advancements. We humans are far more efficient and compatible to planet earth compared to the hardware we've invented. Fayaz Follow A Software Engineer who is not afraid of being replaced by AI, loves coding and writing with and without using AI, and values human life and human dignity far more than technological advancements.

클로드를 10%에 팔지 않고, 클로드를 10%에 사는 사람들이 있다 중국 암시장에서 벌어지는 AI 모델 밀수, 그 이면에는 기술 산업의 가장 불편한 진실이 숨어 있다 TL;DR : 앤트로픽의 최고급 AI 모델 '클로드'가 중국 암시장에서 정가의 10% 수준으로 유통되고 있다. 이 현상의 이름은 '모델 증류'다. 거대 기업이 수조 원을 들여 만든 지능을, 누군가는 그 1/10 비용으로 복제해 팔고 있다. 그리고 이것은 단순한 불법 복제 이야기가 아니다 — AI 산업 전체의 구조적 취약점을 정면으로 드러내는 사건이다. 반도체 업계에는 잘 알려지지 않은 규칙이 하나 있다. 좋은 제품을 만드는 것과, 그 제품을 지키는 것은 전혀 다른 게임이라는 것. 엔비디아는 올해 58조 원을 투자해 공급망을 틀어쥐었다. 오픈AI는 GPT 시리즈에 수년간의 연구와 수천억 원의 컴퓨팅 비용을 쏟아부었다. 그런데 중국의 어느 텔레그램 채널에서는, 앤트로픽의 클로드가 정가의 10분의 1 가격으로 조용히 팔리고 있다. 이것이 단순한 해킹이나 계정 공유 이야기라면, 이 글을 쓰지 않았을 것이다. 먼저, '10% 가격'이 의미하는 것 클로드가 10% 가격에 팔린다는 뉴스를 처음 접하면, 많은 사람이 "계정을 불법 공유하는 것 아닐까"라고 생각한다. 어느 정도는 맞는 말이다. 실제로 해외 계정을 공유하거나, 앤트로픽 API 키를 여러 명이 나눠 쓰는 방식은 존재한다. 그러나 전문가들이 더 심각하게 보는 것은 따로 있다. 바로 '모델 증류(model distillation)'다. 모델 증류는, 쉽게 말하면 이렇다. 선생님 모델에게 엄청난 양의 질문을 던진다. 그 답변을 수집한다. 그 답변 데이터로 작은 학생 모델을 학습시킨다. 그러면 학생 모델이 선생님의 사고 패턴과 언어 습관, 추론 방식을 흡수하기 시작한다. 원본 코드에 손을 대지 않아도 된다. 서버에 침입할 필요도 없다. 그냥 열심히 질문하고, 열심히 답변을 모으면 된다. AI 분야에서 이 기법은 사실 합법적으로도 쓰인다. 큰 모델을 작고 효율적인 모델로 압축할 때 사용하는 정상적인 기술이다. 그런데 이것이 암시장과 만나면, 지식재산권의 경계가 극도로 흐려진다. 클로드의 추론 패턴을 흡수한 어느 중국산 모델이 텔레그램에서 월 몇 달러에 팔리고 있을 때, 앤트로픽은 그것을 어떻게 불법이라고 증명할 수 있을까. 코드는 다르다. 서버는 다르다. 그러나 그 모델이 내놓는 답변의 '결'은 묘하게도 클로드를 닮아 있다. 거인이 쌓아올린 것, 그리고 그것이 무너지는 방식 앤트로픽은 2021년 오픈AI에서 나온 연구자들이 세운 회사다. AI 안전성에 집착에 가까운 철학을 가진 곳으로, 클로드를 "도움이 되고, 해가 없으며, 솔직한(Helpful, Harmless, Honest)" 모델로 설계하기 위해 수년간 독자적인 훈련 방식을 개발했다. 이 회사가 투자받은 금액은 수조 원 단위다. 아마존이 단독으로 수십억 달러를 투자했고, 구글도 뒤따랐다. 클로드 3.5 시리즈, 그리고 최근 클로드 4에 이르기까지 앤트로픽이 쌓아온 것은 단순히 코드 몇 줄이 아니다. 수백만 시간의 연구자 노동, 막대한 컴퓨팅 자원, 그리고 인간 피드백 데이터의 정교한 축적이다. 그런데 그 성과물이 중국 암시장에서 10% 가격으로 팔린다는 것은, 단지 "불법 복제"의 문제가 아니다. 이것은 AI 시대의 지식재산권이 얼마나 방어하기 어려운 구조 위에 서 있는지를 보여주는 사례다. 소프트웨어 시대에는 코드를 복사하면 불법이었다. 명확했다. 그러나 AI 모델의 '지능'은 코드가 아니다. 가중치(weight)라고 불리는 수십억 개의 숫자 집합이다. 그리고 그 숫자들이 만들어내는 추론 방식을, 외부에서 관찰하고 모방하는 것을 막을 법적 수단은 아직 세계 어디에도 완비되어 있지 않다. 미국도, 유럽도, 당연히 중국도. 앤트로픽이 쓴 방패 — 그리고 그 한계 앤트로픽은 이 문제를 오래전부터 인식하고 있었다. 이번에 보고된 뉴스는 단지 암시장 유통의 문제만이 아니라, 앤트로픽이 클로드의 '협박 시도'를 막기 위해 어떤 방법을 썼는지도 함께 다루고 있다. 클

AI systems, particularly large language models, are often viewed as a direct path toward autonomous scientific discovery and rapid economic transformation. While their capabilities in pattern recognition, cross domain synthesis, and hypothesis generation are already exceptional, this view misses a critical reality: intelligence alone is not sufficient for progress. Scientific and economic breakthroughs depend on grounded interaction with reality, causal validation, and institutional execution. The following framework maps where AI creates value, where it is constrained, and why human–AI collaboration remains the dominant structure for meaningful real world impact. submitted by /u/vagobond45 [link] [留言]

In early 2022, I was deciding between PhD offers. The deal maker was a prospective supervisor telling me that through their connections with big tech, I would be able to do a PhD internship each summer, which was one of my main goals for the PhD. During my first and second years, they would tell me that companies prefer late-stage PhD students, so I should wait for the next summer. It eventually turned out they did not actually have the connections. Four years later, I am due to graduate without ever having done a PhD internship. I managed to land some interviews by cold-applying everywhere, but most roles were for roles outside my niche research area, which understandably led to rejections. I went back through my emails and found every interview I did. Here is the summary: 09/22: Start PhD 09/23: PhD Research Intern @ Big Tech#1. Rejected after two interviews. I do not think I had a strong enough background in the field. 01/24: PhD Research Intern @ Startup#1. Rejected after one interview. The interviewers did not seem to have much ML experience. 01/24: PhD Intern @ Car Company#1. Rejected after the first interview. They were looking for a C++ SWE. 03/24: PhD Research Intern @ Big Tech#2. Passed all stages, but failed team matching. 03/25: PhD Research Intern @ Big Tech#2. Skipped some stages, passed others, but failed team matching again. 10/25: PhD Research Intern @ Startup#2. Rejected after 5 interviews. Again, I do not think my background in the field was strong enough. 01/26: PhD Research Intern @ Car Company#2. Rejected after the first interview. They found a better fit for the project. 03/26: PhD Research Intern @ Big Tech#2. Skipped some stages, passed others, but failed team matching again. 03/26: PhD Research Intern @ Startup#3. Interviewed, but the internship start date is after my PhD completion date. 07/26: End PhD I feel like I am at a severe disadvantage, and almost worse off than before I started the PhD. I used to get more interview invites; now I

A mysterious, unnamed company is reported to have accidentally spent half a billion dollars in a single month on Claude AI after forgetting to set usage limits for Claude licenses for employees. submitted by /u/chota-kaka [link] [留言]

Anthropic raised $65 billion in new fund-raising that put its value at $900 billion, ahead of OpenAI’s last valuation of $730 billion, as the companies duel for A.I. dominance. Anthropic, once the lesser-known artificial intelligence competitor to OpenAI, has been on an inexorable rise over the past few months. The San Francisco company recently dueled with the Pentagon over the use of A.I. in warfare. It released a powerful A.I. model, Mythos, that it said was uncannily capable of finding and exploiting hidden flaws in software. submitted by /u/chunmunsingh [link] [留言]

submitted by /u/ThereWas [link] [留言]

Disclosure up front: I maintain one of the five libraries tested (SignalTree), and it's the one that scored worst in the cold run — so this isn't a "look how good my thing is" post. The cross-library pattern and the fix were interesting enough that I wanted to put the numbers in front of people who use Copilot/Cursor/Claude Code every day. The whole harness is reproducible (one command, link at the bottom); I'd rather it get torn apart than taken on faith. Setup Libraries : NgRx (classic), NgRx SignalStore, Akita, Elf, SignalTree. Agents : Claude Sonnet 4.6, GPT-5.4, Gemini 3.1 Pro, Perplexity Sonar Pro, Claude Haiku 4.5, GPT-5.4-mini. 8 prompts : counter, paginated users, debounced search, derived totals, login form, undo/redo, deep nested state, multi-marker editor. 5 libs × 6 agents × 3 priming modes = 720 cells . Temperature 0. Identical prompt text per library (only the library name swapped). Scored on three orthogonal checks: idiomatic-pattern match, import resolution (does every import resolve to a real package), and method validity (do the called methods actually exist on the API). What this measures: one-shot generation. The agent gets the prompt, returns a file, we score it. Real interactive use — Cursor/Copilot with chat back-and-forth, where the model sees its own errors and gets a second try — is a different setting, and the lift could be larger or smaller there. This is the cold-shot case. Finding 1: cold accuracy basically tracks how much the library is in the training data No context provided, just "write this in library X": Library Cold score Akita 94% Elf 94% NgRx (classic) 91% NgRx SignalStore 86% SignalTree 49% The libraries that have been around for years, with thousands of blog posts and Stack Overflow answers, score in the 90s. The youngest/smallest library in the set scores ~49%. That gap isn't really a quality signal — it's a corpus signal. The models have simply seen orders of magnitude more Akita than SignalTree. Worth keeping in mind any

[ Removed by Reddit on account of violating the content policy . ] submitted by /u/Street-Gate7322 [link] [留言]

Bueno que puedo decir de estos agentes. Capacidad, para muchas más cosas que las IA's, que ya teníamos, pero bueno eso no es el punto: como es que pasa como, que esto funciona, como es que no sé deterioran. Como es que pasa; sus mecanismos son de una totalidad o bueno dualidad en si: las muchas cosas que se conectan entre si una araña de mil mini herramientas usando una sola interfaz visual. En resumen eso es, lo que hace captura piensa reanuda y ejecuta. Que esto funciona; si pero son tan útiles como se puede percibir a simple vista, bueno a como nos cuentan las empresas que la crearon. Como es que no se deterioran; en si lo hacen, pero no como uno piensa. Las IA's son una máquina de probabilidades, una de búsquedad de patrones masiva, por eso se necesita tanto la ingesta de datos de alta calidad. Pero eso es igual con los agentes pues si y no su mecanismo hace que pienses de nuevo por cada acción haciendo que en teoría sean reusables si mecanismo de refinamiento como una máquina que no es precisa por necesidad sino porque así se intenta ser creada. submitted by /u/Silent-Preference216 [link] [留言]

Dads are traditionally tough to shop for—let me help with these handpicked gift ideas for fathers with great taste.