标签：#learn

Please post your questions here instead of creating a new thread. Encourage others who create new posts for questions to post here instead! Thread will stay alive until next one so keep posting after the date in the title. Thanks to everyone for answering questions in the previous thread! submitted by /u/AutoModerator [link] [留言]

MLE-Bench scores have jumped from 30% to 80% over the last two years. But how much of that is real algorithmic progress vs. better base models + problem definition shifts + overfitting? Turns out: not much. Once you control for the same step budget and models, and then test on a different set of tasks, the two-year-old AIDE algorithm matches modern agent/evolutionary search systems. Figure from FML-Bench, a new automated ML research benchmark, which unifies the code editing agent, step definition, and val/test split, and tries to benchmark the algorithmic efficiency (search/memory) of the agents. paper link: https://arxiv.org/pdf/2605.17373 test improvement and pairwise win-rate submitted by /u/Educational_Strain_3 [link] [留言]

Hi everyone, If you have purchased (at least one) GPU(s) for ML/DL studies and research: How is your experience and is it worth it? What do you use it for and how is the ROI? I have a MacBook Pro with M4 from some years ago, while MPS is useful in many occasions, it's no substitute for a NVDA GPU with CUDA support. So recently I am considering getting a 5060 Ti 16GB , but a GPU cannot run itself, so I then also need to buy other parts (e.g., CPU, RAM, SSD, motherboard, and so on...), which has been getting more expensive lately, especially the RAM. Since I'm still in job-seeking mode, I will mostly use it for learning DL, RL, and LLM-related things and local experiments (e.g., Stanford CS336), or low-level ones like GPU kernel programming and so on. Do you think a local physical GPU would help, or in my case a cloud service like Modal would suffice? Many thanks! submitted by /u/hedgehog0 [link] [留言]

An LLM-written kernel benchmarked 38% faster on a microbench. Here is what kernel-level validation showed it actually did at runtime. TL;DR Multi-agent LLMs are now writing CUDA kernels (RightNow AI’s AutoKernel, Meta’s KernelEvolve, a multi-agent system claiming 38% speedup on Blackwell). Source-level benchmarks measure clean throughput on a single isolated kernel. They do not measure SM occupancy under co-scheduling, DRAM bandwidth saturation, dispatcher off-CPU during a real serving workload, or NCCL wait correlation with sibling kernels. Kernel-level validation closes that gap: an eBPF trace of the same kernel running under the same workload as production answers all four questions in one capture. The kernel-writing wave Three pieces of work in April surfaced the same pattern: agents generate CUDA kernels, then quote a single throughput number against a baseline. RightNow AI’s AutoKernel (announced Apr 6) – LLM agents iteratively rewrite CUDA kernels for a target metric, claiming substantial speedups on selected microbenchmarks. Meta’s KernelEvolve – similar shape: agents propose kernel variants, rank by throughput, keep the best. Multi-agent system on Blackwell (Apr 29 reports) – claims a 38% speedup on a public kernel benchmark using a coordinated agent setup. All three are real research, all three produce real kernels, and all three report numbers that come from microbenchmarks. The microbench setup is exactly what you want for the optimization loop. It is not what you get in production. What microbenchmarks do not see Run an LLM-generated kernel under nvprof or nsight-compute on an otherwise-idle GPU and the throughput number is real. Put the same kernel in front of a vLLM serving workload and four properties change immediately: SM occupancy under co-scheduling. The kernel that achieves 95% SM occupancy in isolation will achieve 40-50% with three other kernels sharing the same SMs. The optimizer never sees this regime. DRAM bandwidth saturation. A kernel tha

How combining Non-Transferable and Permanent Delegate extensions changed my understanding of digital ownership on Solana. Imagine earning a high rank in an online game. You can use it. You can benefit from it. Other players can see it. But you cannot transfer that rank to someone else. If you break the game's rules, the game company can revoke it. And if the game gets acquired by another company, authority over that rank can be handed over to a new administrator. That was the closest mental model I found for understanding one of the most interesting things I built this week on Solana: a credential token using Token-2022 extensions. What are Token Extensions? One thing I've learned during this phase of the challenge is that Token-2022 is not just about creating tokens. It allows you to define behaviors directly at the token level through extensions. Instead of relying on application logic to enforce rules, the token program itself can enforce them. For this experiment, I combined two extensions: Non-Transferable — prevents ownership from being transferred to another wallet. Permanent Delegate — gives a designated authority the ability to manage the token even after it has been issued. Individually, those are useful. Together, they create something that behaves less like a currency and more like a credential. The Transfer That Was Supposed to Fail After creating the token, I tried transferring it. spl-token transfer Gn5PZzwDENvpQESaFwgVqzCUFba5sSka59iLtjFTYNvz 1 $THIRD_PARTY \ --owner ~/recipient-wallet.json \ --fee-payer ~/.config/solana/id.json \ --program-id TokenzQdBNbLqP5VEhdkAS6EPFLC1PHnBqCXEpPxuEb \ --fund-recipient --allow-unfunded-recipient The result: Recipient: ErYcpQYcpdoaaiufrB3MqQE2QaVEhoACZGcssppszPpY Recipient Token Account: 8CvyoKt11UbkWYRVXey6UKK2gqQcvVvbh9Lv3crzo8C3 Funding ATA: 8Cvyokt11UbkWYRVXey6UKK2qqQcvVvbh9LV3crz0803 Status: ❌ Transaction failed during simulation Error: Transfer is disabled for this mint (Token-2022 restriction) What I liked a

A bit of background about myself: I have been accepted to RWTH Aachen's Computer Science program starting this fall, and one of the things that I am genuinly excited about is exploring the intersection of astrophysics and machine learning. The tricky part is that RWTH's CS department doesn't have a research group focused directly on this intersection. The two closest things I have found are the Quantum Information Systems group (I plan to reach out to the them once I am on campus to understand a bit more about them) and the Learning on Graphs group which does foundational GNN research. The second one got me thinking: graph neural networks feel like they could be well-suited to astrophysicla data, things like galaxy formation, cosmic web structure or particle interaction data all seem graph-like (or am I being waaaay too optimistic here?) So my questions for people who know this space better than I do: Are GNN's already being used in astrophysics research? What other ML subfields would you point someone toward if they are interested in this intersection? I know I could have applied to a more well-suited university for my needs, but RWTH Aachen was my top choice because I am a math nerd and I really like their way of teaching. So do help a brother out. Thanks in advance!!!! submitted by /u/pandemic_179 [link] [留言]

Modern search engines do more than match keywords. When you search for "cozy mystery set in Paris" and get results for "atmospheric detective novel in France" that's vector search at work: documents and queries are converted into lists of numbers, called embeddings, and the search engine finds the documents whose numbers are closest to the query's. Manticore Search supports this natively. Under the hood, it uses a data structure called HNSW: a graph that connects nearby vectors, so it can find nearest neighbors quickly without scanning every document. That makes vector search fast enough to run on millions of documents in milliseconds. But HNSW has an inefficiency. Early in the traversal, almost every distance computation finds a better candidate than the ones already in the result set. As the search goes on, those improvements become rarer, but the algorithm keeps traversing the graph until it exhausts its exploration budget. By that point, the result set has often already converged, and the remaining work does little or nothing to improve it. Early termination fixes this by detecting that point and stopping early. The effect becomes more noticeable as k grows, where k is the number of nearest neighbors the query asks Manticore to return. Returning more neighbors requires more graph exploration, and much of that extra work happens after the result set has already stabilized. That also makes early termination more valuable, because it has more unnecessary work to cut. This gets more pronounced with vector quantization . Quantization compresses stored vectors to save memory, which slightly lowers search precision. To recover it, Manticore uses oversampling : it fetches 3x more candidates than requested, then rescores them using the original full-precision vectors. With the default 3x oversampling, HNSW explores many more candidates per query. Large k values often come from this kind of candidate expansion: an application may ask the vector index for hundreds or thous

Hey everyone, We’re building Antitech , a security layer for AI agents and LLM-powered workflows. We’re opening a small number of free early-access assessments for teams/builders working on AI agents. If you give us access to an endpoint of a Dockerized / sandboxed environment of your agent, we’ll test it against common and emerging AI-agent attack vectors, including: Prompt injection Indirect prompt injection Tool abuse Data leakage / exfiltration Fake authority / malicious context Unsafe agent behavior Weak guardrails and policy bypasses In return, you get a free vulnerability report showing what we found, how serious it is, and practical recommendations to harden your agent. This is completely free. No catch. We’re doing this because we want to work closely with real AI-agent builders while shaping the product. Early participants will also get: A big discount once the final product is ready Insider updates while we build Early access to new features The option to become a design partner Priority access to future assessments What we need from you: An endpoint of a sandboxed/Docker environment Permission to test within agreed boundaries A short feedback call after the report We won’t publicly disclose anything without your permission. If you’re building AI agents and want to know how they can be attacked before someone else finds out the hard way, DM me or comment below. submitted by /u/TheAchraf99 [link] [留言]

How to make LLMs deterministic, in plain English. The version I share with founders and product teams before they make decisions worth real money. You use AI tools every day. But can you explain what happens when you hit send? Most people cannot. And that gap is costing them. Bad prompts. Broken products. Decisions made on the wrong assumptions. The Hard Truth Every LLM explainer out there is written for researchers or so basic it tells you nothing useful. Neither helps you build better products or work with AI more effectively. This is the version I share with senior leaders, founders, and product teams before they make decisions worth real money. 1. It Is Not a Search Engine. It Is Not a Database. It Is a Prediction Machine. When you type a prompt and hit send, the LLM is not finding an answer from somewhere. It is predicting the most likely words to follow your input. Based on patterns it learned from billions of documents. That is the whole process. Wrong: "The AI knows the answer." Right: "The AI predicts the most likely answer based on what it has seen." This changes everything about how you use it. When an AI gives you a wrong answer confidently, it is not broken. It is doing exactly what it was built to do. Predict. Not verify. 2. The Autocomplete Comparison (And Why It Only Gets You Halfway) You have probably heard the phrase "autocomplete on steroids." It is not wrong. But it misses something important. Your phone autocomplete learned from your messages. An LLM learned from most of the written internet. Books. Research papers. Code. Billions of examples. At that scale, the patterns start to look a lot like real thinking. Not because the model understands in the way you do. Because it has seen so much that it can predict what a good answer looks like. When I was building AstroNayak I fed Vedic astrology principles into the system prompt. The LLM produced interpretations that genuinely surprised me. It did not know Vedic astrology. It had seen enough of it t

Without revealing too much information, what were the circumstances? submitted by /u/XTXinverseXTY [link] [留言]

[Paper Notes] JEPA: Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture 🔗 TL;DR: JEPA learns a a generalized semantic representation with less data pairs by predicting missing information in the embedding space , which helps it disregard unnecessary noisy from input(pixel)-level details and learns at a higher abstraction level with good semantic generalization. 1. Innovation & Significance The Bottleneck: Image-text data pair labels are hard to find Pixel level pre-training paired & data augmentation are strongly biased towards trained data distribution, hard to determine proper generalization and level of abstraction. JEA's (Joint Embedding Architecture) collapse probelm: encoder & decoder attempts to cheat by always landing on trivial constant when predicting itself (reconstruction) and gets away with an easy Error=0. The Solution: > Chain-of-thought ⭕ Mask pre-training to reduce data & generalize↓❌ Bad/lower semantic representation without semantic target, could be learning noisy local pixel correlation↓⭕ Learn at the embedding level to omit pixel input and generalize⭕ Adds context encoder & positional encoding to inject context and force model to pick up image inherent structure from reconstructing multiple masked patches with one target.↓❌ JEAs wants to cheat: if I always map all pixels to a constant for both the predictor and end target encoder then the reconstruction error is always collapsed to zero! Hehe~ ↓ ⭕ EMA (Exponential moving avg.): Update target encoder parameters from the EMA of context encoders. This 'delays' the target encoder to prevent collapsing (a trick from the BYOL paper[2020], proven essential to training JEAs with ViT). 2. Model & High-Level Intuitions 2.1 Model Architecture Input: randomly samples block masks from original image within certain aspect ratio changes, and apply mask for context image 2.1.2 Context Context Encoder: ViT encodes context image to embedding SxS_x S x ​ Mask Token : an [1,D] random

Introduction Understanding how to find prime numbers is one of the best ways to develop logical thinking in programming. It looks simple on the surface, but it teaches you how to break a problem into smaller steps, build a solution gradually, and then improve it into a clean and reusable structure. In this blog, we will not jump directly into code. Instead, we will start from basic thinking, slowly convert that thinking into logic, and finally refine it into a proper Python program using functions and loops. The goal is not just to find prime numbers, but to understand how programming logic is actually built in real development. 1. Understanding the Problem First Before writing anything in Python, we need to understand what a prime number actually means. A prime number is a number that: is greater than 1 has exactly two divisors: 1 and itself So the real question becomes: How do we check whether a number has any divisors other than 1 and itself? That is the core problem we are trying to solve. 2. Thinking Like a Human Before Coding Let’s take a number, for example 13. To check if 13 is prime, we naturally try dividing it by smaller numbers: 2 → does not divide 13 3 → does not divide 13 4 → does not divide 13 5 → does not divide 13 and so on If none of these numbers divide 13 completely, then 13 is prime. So the logic is simple: Try dividing the number by possible candidates and see if any divide it perfectly. 3. Turning Thinking into a Basic Algorithm From the above idea, we can form a basic structure: We need: a number to test a variable that moves through possible divisors a way to detect whether a divisor exists We start checking from 2 because every number is divisible by 1 anyway. We also do not need to check beyond half of the number, because a number cannot have a divisor greater than half (except itself). So the idea becomes: Start divisor from 2 Go up to number // 2 If any number divides it evenly, it is not prime 4. First Working Logic (Direct Implementati

Hey everyone, I'm trying to get back into the loop on world models. The last time I followed SSL closely, the buzz was all about Barlow Twins and DINO, but now everything just looks like scaled-up video generation from big industry labs. What is the actual academic research community stressing over right now? submitted by /u/nat-abhishek [link] [留言]

RecursiveMAS (arXiv 2604.25917) showed that agents sharing internal reasoning state outperform agents that share only final outputs. The average accuracy gain across benchmarks was 8.3 points. The mechanism: each agent passes not just its answer but the latent embeddings from its own reasoning process, and the next agent conditions on both. The paper is a good result. The catch is access. RecursiveMAS requires open-weight models with hidden states exposed at inference time. That rules out Claude, GPT-4o, and Gemini. I built a Claude-native version using the Anthropic extended thinking API. The core idea transfers: instead of passing latent vectors, pass the full thinking text. The paper calls it internal state sharing; the Claude version calls it thinking-block relay. The architecture problem Claude's extended thinking blocks carry an encrypted signature tied to the originating conversation. You cannot pass a signed thinking block into a different agent's messages array. The API rejects it. The workaround: extract the text from the thinking block and inject it as a regular user message. # Extract thinking text from Agent 1 thinking_text = next ( ( b . thinking for b in response . content if b . type == " thinking " ), "" ) # Inject into Agent 2 as regular context, not as a thinking block context = f " Prior agent reasoning: \n { thinking_text } " The signature does not transfer. The reasoning does. relay-structured: what I built first The first architecture was a Planner > Critic > Solver loop where each agent emits a compact mental model JSON instead of raw thinking text. Raw thinking at a 1024-token budget is often compressed and fragmented. The hypothesis was that 150 tokens of structured signal carries more information per token than 1024 tokens of compressed prose. The schema each agent emits: { "interpretation" : "how the agent read the problem" , "key_steps" : [ "step 1" , "step 2" ], "rejected_approaches" : [ "approach tried and discarded" ], "confidence" :

You can’t see AC comments yet, but you can see the Accept/Reject consoles. My paper (with scores of 8,6,3) got rejected. submitted by /u/GeeseChen [link] [留言]

submitted by /u/Mushroom-Severe [link] [留言]

Stepping up to give my first-ever presentation at UbuCon 2026 was a massive milestone, and honestly, it was pretty intimidating. The stakes felt high, especially with the live demo. It was a race against the clock to get everything running, and it only finally came together exactly ten minutes before I went on stage. Talk about a close call. While I am proud of what I delivered, I originally wanted to pack even more into the session. I had planned to showcase a simulated mission, Gazebo visualizations and RViz path simulations. While time caught up with me for the presentation, these features are still actively in the works over at the aeronix project. My goal is to have the entire end-to-end setup completed and ready by the end of the year. I connected with some incredible engineers and industry peers and I am looking forward to building on those conversations for future professional collaborations. This experience proved that the best way to grow is to just put yourself out there. Moving forward, I plan to keep speaking on topics that challenge me. It is the ultimate way to deepen my own technical understanding share what I have learned with the community and grow professionally.

i'm trying to train an ASR model using the LibriSpeech recipe from SpeechBrain (without the language model) on a 100-hour dataset of dialectal Arabic speech. the model architecture uses a Conformer-small encoder and a Transformer decoder, with a total of around 13M parameters. the recipe uses a combination of two loss functions: CTC and KL divergence, specifically: 0.3 * CTC + 0.7 * KLDiv during training, both losses drop significantly during the first few weight updates, but then quickly plateau. the CTC loss gets stuck fluctuating around the 60-80 range, while the KL divergence loss remains around the 60s as well for the rest of training. as a result, the model does not converge properly, and the validation WER stays close to 100%. i’ve already tried several things: adjusting the learning rate, changing the number of warmup steps, modifying the number of epochs, tuning the batch size and reducing the vocabulary size from the default 5000 to 1000. none of these changes seem to help. the training dataset is not publicly available and is weakly labeled. the validation and test sets come from the MGB2 dataset. at this point, i genuinely don’t know what the root cause might be. i’ve experimented with many different approaches, but the model still refuses to converge. has anyone encountered a similar issue where their model gets stuck early in training and never improves? if so, what ended up being the cause or solution? any feedback, suggestions, or ideas would be greatly appreciated. submitted by /u/Sweet-Hamster-4991 [link] [留言]

Every time I sit down to learn something, I find myself trapped in the same cycle. I start with a tutorial. Halfway through, someone says there's a better tutorial. I switch. Then I discover a book that supposedly explains the topic better than the tutorial. Then a YouTube video claims the book is outdated. Then a developer on social media recommends an entirely different resource. Before I know it, I've spent three hours researching how to learn instead of actually learning. Does the information overload stop or will there always be another resource, another course, another book, another video, another roadmap, another expert with a different opinion. The internet has made knowledge abundant, but abundance creates a paradox of choice. One person says to learn JavaScript from documentation, another says build projects immediately, another recommends a paid course, someone else insists that free resources are better. Every recommendation sounds convincing. Every path seems important. The result is paralysis. Instead of moving forward, I keep searching, instead of building I keep comparing, instead of learning I keep consuming. finished teaches more than a hundred bookmarked tutorials. At some point, every learner must accept a difficult truth that the goal is not to find the best resource it is to become better. Those are not the same thing. A person can spend months researching the perfect learning path and never write a meaningful line of code while another person can pick a decent resource, make mistakes, build projects, and improve every day. The second person wins not because they found better information but because they used the information they already had. I've started realizing that learning is a lot like fitness. At some point, reading about exercise becomes a form of avoiding exercise. The same thing happens in programming. Reading about coding becomes a way to avoid coding. Researching becomes a substitute for practice. The search for the perfect resourc

AI จะครองโลก เป็นคำที่ได้ยินมานาน เท่าที่ผู้เขียนจำความได้ก็มี Judgement Day ยุคหนัง Terminator แต่หากจะจริงจังขนาดโยงเข้าความเป็นจริงก็ยังไม่มีอะไรชัดเจน แต่วันนี้เรามีหลักฐานพิสูจน์ได้จริงแล้ว ด้วยความ Novelty จาก OpenAI ที่สามารถค้นพบความรู้ใหม่ที่ไม่เคยมีมนุษย์ค้นพบมาก่อน หักล้างความเชื่อที่ว่า AI ทำได้เพียงนำสิ่งที่มนุษย์ค้นพบแล้วมาเรียงต่อกัน ในเดือนพฤษภาคม 2026 reasoning model ภายในของ OpenAI ได้ disprove Erdős Planar Unit Distance Problem ซึ่งเป็นปัญหาและข้อคาดการณ์ทาง Combinatorial geometry ที่ Paul Erdős ตั้งไว้ตั้งแต่ปี 1946 โจทย์ระบุว่า เมื่อวางจุด nn n จุดบนระนาบ จำนวนของคู่จุดที่ห่างกันพอดี 1 หน่วยจะมีได้มากที่สุดเท่าใด Erdős แสดงความเป็นไปได้ผ่านการจัดเรียงแบบ grid ว่าได้จำนวนคู่ที่เติบโตเหนือเส้นตรงเพียงเล็กน้อย และตั้งข้อคาดการณ์ว่าไม่มีโครงสร้างใดทำได้ดีกว่านี้อย่างมีนัยสำคัญ ข้อคาดการณ์นี้ได้รับการยอมรับในวงกว้างตลอด 80 ปีที่ผ่านมา และยังไม่มีข้อคาดการณ์ที่ดีกว่านี้ จนกระทั่ง OpenAI ได้เผยแพร่ Chain of Thought (CoT) เรียบเรียงความยาว 125 หน้า ซึ่งบันทึกลำดับการให้เหตุผลของโมเดลไว้ทั้งกระบวนการว่าโมเดลไปถึงคำตอบอย่างไร กรอบของคำตอบ: lower bound กับ upper bound ก่อนเข้ากระบวนการทำงานของโมเดล ขออธิบาย "กรอบ" ของคำตอบของปัญหานี้ก่อน เพราะคำตอบถูกล้อมไว้ด้วย lower bound จำนวนที่สร้างได้จริงแล้ว อย่างน้อยเท่านี้ และ upper bound เพดานที่พิสูจน์แล้วว่าเกินไม่ได้ ด้าน lower bound นั้น Erdős เอง (1946) ใช้การจัดเรียงแบบ grid แสดงว่าสร้างได้ถึง n1+Ω(1/log⁡log⁡n)n^{1+\Omega(1/\log\log n)} n 1 + Ω ( 1/ l o g l o g n ) ซึ่งมากกว่าเส้นตรงเพียงเล็กน้อย และเข้าใกล้ศูนย์เมื่อ nn n ใหญ่ขึ้น ส่วนด้าน upper bound นั้น Erdős พิสูจน์เพดานแรกไว้ที่ O(n3/2)O(n^{3/2}) O ( n 3/2 ) จากวงกลมหนึ่งหน่วยสองวงตัดกันได้ไม่เกินสองจุด โดยต่อมา Spencer–Szemerédi–Trotter (1984) บีบเพดานนี้ลงมาเป็น O(n4/3)O(n^{4/3}) O ( n 4/3 ) ซึ่งเป็น upper bound ที่ดีที่สุดจนถึงปัจจุบัน และยังคงอยู่หลังการค้นพบของ OpenAI model วิธีเก่าของ Erdős: วงกลมรัศมีเลือกมาลากผ่านจุด grid หลายจุดพร้อมกัน ทำให้ระยะซ้ำมีมาก สิ่งที่ Erdős คาดการณ์คือ คำตอบจริงของ Planar Unit Distance Problem ควรอยู่ชิดด้าน lower