标签：#deeplearning

By zipflow.xyz This is an independent technical analysis of DeepSeek's public research and documentation. It is not an official DeepSeek statement, and it does not claim that the current Vision-Exp API is available through our upstream channel. When DeepSeek released deepseek-v4-flash-vision-exp , the obvious story was that a text-focused model had finally gained native image input. The more useful story is longer: DeepSeek had already spent years exploring visual data, vision-language alignment, OCR, charts, documents, and unified visual understanding and generation. This article reconstructs that public research lineage and separates three things that are often mixed together: What DeepSeek's papers actually disclose What the current API documentation says What we still cannot verify about the newest model's training data 1. DeepSeek-VL: starting from real-world visual data DeepSeek-VL's 2024 paper, Towards Real-World Vision-Language Understanding , did not frame vision as only a captioning problem. It explicitly targeted practical inputs such as web screenshots, PDFs, OCR, charts, and knowledge-oriented visual content. The project also described a taxonomy derived from real user scenarios. That taxonomy was used to build instruction-tuning data for tasks including recognition, transcription, conversion, analysis, commonsense reasoning, logical reasoning, multi-image comparison, and safety-related prompts. The model family combined three major pieces: A hybrid vision encoder A vision-language adaptor A DeepSeek language model The hybrid encoder paired a lower-resolution semantic branch based on SigLIP-L with a higher-resolution branch derived from a SAM-B-style encoder. The design goal was practical: global semantic understanding is not enough for small text, dense documents, OCR, and visual grounding. The three-stage training recipe The paper described a staged approach: Adaptor warm-up: train the vision-language adaptor while the primary vision and language comp

In mid-2026, FP8 and FP4 have become essential tools for making large-scale AI training and inference more efficient . FP8 uses two main formats-E4M3 for better precision on activations and weights, and E5M2 for wider dynamic range on gradientswhile NVIDIA’s NVFP4 takes things further with 4-bit values and micro-block scaling (shared FP8 scales per 16 elements plus a tensor-level scale). These formats dramatically cut memory use and increase throughput on modern GPUs compared with traditional BF16 or FP16, making it possible to train and serve bigger models on the same hardware. The benefits are clear: roughly 2× memory savings with FP8 and up to 3.5× with NVFP4, higher Tensor Core performance, and better energy efficiency. The trade-offs come from reduced numerical range and precision, which can lead to accuracy loss or instability unless carefully managed with techniques such as delayed scaling, stochastic rounding, Hadamard transforms, and selective quantization that skips sensitive layers. When these methods are applied properly, accuracy often stays within 1–2 % of higher-precision baselines on real workloads. Research has moved quickly from the foundational 2022 FP8 paper to 2025 studies showing stable FP4 pre-training of multi-billion-parameter models. Hardware support is mature for FP8 on Hopper GPUs and reaches its peak on Blackwell with native NVFP4 and MXFP8 acceleration. Among frameworks, PyTorch currently leads with native float8 dtypes, Transformer Engine for production training, and TorchAO for optimized inference. JAX offers solid support through Transformer Engine, TensorFlow/Keras provides simpler quantize-to-FP8 options but relies more on TensorRT for high performance, and libraries such as bitsandbytes remain useful for complementary 4-bit memory savings. Practical adoption is already strong for both training and inference, especially when teams start with proven recipes, monitor scaling factors, and prototype on smaller models. Workarounds for r

A small convolutional network beats a plain flatten-and-feed-it-forward network by 7.0 points on CIFAR-10. That's convolutions, pooling, normalisation and skip connections doing honest work. Then I shuffled the rows of every image, destroying no information at all, and that 7.0-point margin fell to 0.3 . Same architecture. Same data, in a strict sense I'll defend in a moment. Almost the entire advantage, gone. The experiment Take one fixed permutation of the 32 row indices. Apply it to every image in the training set and every image in the test set — the same permutation, every time. import torch g = torch . Generator (). manual_seed ( 1234 ) row_perm = torch . randperm ( 32 , generator = g ) def shuffle_rows ( x ): # x: (C, H, W) return x [:, row_perm , :] print ( row_perm [: 8 ]. tolist ()) # [15, 9, 8, 1, 4, 12, 30, 7] That's the whole intervention. Then train two models twice each — once on natural images, once on shuffled ones: Model Params Natural rows Shuffled rows Flatten → 512 → 10 (MLP) 1,578,506 51.4% 51.7% Small CNN 94,538 58.4% 52.0% CNN's margin +7.0 pts +0.3 pts The baseline is a real fully-connected network, not a single linear layer — Flatten → Linear(3072, 512) → ReLU → Linear(512, 10) . It has the capacity to learn anything the CNN can; what it lacks is any reason to look at pixels near each other. Two things in that table are worth sitting with. The CNN wins the natural case with sixteen times fewer parameters — that's the prior paying for itself. And in the shuffled case it doesn't just lose its lead; it drops 6.4 points in absolute terms, down to roughly where the linear model already was. "You destroyed the data" — no, and this is the important part This is the objection everyone raises, so let's take it seriously, because the experiment is worthless if the objection holds. A fixed permutation is a bijection . Nothing is added, nothing is removed, nothing is averaged or blurred: img = torch . arange ( 3 * 32 * 32 , dtype = torch . float32 ). r

The AI leaderboard just had a seismic shift. Qwen3.8 Max, Alibaba's latest open-weight model, has been ranked as the best overall model by the Artificial Analysis Agentic Index — beating out GPT-5.6 Sol from OpenAI, Claude Opus 4.5 from Anthropic, and Gemini Ultra 2 from Google. This isn't just a benchmark win. It's the first time an open-source model has topped a comprehensive agentic intelligence index that measures real-world task performance, not just test scores. What Is the Agentic Index? The Artificial Analysis Agentic Index is an independent benchmark that evaluates AI models on their ability to complete agentic tasks — multi-step reasoning, tool use, code generation, and real-world problem solving. Unlike traditional benchmarks (MMLU, HumanEval) that test static knowledge, the agentic index measures whether a model can actually do things . The index evaluates models across multiple dimensions: Intelligence Index : Composite score across reasoning, coding, math, and instruction following Speed : Output tokens per second under production load Cost : Weighted average cost per intelligence task Endpoint Accuracy : Whether provider endpoints match reference model quality Qwen3.8 Max: The Specs Qwen3.8 Max represents Alibaba's most capable model to date: Parameters : 240B (MoE architecture, ~35B active during inference) Context : 256K tokens native, 1M extended Training : Trained through November 2025 data cutoff Licensing : Open weights for research and commercial use (with restrictions for users in restricted jurisdictions) What makes Qwen3.8 Max notable isn't just raw intelligence — it's the combination of high performance with competitive pricing and speed. The model scores near the top on intelligence while maintaining cost per task well below premium alternatives. Why This Matters for Developers 1. Open-Source is Catching Up — and Pulling Ahead For two years, the gap between open-source models (Llama, Qwen, Mistral) and proprietary frontier models (GPT, Cla

Cuando el objetivo es algo tan delicado como un diagnóstico asistido, confiar en un único modelo es arriesgado. FeliniAI usa tres pipelines complementarios que se refuerzan entre sí, igual que un veterinario combina lo que ve, lo que mide y lo que sabe. Pipeline 1 — Visión: MobileNetV2 Una CNN MobileNetV2 (PyTorch, transfer learning) clasifica imágenes de la piel/pelaje del gato en categorías visuales. Elegí MobileNetV2 por su equilibrio entre precisión y ligereza: corre rápido en CPU, lo que mantiene la inferencia por debajo de 1 segundo. Alcanza un 93,4% de accuracy visual . Pipeline 2 — Clínico: XGBoost + ICADA El núcleo del sistema es un clasificador XGBoost que trabaja sobre 33 features clínicas derivadas de los criterios ICADA (los criterios estandarizados de dermatitis atópica felina): estacionalidad, distribución de las lesiones, prurito, respuesta a tratamientos previos. Sobre un dataset de 8.000 casos , este módulo logra un F1 macro de 0.9675 en validación cruzada 5-fold. La búsqueda de hiperparámetros se hizo con Optuna y la explicabilidad con SHAP. Pipeline 3 — LLM: la síntesis Finalmente, un LLM ( Llama 3.3 70B vía Groq ) integra las salidas de los dos modelos anteriores y las traduce en una recomendación legible: qué tipo de alergia es más probable, con qué confianza y qué pasos sugerir. El LLM no diagnostica solo: orquesta y comunica lo que han calculado los modelos especializados. Por qué tres pipelines y no uno Porque cada uno cubre el punto ciego del otro. La visión capta lo que una foto muestra pero un cuestionario no; el modelo clínico capta el historial que una foto no puede mostrar; el LLM convierte ambos en algo accionable. Es un patrón de ensemble heterogéneo aplicado a datos de naturaleza distinta. Resultados F1 macro (clínico): 0.9675 , accuracy 0.9909. Accuracy visual: 93,4%. 4 tipos de alergia, 33 features clínicas, <1s de inferencia. Qué aprendí Que en dominios sensibles, la arquitectura correcta no es "el modelo más grande", sino varios

Beyond Size: The Three Pillars of Test-Time Scaling in Large Language Models The narrative of artificial intelligence for the last decade has been dominated by a single, powerful trend: scaling. From the early days of AlexNet to the massive clusters powering GPT-4, the formula seemed simple—more data and more parameters lead to better performance. This paradigm, famously codified as the "Scaling Laws," suggested that we could predict model improvements simply by looking at the amount of compute poured into the pre-training phase. However, as the industry pushes against the boundaries of available high-quality data and the physical limits of hardware, a new dimension of scaling is emerging. It isn't about how large the model is, but how long it "thinks" before it speaks. This shift toward "test-time scaling" marks a transition from static intelligence to dynamic reasoning. Instead of relying solely on the patterns learned during training, models are now being equipped with the computational budget to explore, verify, and refine their answers at the point of inference. While the concept was popularized by the release of models like OpenAI’s o1 series , the underlying mechanics remained somewhat opaque. A recent comprehensive study by Hariri et al. (2026), titled " Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibility ", provides a much-needed formal framework for understanding this new frontier. The Three Regimes of Inference Compute The core contribution of the Hariri et al. paper is the formalization of test-time scaling into three distinct structural regimes. Rather than treating all "extra compute" as a single scalar budget, the authors map how compute is allocated across the implicit prefix tree of an autoregressive model. 1. Single-Trajectory Sequential Scaling This is the most familiar regime, often associated with Chain-of-Thought (CoT) prompting. In this mode, the model generates a single sequence of tokens. Compute is scaled

Why Decouple Reasoning from Motor Control General-purpose robots have to pull off two very different jobs at once. They need to read a cluttered, full-room visual scene, hold a multi-minute plan in memory, and converse with a person — and, in the same instant, close a high-frequency control loop that keeps a balancing humanoid upright and moves a delicate hand without dropping whatever it holds. Cramming both jobs into a single end-to-end network forces uncomfortable trade-offs: the large context window you want for reasoning fights the low latency you need for torque control. On July 28, 2026, Google DeepMind pushed directly against that trade-off with Gemini Robotics 2 , followed on July 30 by Gemini Robotics ER 2. Rather than one monolithic network, the suite splits the problem across three specialized models — whole-body vision-language-action (VLA) control, high-level embodied reasoning, and on-device adaptation — each tuned to a different cadence and context size. The same modular thinking is visible across recent robotics and VLA research collected on the arXiv robotics listings and on Hugging Face Papers , where decomposed perception-planning-control stacks have become a recurring pattern. Understanding DeepMind's specific split clarifies why this architecture is gaining traction. The Three-Model Split ER 2: High-Level Task Reasoning Gemini Robotics ER 2 is the cognitive planner of the stack. It is a vision-language model built for embodied reasoning: it ingests the live camera feed and a natural-language instruction, then decomposes a task that may run several minutes into structured sub-goals. Beyond planning, ER 2 manages dialogue with a human supervisor, interprets spatial context, and coordinates multiple robots operating in a shared workspace — deciding which sub-task gets handed to which platform. Operating more slowly than the control layer (roughly a few times per second), ER 2 trades frequency for breadth of context. That separation matters: a reas

You loaded an image, got a tensor shaped (batch, height, width, channels) , and your convolution wants (batch, channels, height, width) . Stack Overflow says permute . Someone else says transpose . And reshape(2, 3, 28, 28) gives you the right shape too — so why is everyone making this complicated? Because two of those three are the same tool, and the third one silently destroys your data. The short answer transpose(dim0, dim1) swaps exactly two dimensions. permute(...) reorders all of them in one call, and you must list every dimension. transpose is a special case of permute . Both return a view — no data is copied, only the strides change — which also means both leave you with a non-contiguous tensor. reshape is not in this family at all. It reinterprets the flat memory under a new shape without moving anything, so it can produce the shape you asked for while completely scrambling what the numbers mean. import torch t = torch . arange ( 24 ). reshape ( 2 , 3 , 4 ) print ( t . transpose ( 0 , 1 ). shape ) # torch.Size([3, 2, 4]) — swapped dims 0 and 1 print ( t . permute ( 2 , 0 , 1 ). shape ) # torch.Size([4, 2, 3]) — full reorder transpose — swap two axes transpose(dim0, dim1) takes two dimension indices and swaps them. Everything else stays put. t = torch . arange ( 24 ). reshape ( 2 , 3 , 4 ) print ( t . shape ) # torch.Size([2, 3, 4]) print ( t . transpose ( 0 , 1 ). shape ) # torch.Size([3, 2, 4]) print ( t . transpose ( 1 , 2 ). shape ) # torch.Size([2, 4, 3]) The order of the two arguments doesn't matter — t.transpose(0, 1) and t.transpose(1, 0) are the same thing. A swap is a swap. On a 2-D tensor this is the matrix transpose you already know, and .T is the shorthand: m = torch . arange ( 6 ). reshape ( 2 , 3 ) print ( m . T . shape ) # torch.Size([3, 2]) print ( m . transpose ( 0 , 1 ). shape ) # torch.Size([3, 2]) — identical One caution on .T : on tensors with more than two dimensions, .T reverses every dimension, and modern PyTorch has deprecated that

How I applied Exploratory Data Analysis, Feature Engineering, Pipelines, and Ensemble Models to solve a real-world machine learning problem—and the lessons I learned along the way. Introduction There comes a point in every machine learning learner's journey when watching tutorials and completing small practice exercises are no longer enough. After spending weeks understanding statistics, exploratory data analysis (EDA), feature engineering, preprocessing techniques, and classical machine learning algorithms, I wanted to answer one question: Can I apply everything I've learned to a real machine learning competition? That's when I decided to participate in a Kaggle Playground competition. Unlike classroom datasets, Kaggle competitions force you to think like a machine learning engineer. You're responsible for understanding messy data, building preprocessing pipelines, selecting models, evaluating performance, debugging errors, and finally creating a submission that competes with thousands of participants. This article documents my complete journey—from loading the dataset to building production-style preprocessing pipelines and training multiple ensemble models. Along the way, I'll also share the challenges I faced, what worked well, and the lessons I'll carry into future competitions. Why Kaggle? Learning machine learning isn't just about knowing algorithms. Real-world ML requires answering questions like: Which features are useful? How should missing values be handled? Should categorical variables be one-hot encoded or ordinal encoded? Which preprocessing steps belong inside a pipeline? How do different ensemble models compare? Kaggle provides an environment where all of these questions matter. Instead of building a model that works only inside a notebook, you're solving a problem under realistic constraints and evaluating your solution on unseen data. Competition Goal The objective of this Playground competition was to predict the target class based on a combinatio

https://www.youtube.com/watch?v=sVeEc3H6bA4 权重即数据：神经网络权重空间学习如何成为 AI 的下一类训练集 以下位 TWIML AI Podcast 第 772 期《Why Models Are AI's Next Training Dataset》访谈转录整理，嘉宾为圣加仑大学 AI 与机器学习教授 Damian Borth，主持人 Sam Charrington。 介绍详细内容之前，先说说WSL是否等同于模型蒸馏？ 答案是不是一回事，但容易混着叫。先把两件事拆开，再对照 Borth 的"权重空间学习（WSL）"你就清楚了。 1. Anthropic 骂阿里那件事是什么 Anthropic 2026 年 6 月致信美国参议院，说阿里 Qwen 团队在 4/22–6/5 期间用近 2.5 万个假账号调 Claude 约 2880 万次 ，把 Claude 的回答当训练数据去训自己的模型，他们叫它" 蒸馏攻击（distillation attack） "。 这本质上是 黑盒/API 层的数据蒸馏 ： 教师=Claude（只看得到输出文本） 学生=Qwen 系模型 方法=拿 Claude 的生成文本（硬标签，最多再加点软标签）当语料去训学生 目的=迁移能力、省训练钱 注意：这跟"白盒蒸馏"还不一样，阿里（按指控）根本没拿到 Claude 的权重，拿到的是 对话文本 。行业里把"用强模型输出当训练数据"泛称为蒸馏，但严格学术定义里这只是黑盒 KD 或数据蒸馏。 2. Borth 的"权重当数据"是不是蒸馏 形式上沾边，本质上不同。 维度 经典/黑盒蒸馏（Anthropic 指控那种） Borth 权重空间学习（WSL） 学习对象 教师模型的 输出 （文本/软标签/中间激活） 一堆已训练模型的 权重本身 （参数张量） 数据形态 (x, 教师输出) 配对样本 把模型权重序列化、令牌化后的"权重语料" 目标 学生模仿教师行为，压缩模型 学"模型种群"的流形：预测准确率 / 生成新权重 / 跨架构采样 要不要原始数据 黑盒蒸馏可以完全不用原数据，只用教师输出 完全不用任何输入输出数据 ，连教师行为都不看 典型操作 用 Claude 回答训 Qwen 下载 HF 上 2000 个 CV 模型 → 自编码器压成隐空间 → 采样出遥感模型权重 Borth 自己在论文里也承认：WSL 可以看作" 直接在权重上做的、基于训练的知识复用 "，但它不需要像 KD 那样去跑原数据集拿激活、也不需要教师在线推理，它是把"训练好的模型集合"当成 第三种数据模态 （继文本、图像之后）。 简单说： 蒸馏是" 看菜谱做出来的菜（输出）来学做饭 " WSL 是" 把几百道做好的菜称重、切片、分析配料分布，然后直接捏出一道新菜的重量配方 "——连火都没开，更没尝过菜味。 3. 为什么大家会搞混 因为两者都叫"复用已有模型的知识"，而且 WSL 生成出的权重确实能当初始化、能跨域迁移（比如用 ImageNet 模型权重训出遥感模型，350 GPU 小时干掉 12000 GPU 小时的从头训）， 效果上像"蒸馏了前辈经验" 。但机制上： KD 的知识载体是 前向行为 （logits / 文本） WSL 的知识载体是 参数几何结构 （权重空间里的流形、对称性、轨迹） 所以 Borth 在访谈里特意说"权重不仅是学习的输出，也可以是学习的输入"——这句话的潜台词就是： 别把它归类成 KD，它是一个新模态的学习问题 。 4. 一句话收口 Anthropic 抱怨阿里，是"你偷用我家模型吐的字句当教材"；Borth 的路子是"我把全网开源模型（含你家的，只要开源许可允许）的 权重文件 当语料，训一个会造权重的元模型"——前者踩的是 API 条款和商业秘密红线，后者用的是 已发布权重 （Hugging Face 上大多有许可证），技术族谱上离"蒸馏"比离"神经架构搜索 + 超网络"更远。 第（一）部分 节目开场与研究总览：当训练数据枯竭，权重成为新燃料 （0% - 8%） 节目引入与核心命题 ：主持人点明当下 AI 领域最严峻的问题之一——高质量训练数据越来越难找，部分研究者押注合成数据，另一部分押注推理时计算（test-time reasoning）。而本期嘉宾 Damian Borth 提出了一条截然不同的路径：每一个训练好的模型都凝结了数千乃至数百万 GPU 小时"什么管用"的探索经验，这些权重不应只被视为训练过程的终点，而应成为下一次训练的 起点和数据本身 。 嘉宾背景与研究方向 ：Damian Borth 是瑞士圣加仑大学 AI 与机器学习教授。他的核心研究线索是"权重空间学习"（weight space learning / w

AI feels like it "suddenly" got smart in the last few years. It didn't. It's been evolving in distinct stages for over 70 years — each one building on the limits of the last. Here's the journey, broken down simply. Stage 1: Rule-Based AI (1950s-1980s) The earliest AI wasn't "intelligent" — it was a giant pile of if-else logic written by humans. How it worked: Programmers manually coded rules. "If symptom X and symptom Y, then diagnose Z." Chess engines, expert systems, early chatbots like ELIZA — all rule-based. The limit: These systems couldn't learn. Every scenario had to be explicitly programmed. Show it something outside its rules, and it broke. Stage 2: Machine Learning (1990s-2000s) Instead of hand-coding every rule, engineers started teaching systems to find patterns in data themselves. How it worked: Algorithms like decision trees, support vector machines, and linear regression learned relationships from labeled examples — spam vs. not spam, fraud vs. not fraud. The limit: These models needed carefully hand-engineered "features" (inputs) prepared by humans. They also struggled with messy, unstructured data like raw images or audio. Stage 3: Deep Learning (2010s) This is where things accelerated. Neural networks with many layers ("deep" networks) could learn features automatically from raw data, given enough compute and data. How it worked: Instead of a human deciding "look at edges, then shapes, then objects" in an image, the network learned that hierarchy itself. This powered breakthroughs in image recognition, speech-to-text, and translation. The limit: Deep learning was narrow. A model trained to recognize cats couldn't write an email. Each task needed its own model trained from scratch. Stage 4: Generative AI & LLMs (2018-Present) The current stage. Large Language Models like GPT and Claude are trained on massive amounts of text to predict "what comes next" — and in doing so, they pick up grammar, facts, reasoning patterns, and coding ability, all from o

LSTMs are still the go-to architecture for a lot of time series work, but they're annoying to trust. You get a number out the other end and no real sense of why the model landed there. This tutorial walks through training an LSTM on daily temperature data, then pulling it apart with three explainability methods: permutation importance, SHAP, and Integrated Gradients. Who this is for: people who already know some Keras and want to add interpretability to a forecasting model, not a from-scratch intro to neural nets. 1. Getting the data into shape LSTMs want a 3D tensor — (samples, timesteps, features) — so before anything else we need to turn a flat column of temperatures into overlapping 7-day windows, each one paired with the value on day 8. import numpy as np import pandas as pd from sklearn.preprocessing import MinMaxScaler # 1. Load data df = pd . read_csv ( " weather_data.csv " ) data = df [ ' Temperature ' ]. values . reshape ( - 1 , 1 ) # 2. Scale the data for stable neural network training scaler = MinMaxScaler ( feature_range = ( 0 , 1 )) scaled_data = scaler . fit_transform ( data ) # 3. Create sequences: 7 days of lag to predict the 8th day X , y = [], [] for i in range ( 7 , len ( scaled_data )): X . append ( scaled_data [ i - 7 : i ]) y . append ( scaled_data [ i ]) X , y = np . array ( X ), np . array ( y ) print ( f " Input shape: { X . shape } " ) # Output: (Samples, 7, 1) Scaling matters more than it sounds like it should — LSTMs trained on unscaled temperature values are prone to exploding gradients, and training just falls apart. The windowing step is really the whole trick here: every prediction only ever sees the past seven days, nothing more. 2. Building the model Two stacked LSTM layers, dropout after each one, early stopping so we don't have to babysit the epoch count. from tensorflow.keras.models import Sequential from tensorflow.keras.layers import LSTM , Dense , Dropout , Input from tensorflow.keras.callbacks import EarlyStopping # 1. Build

Problem Statement For roughly a decade, vision-language models have been declared to be approaching or matching human performance on scene description (captioning). The evidence for that claim has almost always come from the same family of benchmarks—most famously MS-COCO. Those images are typically clean, well-lit, and depict either no people or people performing simple, isolated actions (sitting, walking, holding an object). They rarely require the model to parse multi-agent social dynamics, subtle intentions, or the kind of relational reasoning humans perform effortlessly when watching a movie scene or a street interaction. Because the evaluation data are easy, the reported numbers look excellent. Automatic metrics such as BLEU-4, CIDEr, or even embedding-based scores like BERTScore further inflate the impression of progress: they reward surface lexical overlap more than genuine semantic fidelity. At the same time, almost no work has systematically catalogued which visual-cognitive failures models still commit, or how those failure modes have changed as architectures moved from CNN+LSTM captioners to today’s multimodal large language models (MLLMs). The result is a field that can claim “human-level performance” while remaining largely blind to whether the models actually understand the scenes that matter most in real applications—scenes full of people interacting. The authors therefore set out to answer two concrete questions that the existing literature left open: (1) How much of the apparent progress is an artifact of easy data? (2) Which specific error types have been eliminated and which stubbornly remain? Core Idea The core insight is that progress looks dramatically different once you force models to describe complex social behavior and once you measure not only overall accuracy but a taxonomy of visual-cognitive errors. By constructing a new 100-image Complex Social Behavior (CSB) dataset drawn from movie frames that require reasoning about multi-person in

You don't have a million labeled images or a GPU farm — and you don't need them. Transfer learning lets you stand on a model someone else trained and reach high accuracy with a few examples in minutes. Here's the idea, visualized. ♻️ Race scratch vs transfer: https://dev48v.infy.uk/dl/day17-transfer-learning.html The insight The early layers of a trained network learn general features — edges, textures, shapes — that are useful for almost any vision task. Only the last layers are task-specific. So why relearn edges from scratch? Two ways to do it Feature extraction: freeze the pretrained backbone, replace the final classifier with a small new "head," and train only the head on your data. Fast, needs little data. Fine-tuning: also unfreeze the top few backbone layers and train them at a low learning rate so you adapt without wrecking what they learned. The demo races two accuracy curves: "from scratch" crawls up and plateaus low (not enough data); "transfer learning" starts high and climbs fast. Tweak the example count and freeze/fine-tune to see them respond. Why it matters now This is exactly why fine-tuning an open LLM works: a foundation model already learned language; you adapt it cheaply. Transfer learning is what makes deep learning practical for the rest of us. 🔨 Full recipe (load pretrained → freeze → new head → train → optionally fine-tune low-LR) on the page: https://dev48v.infy.uk/dl/day17-transfer-learning.html Part of DeepLearningFromZero. 🌐 https://dev48v.infy.uk

A deep dive into the podcasts, papers, tweets, and tutorials of the engineer who made me add a fifth...

In the world of Generative AI, there is a massive difference between asking for a "pancake recipe" and asking for "eligibility criteria for phase III immunotherapy trials." In specialized fields like healthcare, a standard vector search often fails because medical terminology is dense, specific, and unforgiving. 🏥 Today, we are building a High-Precision Medical RAG (Retrieval-Augmented Generation) engine. We will move beyond simple semantic search by implementing Hybrid Search (Dense + Sparse vectors) using the powerhouse BGE-M3 model, storing it in Qdrant , and fine-tuning the results with FlashRank . This approach ensures that technical medical terms (like EGFR L858R mutation ) aren't lost in the "vibe" of a vector space. Keywords: Hybrid Search , Medical RAG , BGE-M3 Embeddings , Qdrant Vector Database , Clinical Trial Retrieval . The Architecture: Why Hybrid Search? Traditional RAG relies on "Dense Vectors" (semantic meaning). However, in clinical trials, keywords matter. A patient searching for "Pembrolizumab" needs that exact drug, not just "something related to cancer." By using BGE-M3 , we get the best of both worlds: Dense Retrieval : Captures the context and intent. Sparse Retrieval (Lexical) : Captures specific keywords and medical codes. Reranking : Re-evaluates the top hits to ensure the most clinically relevant document is on top. graph TD A[User Query: Medical Case] --> B{BGE-M3 Encoder} B -->|Dense Vector| C[Qdrant Collection] B -->|Sparse Vector| C C --> D[Hybrid Search Results] D --> E[FlashRank Reranker] E --> F[Top K Relevant Documents] F --> G[LLM: Final Synthesis] G --> H[Actionable Clinical Insight] Prerequisites 🛠️ Before we dive in, make sure you have your environment ready: Qdrant : Our high-performance vector database. BGE-M3 : A state-of-the-art embedding model that supports dense, sparse, and multi-vector retrieval. FlashRank : An ultra-fast, lightweight reranking library. LangChain : To orchestrate our RAG pipeline. pip install qdrant-c

Modern AI looks like magic from the outside. You type a sentence and a machine writes back something coherent, finishes your function, or turns a paragraph into Japanese. It's tempting to assume something exotic is happening in there. It isn't. The architecture behind almost every model you've heard of rests on a handful of plain engineering fixes, each one invented to get around a specific, annoying problem. No single genius moment, no secret sauce. Just people noticing their networks were broken and patching them. This is the story of three of those patches. If you can read a stack trace, you can follow all three. The wall everyone hit Around 2014, the recipe for a smarter neural network seemed obvious: make it deeper. More layers meant more capacity, which should have meant better results. Except past a certain point it stopped working. Deeper networks got worse , and not in the way you'd guess. The tell was the training error. A 56-layer network did worse on the very data it was being trained on than a 20-layer one. That rules out the usual suspect, overfitting, because the deep network couldn't even memorize the answers in front of it. The problem wasn't capacity. The network just couldn't be trained. Two things were going wrong. The error signal that teaches each layer (the gradient) has to travel backward through every layer to reach the early ones. Push a number through dozens of layers and it tends to either shrink to nothing or blow up, so the early layers got almost no usable feedback. And even when you wrestled the signal into shape, the optimization itself got harder the deeper you went. So depth, the thing that was supposed to make networks powerful, was the thing breaking them. Here's how three ideas knocked that wall down. Idea one: give the signal a shortcut The first fix is almost insultingly simple. Instead of forcing every layer to transform its input, you let the input skip ahead and get added back in later. Picture a block of layers that takes

Modern AI looks like magic from the outside. You type a sentence and a machine writes back something coherent, finishes your function, or turns a paragraph into Japanese. It's tempting to assume something exotic is happening in there. It isn't. The architecture behind almost every model you've heard of rests on a handful of plain engineering fixes, each one invented to get around a specific, annoying problem. No single genius moment, no secret sauce. Just people noticing their networks were broken and patching them. This is the story of three of those patches. If you can read a stack trace, you can follow all three. The wall everyone hit Around 2014, the recipe for a smarter neural network seemed obvious: make it deeper. More layers meant more capacity, which should have meant better results. Except past a certain point it stopped working. Deeper networks got worse , and not in the way you'd guess. The tell was the training error. A 56-layer network did worse on the very data it was being trained on than a 20-layer one. That rules out the usual suspect, overfitting, because the deep network couldn't even memorize the answers in front of it. The problem wasn't capacity. The network just couldn't be trained. Two things were going wrong. The error signal that teaches each layer (the gradient) has to travel backward through every layer to reach the early ones. Push a number through dozens of layers and it tends to either shrink to nothing or blow up, so the early layers got almost no usable feedback. And even when you wrestled the signal into shape, the optimization itself got harder the deeper you went. So depth, the thing that was supposed to make networks powerful, was the thing breaking them. Here's how three ideas knocked that wall down. Idea one: give the signal a shortcut The first fix is almost insultingly simple. Instead of forcing every layer to transform its input, you let the input skip ahead and get added back in later. Picture a block of layers that takes

Anthropic’s Fable/Mythos shutdown is the first real model export-control shock The important AI story this week is not just that Anthropic launched bigger Claude models. It is that the US government then told Anthropic to switch two of them off for foreign nationals — and Anthropic says the practical answer was to disable them for customers while it works through compliance. That is a very different kind of platform risk than rate limits or pricing changes. If you are building on frontier models, model access can now move because of export-control decisions, safety claims, and geopolitical pressure. What happened Anthropic announced Claude Fable 5 and Claude Mythos 5 on June 9. Fable 5 was described as Anthropic’s most capable generally available model, with stronger performance across software engineering, knowledge work, vision, scientific research, and longer complex tasks. Mythos 5 was positioned above that: an upgrade to Claude Mythos Preview, with Anthropic calling out cyber-defence and life-sciences use cases. Three days later, Anthropic published a blunt update: the US government had issued an export-control directive requiring Anthropic to suspend all access to Fable 5 and Mythos 5 by any foreign national, whether inside or outside the United States — including foreign-national Anthropic employees. Anthropic said the order arrived at 5:21pm ET on June 12, did not include detailed specifics, and that its understanding was that the government believed it had become aware of a jailbreaking method for Fable 5. Anthropic said access to other models was not affected, but the “net effect” was that it had to abruptly disable Fable 5 and Mythos 5 for customers to ensure compliance. Al Jazeera’s follow-up on June 19 frames the downstream effect clearly: allied countries and companies are now being forced to think harder about dependence on US frontier-model access. It also reports that Anthropic had granted roughly 200 institutions across 15 countries access to Claud

Pick the wrong loss function and your model optimises the wrong thing — perfectly. The loss is the single number training tries to shrink, so it quietly defines what "wrong" even means. I built an interactive visualiser of MSE, MAE, and cross-entropy so you can see why the choice matters. 🎯 Drag the prediction: https://dev48v.infy.uk/dl/day6-loss-functions.html This is Day 6 of DeepLearningFromZero. Loss = one number for "how wrong" The network's output is compared to the truth and collapsed into one scalar. Everything in training exists to make that number smaller. Choose the loss and you've defined the network's entire goal. MSE — square the error (regression) const mse = ( pred , y ) => ( pred - y ) ** 2 ; Squaring means off-by-4 hurts 16×, off-by-1 hurts 1×. MSE obsesses over large errors — great when big misses are unacceptable, risky when outliers will drag the model around. MAE — absolute error, outlier-robust const mae = ( pred , y ) => Math . abs ( pred - y ); Linear penalty: off-by-4 hurts exactly 4× off-by-1. One wild outlier can't dominate. The trade-off is a constant gradient, so it can be slower and less precise near the answer. Cross-entropy — for classification When the output is a probability, you don't use MSE. Cross-entropy rewards confident-and-right and brutally punishes confident-and-wrong: const bce = ( p , y ) => - ( y * Math . log ( p ) + ( 1 - y ) * Math . log ( 1 - p )); Predict 1% for the true class and the loss screams toward infinity. In the demo, switch to Classification and slide p toward 0 to watch it explode. The slope is what learning actually uses Backprop doesn't follow the loss value — it follows the loss's gradient (slope) downhill. That's why the shape matters: cross-entropy's steep slope when very wrong gives a strong corrective push, helping classifiers learn faster than MSE would. grad = dLoss / dPred ; // gradient descent steps along this Choosing the loss is a design decision Predicting a price? MSE or MAE. Yes/no? Binary