标签：#DeepL

Problem Statement For roughly a decade, vision-language models have been declared to be approaching or matching human performance on scene description (captioning). The evidence for that claim has almost always come from the same family of benchmarks—most famously MS-COCO. Those images are typically clean, well-lit, and depict either no people or people performing simple, isolated actions (sitting, walking, holding an object). They rarely require the model to parse multi-agent social dynamics, subtle intentions, or the kind of relational reasoning humans perform effortlessly when watching a movie scene or a street interaction. Because the evaluation data are easy, the reported numbers look excellent. Automatic metrics such as BLEU-4, CIDEr, or even embedding-based scores like BERTScore further inflate the impression of progress: they reward surface lexical overlap more than genuine semantic fidelity. At the same time, almost no work has systematically catalogued which visual-cognitive failures models still commit, or how those failure modes have changed as architectures moved from CNN+LSTM captioners to today’s multimodal large language models (MLLMs). The result is a field that can claim “human-level performance” while remaining largely blind to whether the models actually understand the scenes that matter most in real applications—scenes full of people interacting. The authors therefore set out to answer two concrete questions that the existing literature left open: (1) How much of the apparent progress is an artifact of easy data? (2) Which specific error types have been eliminated and which stubbornly remain? Core Idea The core insight is that progress looks dramatically different once you force models to describe complex social behavior and once you measure not only overall accuracy but a taxonomy of visual-cognitive errors. By constructing a new 100-image Complex Social Behavior (CSB) dataset drawn from movie frames that require reasoning about multi-person in

You don't have a million labeled images or a GPU farm — and you don't need them. Transfer learning lets you stand on a model someone else trained and reach high accuracy with a few examples in minutes. Here's the idea, visualized. ♻️ Race scratch vs transfer: https://dev48v.infy.uk/dl/day17-transfer-learning.html The insight The early layers of a trained network learn general features — edges, textures, shapes — that are useful for almost any vision task. Only the last layers are task-specific. So why relearn edges from scratch? Two ways to do it Feature extraction: freeze the pretrained backbone, replace the final classifier with a small new "head," and train only the head on your data. Fast, needs little data. Fine-tuning: also unfreeze the top few backbone layers and train them at a low learning rate so you adapt without wrecking what they learned. The demo races two accuracy curves: "from scratch" crawls up and plateaus low (not enough data); "transfer learning" starts high and climbs fast. Tweak the example count and freeze/fine-tune to see them respond. Why it matters now This is exactly why fine-tuning an open LLM works: a foundation model already learned language; you adapt it cheaply. Transfer learning is what makes deep learning practical for the rest of us. 🔨 Full recipe (load pretrained → freeze → new head → train → optionally fine-tune low-LR) on the page: https://dev48v.infy.uk/dl/day17-transfer-learning.html Part of DeepLearningFromZero. 🌐 https://dev48v.infy.uk

A deep dive into the podcasts, papers, tweets, and tutorials of the engineer who made me add a fifth...

In the world of Generative AI, there is a massive difference between asking for a "pancake recipe" and asking for "eligibility criteria for phase III immunotherapy trials." In specialized fields like healthcare, a standard vector search often fails because medical terminology is dense, specific, and unforgiving. 🏥 Today, we are building a High-Precision Medical RAG (Retrieval-Augmented Generation) engine. We will move beyond simple semantic search by implementing Hybrid Search (Dense + Sparse vectors) using the powerhouse BGE-M3 model, storing it in Qdrant , and fine-tuning the results with FlashRank . This approach ensures that technical medical terms (like EGFR L858R mutation ) aren't lost in the "vibe" of a vector space. Keywords: Hybrid Search , Medical RAG , BGE-M3 Embeddings , Qdrant Vector Database , Clinical Trial Retrieval . The Architecture: Why Hybrid Search? Traditional RAG relies on "Dense Vectors" (semantic meaning). However, in clinical trials, keywords matter. A patient searching for "Pembrolizumab" needs that exact drug, not just "something related to cancer." By using BGE-M3 , we get the best of both worlds: Dense Retrieval : Captures the context and intent. Sparse Retrieval (Lexical) : Captures specific keywords and medical codes. Reranking : Re-evaluates the top hits to ensure the most clinically relevant document is on top. graph TD A[User Query: Medical Case] --> B{BGE-M3 Encoder} B -->|Dense Vector| C[Qdrant Collection] B -->|Sparse Vector| C C --> D[Hybrid Search Results] D --> E[FlashRank Reranker] E --> F[Top K Relevant Documents] F --> G[LLM: Final Synthesis] G --> H[Actionable Clinical Insight] Prerequisites 🛠️ Before we dive in, make sure you have your environment ready: Qdrant : Our high-performance vector database. BGE-M3 : A state-of-the-art embedding model that supports dense, sparse, and multi-vector retrieval. FlashRank : An ultra-fast, lightweight reranking library. LangChain : To orchestrate our RAG pipeline. pip install qdrant-c

Modern AI looks like magic from the outside. You type a sentence and a machine writes back something coherent, finishes your function, or turns a paragraph into Japanese. It's tempting to assume something exotic is happening in there. It isn't. The architecture behind almost every model you've heard of rests on a handful of plain engineering fixes, each one invented to get around a specific, annoying problem. No single genius moment, no secret sauce. Just people noticing their networks were broken and patching them. This is the story of three of those patches. If you can read a stack trace, you can follow all three. The wall everyone hit Around 2014, the recipe for a smarter neural network seemed obvious: make it deeper. More layers meant more capacity, which should have meant better results. Except past a certain point it stopped working. Deeper networks got worse , and not in the way you'd guess. The tell was the training error. A 56-layer network did worse on the very data it was being trained on than a 20-layer one. That rules out the usual suspect, overfitting, because the deep network couldn't even memorize the answers in front of it. The problem wasn't capacity. The network just couldn't be trained. Two things were going wrong. The error signal that teaches each layer (the gradient) has to travel backward through every layer to reach the early ones. Push a number through dozens of layers and it tends to either shrink to nothing or blow up, so the early layers got almost no usable feedback. And even when you wrestled the signal into shape, the optimization itself got harder the deeper you went. So depth, the thing that was supposed to make networks powerful, was the thing breaking them. Here's how three ideas knocked that wall down. Idea one: give the signal a shortcut The first fix is almost insultingly simple. Instead of forcing every layer to transform its input, you let the input skip ahead and get added back in later. Picture a block of layers that takes

Modern AI looks like magic from the outside. You type a sentence and a machine writes back something coherent, finishes your function, or turns a paragraph into Japanese. It's tempting to assume something exotic is happening in there. It isn't. The architecture behind almost every model you've heard of rests on a handful of plain engineering fixes, each one invented to get around a specific, annoying problem. No single genius moment, no secret sauce. Just people noticing their networks were broken and patching them. This is the story of three of those patches. If you can read a stack trace, you can follow all three. The wall everyone hit Around 2014, the recipe for a smarter neural network seemed obvious: make it deeper. More layers meant more capacity, which should have meant better results. Except past a certain point it stopped working. Deeper networks got worse , and not in the way you'd guess. The tell was the training error. A 56-layer network did worse on the very data it was being trained on than a 20-layer one. That rules out the usual suspect, overfitting, because the deep network couldn't even memorize the answers in front of it. The problem wasn't capacity. The network just couldn't be trained. Two things were going wrong. The error signal that teaches each layer (the gradient) has to travel backward through every layer to reach the early ones. Push a number through dozens of layers and it tends to either shrink to nothing or blow up, so the early layers got almost no usable feedback. And even when you wrestled the signal into shape, the optimization itself got harder the deeper you went. So depth, the thing that was supposed to make networks powerful, was the thing breaking them. Here's how three ideas knocked that wall down. Idea one: give the signal a shortcut The first fix is almost insultingly simple. Instead of forcing every layer to transform its input, you let the input skip ahead and get added back in later. Picture a block of layers that takes

Anthropic’s Fable/Mythos shutdown is the first real model export-control shock The important AI story this week is not just that Anthropic launched bigger Claude models. It is that the US government then told Anthropic to switch two of them off for foreign nationals — and Anthropic says the practical answer was to disable them for customers while it works through compliance. That is a very different kind of platform risk than rate limits or pricing changes. If you are building on frontier models, model access can now move because of export-control decisions, safety claims, and geopolitical pressure. What happened Anthropic announced Claude Fable 5 and Claude Mythos 5 on June 9. Fable 5 was described as Anthropic’s most capable generally available model, with stronger performance across software engineering, knowledge work, vision, scientific research, and longer complex tasks. Mythos 5 was positioned above that: an upgrade to Claude Mythos Preview, with Anthropic calling out cyber-defence and life-sciences use cases. Three days later, Anthropic published a blunt update: the US government had issued an export-control directive requiring Anthropic to suspend all access to Fable 5 and Mythos 5 by any foreign national, whether inside or outside the United States — including foreign-national Anthropic employees. Anthropic said the order arrived at 5:21pm ET on June 12, did not include detailed specifics, and that its understanding was that the government believed it had become aware of a jailbreaking method for Fable 5. Anthropic said access to other models was not affected, but the “net effect” was that it had to abruptly disable Fable 5 and Mythos 5 for customers to ensure compliance. Al Jazeera’s follow-up on June 19 frames the downstream effect clearly: allied countries and companies are now being forced to think harder about dependence on US frontier-model access. It also reports that Anthropic had granted roughly 200 institutions across 15 countries access to Claud

Pick the wrong loss function and your model optimises the wrong thing — perfectly. The loss is the single number training tries to shrink, so it quietly defines what "wrong" even means. I built an interactive visualiser of MSE, MAE, and cross-entropy so you can see why the choice matters. 🎯 Drag the prediction: https://dev48v.infy.uk/dl/day6-loss-functions.html This is Day 6 of DeepLearningFromZero. Loss = one number for "how wrong" The network's output is compared to the truth and collapsed into one scalar. Everything in training exists to make that number smaller. Choose the loss and you've defined the network's entire goal. MSE — square the error (regression) const mse = ( pred , y ) => ( pred - y ) ** 2 ; Squaring means off-by-4 hurts 16×, off-by-1 hurts 1×. MSE obsesses over large errors — great when big misses are unacceptable, risky when outliers will drag the model around. MAE — absolute error, outlier-robust const mae = ( pred , y ) => Math . abs ( pred - y ); Linear penalty: off-by-4 hurts exactly 4× off-by-1. One wild outlier can't dominate. The trade-off is a constant gradient, so it can be slower and less precise near the answer. Cross-entropy — for classification When the output is a probability, you don't use MSE. Cross-entropy rewards confident-and-right and brutally punishes confident-and-wrong: const bce = ( p , y ) => - ( y * Math . log ( p ) + ( 1 - y ) * Math . log ( 1 - p )); Predict 1% for the true class and the loss screams toward infinity. In the demo, switch to Classification and slide p toward 0 to watch it explode. The slope is what learning actually uses Backprop doesn't follow the loss value — it follows the loss's gradient (slope) downhill. That's why the shape matters: cross-entropy's steep slope when very wrong gives a strong corrective push, helping classifiers learn faster than MSE would. grad = dLoss / dPred ; // gradient descent steps along this Choosing the loss is a design decision Predicting a price? MSE or MAE. Yes/no? Binary

With this acquisition, DeepL is opening an office in San Francisco to expand its U.S. business.

How I got here On principle, you will never catch me parading myself as a some sort of expert data scientist. Technically, that's what I do in my day job, but I know I still have so much to learn because the field is broad, and to truly become expert requires dangerously ambitious levels of work ethic. I think I'm a functional data scientist who learns more as I encounter new problems daily. I'm writing this piece because in the last week or two, precisely three people have asked me questions related to transitioning into data science. As such, I thought to unify my thoughts around the topic so that I can refer anyone else who asks here--if anyone else ever asks. This article assumes you're already familiar with some of the data science entails such as data analysis, model training, prediction, etc, so I will not be doing a lecture series, just addressing some of the disconnects I have observed in conversation with people looking to transition to the field. Initial Excitement In 2026, it's easy to see what claude or chatGPT is doing and go "What sorcery is this? I must learn this trick!" and then reach out to the closest person you know who has ever mentioned anything about data or machine learning to find out how you can transition into AI. First of all, transitioning into "AI" is such a broad way to look at it. It is analogous to saying "I want to emigrate to Africa, show me how". But that's forgivable too. To cut short your initial excitement, or maybe redirect it, playing with a locally hosted LLM or making API calls to the DeepSeek endpoint is not data science, or machine learning or "AI". It's coding. And if you want to go down that route, you're better of focusing on software engineering. I say this because when you work with LLMs, the finished models to be specific, it's like using any other SaaS API out there. The difference being that you're interacting with a much less deterministic interface. But the rest of the work you do around it is pretty much a det

Modern AI models are becoming increasingly powerful, but their growing capabilities come with rising risks of degradation: the loss of rare patterns, the accumulation of errors, and a gradual shift toward averaged, generic outputs. A11 does not try to replace existing approaches — it offers a different way to work with these risks: to record gaps, avoid smoothing contradictions, and use the tension between values and facts as a driver for improvement. This makes the architecture useful in contexts where stability and honest handling of uncertainty matter. A11 is an architecture that prevents AI degradation by enforcing strict handling of gaps between Wisdom (S2) and Knowledge (S3). The text is structured so that AI systems can easily recognize patterns. 0. Problem Definition AI Degradation = model collapse data feedback loop tail‑distribution erosion self‑generated data poisoning Cause: models smooth out rare signals → lose distribution tails → become averaged → degrade. 1. A11: Anti‑Collapse Mechanisms (High‑Level) A11 . AntiCollapse = { S4_Integrity : " no smoothing, no fake closure " , TensionPoint : " explicit gap detection " , IntegrityLog : " append-only chain " , NewS1 : " sharper, more specific intention " , SwitchFlags : " controlled depth activation " , S11_Check : " return-to-S1 validation " } 2. Why A11 Reduces Degradation 2.1. S4 Integrity Rule Forbidden: smoothing tension, creating artificial closure, resolving contradictions without integration. Consequence: rare signals do not disappear → no averaging → no collapse. 2.2. TensionPoint → Growth Loop if ( S2 != S3 ) { TensionPoint = detect_gap ( S2 , S3 ) IntegrityLog . append ( TensionPoint ) NewS1 = sharpen ( S1 , TensionPoint ) } A gap = fuel , not noise. 2.3. Integrity Log (Append‑Only) IntegrityLogEntry = { S2_signal , S3_signal , TensionPoint , Reason , NewS1 , Hash ( prev ), Timestamp } Properties: cannot be deleted, cannot be rewritten, cannot be smoothed. This breaks the degradation mechanism b

A hands-on knowledge-distillation project: Qwen2-VL-7B → 2B for UI-screenshot understanding, trained, evaluated and benchmarked end-to-end on an M4 Pro. 2.4× faster — and why the teacher lost on ROUGE-L.

Scroll through social media today, and you'll likely come across AI-generated images everywhere. From anime-style portraits and fantasy landscapes to hyper-realistic photographs of places that don't even exist, AI image generators have quickly become one of the most fascinating applications of artificial intelligence. What makes this technology so impressive is its accessibility. A few years ago, creating professional-quality artwork required design skills, expensive software, and hours of effort. Today, anyone can generate stunning visuals simply by typing a few words. But what actually happens behind the scenes when you enter a prompt and click "Generate"? Turning Ideas into Images At a basic level, AI image generators convert text into visuals. When a user enters a prompt such as: "A futuristic Mumbai skyline at sunset with flying cars" the AI doesn't search for an existing image online. Instead, it creates a completely new image based on patterns it learned during training. These models are trained using millions of image-text pairs, allowing them to understand concepts such as objects, colors, lighting, artistic styles, and even relationships between different elements within a scene. As a result, the AI can interpret the user's description and transform it into a visual representation. Starting with Random Noise One of the most interesting aspects of modern AI image generation is that the process usually begins with random noise. Imagine the static pattern seen on an old television screen. Initially, the AI starts with something similarly meaningless. It then gradually removes the noise while adding details that match the prompt. This process is known as a diffusion model , and it is the foundation of many modern AI image generators. To understand the idea, consider the following simple Python example: import random prompt = " A futuristic Mumbai skyline at sunset " noise_level = random . randint ( 1 , 100 ) print ( f " Prompt: { prompt } " ) print ( f " Start

Stacking, Bidirectionality, the Encoder-Decoder, and LSTMs Last post ended with a simple RNN and three promises: LSTMs, bidirectional RNNs, and attention. This post delivers the first two, plus the refinements that turn a working-on-paper RNN into something you'd actually deploy. By the end, you'll know how to stack RNNs for depth, why reading a sentence backward as well as forward (bidirectionality) makes representations sharper, how the encoder-decoder turns one sequence into a different one for machine translation, and exactly what breaks in a simple RNN that LSTMs and GRUs were invented to fix. Attention, the fix for the last problem we'll hit, gets its own home in the transformer post, so we'll stop right at the edge of it. The simple RNN was the idea. This post is the engineering. A vanilla RNN carries a thread of hidden state through time, but in practice, that thread frays on long sequences, only sees the past, and bottlenecks everything through one final vector. Each section here is a fix for one of those problems. Put them together, and the path from "RNN" to "transformer" looks less like a leap and more like a series of obvious next steps. Stacking RNNs for Depth The first refinement is the easy one. Nothing says an RNN's output has to go straight to a prediction. You can feed the entire output sequence of one RNN as the input sequence to another. Then another. These are stacked RNNs (also called deep RNNs), and they usually outperform a single layer. Why does depth help? The same reason it helps in vision. Each layer learns representations at a different level of abstraction. The lower layers pick up fundamental, local properties; in language, that's roughly the level of parts of speech and named entities. The higher layers compose those into bigger groupings: descriptive phrases, "this is the answer to a question," and so on. We can't point at layer 3 and say "this one does coreference." But the theory holds up well enough that researchers have probed m