标签：#diffusion

If you approach generative models as ""networks that create images,"" the field quickly turns into a collection of disconnected architectures. A more useful developer mental model starts one level lower: What probability structure could have produced the data, and how can we represent, learn, and infer that structure without making the computation impossible? That question connects autoregressive models, VAEs, flow-based models, GANs, and diffusion models. Their architectures look very different, but they all respond to the same underlying tension: high-dimensional data distributions are difficult to represent, learn, normalize, sample from, and reason about. Generative modeling can therefore be organized around three interacting problems: Representation: How do we represent a complex high-dimensional joint distribution? Learning: How do we make the model distribution approach the data distribution? Inference: Given an observation, how do we reason backward about hidden variables or the process that generated it? Once these three pieces are connected, the major families of deep generative models become much easier to understand. From prediction to distribution learning A discriminative model usually begins with a prediction problem. Given an input x , predict the most likely output y : f ( x ) = y ar g max ​ p ( y ∣ x ) The model focuses directly on the conditional relationship required for prediction. A generative model asks a broader question. Instead of learning only the path from x to y , it models the probability structure from which the data arises. For class-conditional modeling, for example, we can model p ( x ∣ y ) together with the prior p ( y ) and recover the posterior using Bayes' rule: p ( y ∣ x ) = p ( x ) p ( x ∣ y ) p ( y ) ​ In unsupervised generative modeling, the target becomes the data distribution itself. We assume the training samples come from some unknown distribution: x 1 ​ , x 2 ​ , … , x N ​ ∼ i.i.d. p data ​ ( x ) The model then construc

The company's new fundraising total now stands at $232 million.

Turning entire novels into narrated, lip-synced, motion video — locally, on a single 16GB GPU, with FLUX, Wan2.2, PuLID, MuseTalk, and ComfyUI. There's an old line: a picture is worth a thousand words. I'd extend it — a video is worth a thousand pictures. So I spent the last few months testing that idea the hard way: I built a pipeline that turns an entire novel into a narrated, lip-synced, motion video. Feed it Pride and Prejudice , or a 400-year-old tale from Strange Tales from a Chinese Studio (聊斋志异), and out comes a finished film with consistent character faces across a hundred-plus scenes. One person. One RTX 4060Ti with 16GB of VRAM. Everything local, no cloud inference. I called it iTube. And honestly — the videos came out better than I expected. Watching a novel you know become a moving, voiced, paced thing is a genuinely different experience from reading a summary of it. That's the bet: that video is a better medium for conveying a story than text is, and the finished clips have mostly convinced me the bet was right. See for yourself before reading another word about how it works: 🎬 Pride and Prejudice — a Western public-domain classic 🎬 聊斋志异 / Strange Tales from a Chinese Studio — a playlist of classical Chinese tales But this post isn't a demo reel. It's about the part nobody warns you about — the part between the models — because that's where the real work turned out to be. The models are instruments, not magic boxes Here's the single most important thing I learned, and the thing I'd want any collaborator to understand before touching a pipeline like this: Each AI package can do exactly one thing well, and your entire job is knowing precisely where each one's ability ends — then writing your script logic to route around the gaps. The stack is not exotic. Most people in generative media know these pieces: FLUX generates the still image for a scene. That's all it does — a single frame. Beautiful, controllable with the right prompt, but frozen. Wan2.2 takes

Un-0 is an image-generation system tool that shows for the first time how the company's technology can replicate conventional AI systems.

⚠️ この記事はアフィリエイト広告（プロモーション）を含みます。リンク先で発生した収益の一部が運営者に支払われますが、読者の購入価格には一切影響ありません。 By the end of this article you'll have two runnable Python scripts: a CLIP-based pre-filter that re-checks SDXL Turbo output before it ever hits Pinterest, and a prompt sanitizer that strips artist names + trademarked characters so you don't eat a DMCA. I ran this pipeline for 41 days, generated 6,180 images, and went from a 9.7% Pinterest rejection rate down to 2.6%. Here's exactly what broke and what fixed it. Why SDXL Turbo (1-step, ~0.3s on a 4090) beats SD 1.5 for Pinterest volume First, the conclusion: if you're mass-producing pins, SDXL Turbo's single-step guidance_scale=0.0 generation is the only thing that makes the unit economics work. On my RTX 4090 I clock 0.31s per 512x512 image with Turbo vs 4.8s for a 30-step SDXL base run. That's 15x. Over 6,180 images that's the difference between 32 minutes and 8.2 hours of GPU time. But Turbo has a nasty side effect nobody warns you about: because it's distilled and runs at low resolution by default, its built-in StableDiffusionXLPipeline safety checker (when enabled) throws far more false positives on perfectly benign images — beaches, lingerie-free fashion flatlays, even close-up food. In my first 600-image batch, 58 images came back as black squares from the NSFW checker. 51 of them were photos of latte art and knitted sweaters . So I ripped out the default checker and built my own two-stage gate. Stage 1: Replacing the diffusers safety_checker with a tunable CLIP gate in Python The default safety_checker in diffusers is a binary black box — you get a black image and zero signal about why . For a production loop you need a confidence score so you can set your own threshold. I use OpenCLIP's ViT-B-32 to score each output against a small set of NSFW concept prompts, then compare to a safe-concept baseline. This code actually runs (tested on diffusers==0.27.2 , open_clip_torch==2.24.0 ): import torch import open_clip from PIL import Ima