I compiled Doom's renderer into a 21B-parameter transformer -- no training anywhere [P]

This is the project my last two posts were building towards (this is the last of this silliness). I ported the Doom rendering algorithm to run inside a transformer. Instead of training a model, I used a compiler I wrote which converts computation graphs into transformer weights, and then ported Doom's algorithm into a compatible graph. The generated checkpoints can be loaded in Hugging Face without trust_remote_code -- it's just a standard transformers checkpoint. You feed the model a prompt representing the scene data, and generate until the model stops. The result is a token sequence which includes simple pixel drawing commands (to move the cursor, draw a pixel, etc). When you mechanically apply those drawing commands you get the rendered frame. The article includes the entire host program necessary to load the checkpoint, generate the render, and parse the output into the famous E1M1 frame. This host code is 43 lines of python. The python to define the computation graph is much longer, but that gets compiled into the transformer itself. One frame is a 3,614-token prompt plus 53,747 generated tokens -- just over 40 minutes on a B200. The original Doom could achieve 35 FPS on a 486. This achieves 35 FPD (frames per day) on a B200. Write-up: https://ood.dev/posts/doom/ Weights: https://huggingface.co/physicsrob/torchwright-doom-e1m1 Github for the source code which gets compiled: https://github.com/physicsrob/torchwright_doom/ submitted by /u/notforrob [link] [留言]