标签：#torch

Google is introducing HEIR (Homomorphic Encryption Intermediate Representation), an open-source compiler and development toolchain designed to make encrypted computation easier to deploy. In particular, HEIR can compile pre-trained AI models built for conventional, unencrypted inputs so they can instead operate on encrypted data. By Sergio De Simone

Every LLM-powered app I'd built up to this point followed the same recipe (pun intended): call an API, write a good prompt, wrap it in a nice UI. That's a legitimate way to build things, but at some point I wanted to actually understand what was happening inside the model I was calling and not just how to prompt one. So for my recipe app Rasaveda , I decided to skip the API entirely. Intially, I had one made, but then I felt like I was not making any clear progress in actual machine building. So I ditched the entire external API callings. No OpenAI, no HuggingFace inference endpoint, no pretrained weights. I wrote a decoder-only transformer from scratch in PyTorch, trained it on a single Colab T4, and shipped it as the actual language model powering the app in production. This post is a lazy attempt at what that looked like. The architecture, the training runs, the mistakes, and what I'd tell someone about to try the same thing (do at your own risk). What Rasaveda actually does Rasaveda is a full-stack recipe intelligence app: you give it the ingredients sitting in your kitchen, it does a semantic vector search (ChromaDB + all-MiniLM-L6-v2 ) over 365 recipes to find the best matches, tells you exactly what you're missing, and can critique or explain any cooking step conversationally. It also has a somewhat unnecessary but delightful feature where you pick a theme by clicking one of 36 Indian states on a geographically accurate SVG map (original idea lol). The part I actually want to talk about is RasavedaGPT , the model that generates every word of AI output in the app, running in-process inside the FastAPI backend. Why build the model instead of calling one Two reasons, one practical and one selfish. The practical one: I wanted a fully self-contained, dependency-free inference path without any API keys, no rate limits, no per-token cost, no vendor to go down at 2am. For a small, domain-specific task like "reason about recipes," a giant general-purpose model is over

PyTrees look deceptively simple. You flatten a nested Python object into leaves, keep a structure descriptor, and later rebuild or map over the same shape. That abstraction is powerful enough to carry optimizer states, model parameters, batched inputs, gradients, and sharding annotations. It is also just ambiguous enough that three major frameworks implement three subtly different languages under the same idea. This note compares JAX jax.tree_util , PyTorch torch.utils._pytree , and TensorFlow tf.nest . I tested the behavior in two environments: an older stack with JAX 0.4.35, PyTorch 2.2.2, TensorFlow 2.20.0, and a newer stack with JAX 0.10.0, PyTorch 2.12.0, TensorFlow 2.21.0. Most flatten/unflatten semantics were stable across these versions. The main version-sensitive result is PyTorch: _pytree.tree_map in 2.2.2 accepts only one pytree, while 2.12.0 supports multiple pytrees and behaves much closer to JAX prefix-style mapping. The short version: JAX treats pytrees as a transformation language, PyTorch is converging toward that model in torch.func , and TensorFlow exposes a broader nested-structure utility through tf.nest . Those differences show up exactly where backend-agnostic libraries usually hurt: None , dictionary order, custom containers, tree_map , autodiff, and vectorization. The Shape Of The APIs The three APIs have the same surface story but not the same contract. from jax import tree_util as jtu from torch.utils import _pytree as tpu import tensorflow as tf leaves , treedef = jtu . tree_flatten ( tree ) tree = jtu . tree_unflatten ( treedef , leaves ) tree = jtu . tree_map ( f , * trees ) leaves , spec = tpu . tree_flatten ( tree ) tree = tpu . tree_unflatten ( leaves , spec ) tree = tpu . tree_map ( f , tree ) # PyTorch 2.2.2 tree = tpu . tree_map ( f , * trees ) # PyTorch 2.12.0 leaves = tf . nest . flatten ( tree ) tree = tf . nest . pack_sequence_as ( structure , leaves ) tree = tf . nest . map_structure ( f , * structures ) Flattening means "whi