标签：#finetuning

Take control of your AI models with our newest skill, designed to make local fine-tuning efficient.

I fine-tuned Gemma 4 E2B. The adapter merged cleanly. The export to .litertlm completed without errors. I pushed the model to my phone, initialized the engine, and everything looked green. Then I tried to create a conversation and got this: Failed to apply template: unknown method: map has no method named get (in template:238) No model loading failure. No quantization error. The model initialized, the tokenizer loaded, and then the runtime choked on a Jinja template feature it does not support. This failure only surfaces when you actually try to run inference, not when you load the model. If you are demoing at a hackathon, this is the worst possible time to discover a compatibility issue. I hit this exact bug while building Redacto, a zero-trust PII redaction app that runs Gemma 4 E2B entirely on-device. This post walks through the full fine-tune-to-deploy pipeline: how to QLoRA a model on Colab, export it for LiteRT-LM, and avoid the undocumented template trap that will block your deployment. The Full Pipeline Here is what the fine-tune-to-deploy pipeline looks like end to end: HuggingFace base weights -> QLoRA fine-tune (Colab) -> Merge adapter into base -> Patch chat template <-- the step nobody tells you about -> Quantize + export to .litertlm -> Push to device Each stage has its own failure modes. The template patch step is the one that was undocumented at the time, and it is the one that will cost you hours if you do not know it exists. A note on framing before we dig in: this was an under-resourced fine-tune. I trained on 3,000 of the 400,000 samples in the ai4privacy/pii-masking-400k dataset for a single epoch, and the label format did not fully match what Redacto expected downstream. The point of this post is not the fine-tune's accuracy - it is the deployment mechanics I had to work through to get any fine-tuned model onto the device at all. Step 1: QLoRA Fine-Tuning on Colab QLoRA (Quantized Low-Rank Adaptation) lets you fine-tune a quantized model by tra

There’s something uniquely beautiful about old books. The smell of weathered paper, the texture of...