Evaluating Kimi 2.5 vs Kimi 2.6: What happens to agent skills when the model gets smarter?

When a stronger model ships, there are two questions every skill author should want answered, and evals are the only honest way to answer either: Which skills just got absorbed? A model that now knows how to do X natively does not need a skill telling it to do X. Fewer skills to maintain, leaner context, lower cost. Which skills still matter? Behaviour-level guidance (conventions, preferences, project-specific workflows) is not something pretraining will fill in for you. Those skills should keep paying. Moonshot gave us early access to Kimi K2.6. We ran the Tessl agent skill evaluation harness on the same 21 skills and 100 paired scenarios against three solvers: Kimi K2.5, Kimi K2.6, and Claude Sonnet 4.5. A solver is the model whose output the grader scores; a paired scenario is the same task run twice per solver, once without the skill installed and once with it. These are early signals from one pre-release on one skill set. A deeper cross-model analysis with clean baselines across the board is in progress and will be its own piece. What does our setup look like? Scenarios and rubrics are held fixed across the two Moonshot runs. The only variable is the solver. Solver A: Kimi K2.5 Solver B: Kimi K2.6 Scenario generator: Claude Sonnet 4.5, up to 5 scenarios per skill, derived from each skill's SKILL.md Grader: Claude Sonnet 4.5, weighted-checklist rubric derived from the same SKILL.md Per skill × per solver: every scenario solved twice, baseline (no skill installed) and with-skill Per-skill n=5 is noisy; the aggregate over 100 scenarios is where the signal lives. Three findings: Kimi 2.6 is a better model than K2.5: Without skills, K2.6 sits ~2 pp (percentage points) above K2.5 in aggregate, with double-digit moves on specific skills. Kimi 2.6 holds its own against Sonnet 4.5. We picked Sonnet 4.5 as a competitive baseline, and found in this evaluation set that the K2.6 performed better both in the with/without skill scenario by around ~8 p.p . Skills remain a dura