distilled
Welcome to this week's Distilled! We're diving into two papers that tackle very different but equally critical problems in AI development. First up: a surprisingly tiny fraction of training tokens can derail your entire LLM, and second, we're looking at how GLM-5 is pushing
distilled
Why Your LLM Goes Crazy During Training (And How 0.01% of Tokens Are to Blame) TL;DR — When training language models with reinforcement learning, about 1 in 10,000 tokens get wildly oversized updates that destabilize everything. Masking just these troublemakers fixes the problem. What It Is Researchers at
distilled
GLM-5: Teaching AI to Actually Build Software, Not Just Write Code Snippets TL;DR — GLM-5 shifts from helping you write code to actually acting as a software engineer that can handle complex, multi-hour development tasks autonomously. What It Is The team at Zhipu AI built GLM-5 to solve a problem