La fondation de TOUTE l'IA était CASSÉE... PERSONNE ne l'avait vu.
This video discusses a significant breakthrough in AI architecture, potentially revolutionizing how AI models are built. The core of this innovation comes from the team behind Kimi, an AI model developed by Moonshot AI. This research challenges a foundational element of AI design that has remained unchanged since 2015.
For years, AI models have been built upon a layered structure, where each layer processes information progressively, moving from simple word recognition to complex reasoning. This layered approach, however, faced a fundamental limitation: the "vanishing gradient problem." As signals representing learning corrections traveled from the deepest layers back to the initial ones, they weakened significantly over many layers, making it difficult for early layers to learn effectively. This was akin to shouting a message down a long corridor, where only a whisper is heard at the end.
