Can AI See What We Dream? An MIT Student's Quest into Brain Decoding | Kelly Zhang | TEDxBoston

The speaker, Kelly Zang, was inspired by her grandfather, who taught her to question and solve problems, viewing the world as a puzzle. His struggle with Alzheimer's, which left him silent and unresponsive, sparked her deep desire to understand what he was experiencing internally, asking, "Are you still there?" This led her to challenge the assumption that a silent voice means a dark mind, proposing that we might just be listening at the wrong frequency. A junior at MIT studying computer science and finance, Kelly developed a novel approach to translate brain signals from fMRI scans into a real-time video stream of a person's mental imagery. She realized the human mind's "black box" was a data problem, previously unsolved due to specialists being siloed in their disciplines. As a "hybrid" student, she connected biology, finance, and machine learning, observing that the brain's visual cortex processes information hierarchically, similar to a deep neural network. Her high-level architecture begins with raw brain activity, passed through a vision transformer to identify core patterns. A latent representation acts as a bridge, translating these patterns into a mathematical concept. A latent diffusion model then sketches this concept into a high-resolution image, using real-world knowledge to fill in details. Finally, stable diffusion animates these images into a fluid, coherent video stream of the dream. This reveals that imagination is biological, as imagining something fires similar neural patterns to actually seeing it. This technology has profound implications beyond recording dreams, offering a way for PTSD veterans to show their nightmares, stroke victims to communicate, and Alzheimer's patients to share their internal world. Kelly emphasizes that this innovation stemmed from daring to connect disparate fields, not from being a specialist. She encourages everyone to "build a bridge" between disciplines, fostering curiosity and interdisciplinary collaboration to unlock complex problems.