Summary: The authors present a novel method for reconstructing visual perception from brain activity using only 1 hour of fMRI training data, a significant reduction from previous methods. By pretraining the model across 7 subjects and then fine-tuning on minimal data from a new subject, they achieve high-quality reconstructions. Their functional alignment procedure maps brain data to a shared latent space, improving out-of-subject generalization. The approach also excels in image retrieval and reconstruction metrics compared to single-subject methods, showcasing the potential for accurate reconstructions with limited training data. MindEye2 demonstrates the feasibility of achieving precise reconstructions from a single MRI visit.
https://arxiv.org/abs/2403.11207