本文的第一作者为硕士生王海保，题目为“Neural Encoding for Human Visual Cortex with Deep Neural Networks Learning “What” and “Where” ”，被IEEE Transactions on Cognitive and Developmental Systems 录用。

以下为文章摘要：

Neural encoding, a crucial aspect to understand human brain information processing system, aims to establish a quantitative relationship between the stimuli and the evoked brain activities. In the field of visual neuroscience, with the ability to explain how neurons in primary visual cortex work, population receptive field (pRF) models have enjoyed high popularity and made reliable progress in recent years. However, existing models rely on either the inflexible prior assumptions about pRF or the clumsy parameter estimation methods, severely limiting the expressiveness and interpretability. In this paper, we propose a novel neural encoding framework by learning “what” and “where” with deep neural networks. It involves two separate aspects: the spatial characteristic (“where”) and feature selection (“what”) of neuron populations in visual cortex. Specifically, our approach first encodes visual stimuli into hierarchically intermediate features through a pretrained deep neural network (DNN), then converts DNN features into refined features with the channel attention and spatial receptive field to learn “where”, and finally regresses refined features simultaneously onto voxel activities to learn “what”. The sparsity regularization and smoothness regularization are adopted in our modeling approach, so that the crucial receptive field can be estimated automatically without prior assumptions about shapes. Furthermore, an attempt is made to extend the voxel-wise modeling approach to multi-voxel joint encoding models, and we show that it is conducive to res- cuing voxels with poor signal-to-noise characteristics. Extensive empirical results demonstrate that the method developed herein provides an effective strategy to establish neural encoding for human visual cortex, with the weaker prior constraints but the higher encoding performance.

中文摘要参考：

神经编码模型旨在通过建立刺激和诱发脑活动之间的定量关系，揭示大脑如何处理感知信息的过程。在视觉神经科学领域，许多研究致力于建立视觉皮层的神经编码模型，并证明群体感受野 (population Receptive Field, pRF) 模型可以用来解释视觉皮层神经元是如何工作的。然而，这些模型要么依赖于对 pRF 空间特性的强先验假设，要么依赖于需要人工调整的笨拙的参数估计方法。因此，目前的神经编码模型在脑活动神经编码方面表现并不理想。针对这些问题，本文提出了一个新的 “what” 和 “where” 深度神经编码模型。该模型从视觉皮层神经元群体的空间特性 (“where”) 和特征选择特性 (“what”) 两个方面进行研究，通过三阶段的特征细化实现视觉神经信息的有效编码。它首先利用深度神经网络从自然图像中提取层次化的特征。然后，通过通道注意力机制和空间感受野对原始特征进行细化，最后同时回归到体素活动上。同时，我们还尝试将该模型扩展到多体素联合编码模型，提供了一种提高低信噪比体素的编码效果的可能方式。在公开的 fMRI 数据集 (Vim-1) 上的大量实验结果表明，本文提出的方法提供了一种建立人类视觉皮层神经编码的有效策略，具有较弱的先验约束和较高的编码性能。

文章链接：https://ieeexplore.ieee.org/document/9134376