新闻动态

成果发表于2023年5月10日《科学 机器人》(Science Robotics)

位置识别是机器人理解并导航世界的一种基本空间智能能力。然而，由于资源限制和环境变化，机器人在自然环境中识别地点仍然是一项具有挑战性的任务。相比之下，人类和动物能够在不同条件下稳健且高效地识别数十万个地点。在这里，我们报告了一个受大脑启发的通用地点识别系统，称为NeuroGPR，它通过模仿多模态感知、编码和计算的神经机制，使机器人能够在空间和时间的连续体中识别地点。我们的系统包括一个多模态混合神经网络（MHNN），它从传统和神经形态传感器中编码和整合多模态线索。具体来说，为了编码不同的感官线索，我们构建了各种空间视图细胞、地点细胞、头部方向细胞和时间细胞的神经网络。为了整合这些线索，我们设计了一个多尺度液态状态机，它能够使用不同的神经动态和受生物启发的抑制电路有效地异步处理和融合多模态信息。我们将MHNN部署在Tianjic上，Tianjic是一种混合神经形态芯片，并将其实现在四足机器人上。我们的结果表明，与传统和现有的生物启发方法相比，NeuroGPR表现出更好的性能，展现出对多样化环境不确定性的鲁棒性，包括感知歧义、运动模糊、光线或天气变化。在Tianjic上作为整体多神经网络工作负载运行NeuroGPR，展示了其优势，与常用的移动机器人处理器Jetson Xavier NX相比，延迟降低了10.5倍，功耗降低了43.6%。

Brain-inspired multimodal hybrid neural network for robot place recognition

Fangwen Yu, Yujie Wu, Songchen Ma, Mingkun Xu, Hongyi Li, Huanyu Qu,Chenhang Song, Taoyi Wang, Rong Zhao, LupingShi

Abstract：Place recognition is an essential spatial intelligence capability for robots to understand and navigate the world. However, recognizing places in natural environments remains a challenging task for robots because of resource limitations and changing environments. In contrast, humans and animals can robustly and efficiently recognize hundreds of thousands of places in different conditions. Here, we report a brain-inspired general place recognition system, dubbed NeuroGPR, that enables robots to recognize places by mimicking the neural mechanism of multimodal sensing, encoding, and computing through a continuum of space and time. Our system consists of a multimodal hybrid neural network (MHNN) that encodes and integrates multimodal cues from both conventional and neuromorphic sensors. Specifically, to encode different sensory cues, we built various neural networks of spatial view cells, place cells, head direction cells, and time cells. To integrate these cues, we designed a multiscale liquid state machine that can process and fuse multimodal information effectively and asynchronously using diverse neuronal dynamics and bioinspired inhibitory circuits. We deployed the MHNN on Tianjic, a hybrid neuromorphic chip, and integrated it into a quadruped robot. Our results show that NeuroGPR achieves better performance compared with conventional and existing biologically inspired approaches, exhibiting robustness to diverse environmental uncertainty, including perceptual aliasing, motion blur, light, or weather changes. Running NeuroGPR as an overall multi–neural network workload on Tianjic showcases its advantages with 10.5 times lower latency and 43.6% lower power consumption than the commonly used mobile robot processor