| 摘 要: 目前,大多数注意力机制是通过一系列不同的卷积等操作来生成权重的,而少数注意力机制则基于规律自定义函数来推断权重,两者各有优劣。在无参注意力机制(SimAM)的理论框架下,考虑到优化能量函数在某些情况下的局限性,提出通过在权重中引入少量参数来进行权重调整,从而提高模块的适应性。为了验证这一方法的可行性,在CIFAR数据集和ImageNet-100数据集上进行了图像识别实验。实验结果表明,通过在有偏好的无参注意力中引入少量参数,模型在CIFAR数据集上获得的 Top-1Acc从94.024%提升到94.124%,而在ImageNet-100数据集上获得的Top-1Acc从86.77%提升到86.89%。同时,GradCAM++可视化图直观地展示了该方法对不同类型图像中的目标关注范围更准确且适应性更强。 |
| 关键词: 卷积神经网络;无参注意力机制;有参注意力机制 |
|
中图分类号: TP181
文献标识码: A
|
|
| Adaptive Preference Attention |
|
LIU Gaole1, LAN Qiang2, TONG Qingshan1
|
(1.School of Mathematics and Statistics, Changsha University of Science and Technology, Changsha 410000, China;
2.National University of Defense Technology, Changsha 410073, China)
839628590@qq.com; lanqiang@nudt.edu.cn; tongqs@csust.edu.cn
|
| Abstract: Currently, most attention mechanisms generate weights through various convolutional operations, while a few attention mechanisms infer weights based on rule-defined functions, each with its own advantages and disadvantages. Under the theoretical framework of paramete-r free attention mechanisms (SimAM), considering the limitations of optimizing energy functions in certain scenarios, this paper proposes adjusting weights by introducing a small number of parameters into the weighting process to enhance module adaptability. To validate the feasibility of the approach, image recognition experiments are conducted on the CIFAR dataset and ImageNe-t 100 dataset. Experimental results demonstrate that by introducing a small number of parameters into the paramete-r free attention with preference, the mode'l s Top-1 Acc improves from 94.024% to 94.124% on the CIFAR dataset, and from 86.77% to 86.89% on the
ImageNe-t 100 dataset. Meanwhile, GradCAM+ + visualizations intuitively reveal that the proposed method shows a
more accurate and adaptable target attention range in different types of images. |
| Keywords: convolutional neural network; paramete-r free attention mechanism;parameterized attention mechanism |
