摘要详情

ID / 提交时间

33 / 2023-08-29 10:40:18

标题

The two-way knowledge interaction interface between humans and neural networks

关键字

interpretability,interaction interface,knowledge distillation,graph neural networks

主题及专题

Special Sessions > SS11 - Human Activity and Scene Understanding

状态

终稿

作者

Zhanliang He / Xidian University

Nuoye Xiong / Xidian University

Hongsheng Li / Xidian University

Peiyi Shen / Xidian University

Guangming Zhu / Xidian University

Liang Zhang / Xidian University

摘要

Despite neural networks (NN) have been widely applied in various fields and generally outperforms humans, they still lack interpretability to a certain extent, and humans are unable to intuitively understand the decision logic of NN. This also hinders the knowledge interaction between humans and NN, preventing humans from getting involved to give direct guidance when NN’s decisions go wrong. While recent research in explainable AI has achieved interpretability of NN from various perspectives, it has not yet provided effective methods for knowledge exchange between humans and NN. To address this problem, we constructed a two-way interaction interface that uses structured representations of visual concepts and their relationships as the “language” for knowledge exchange between humans and NN. Specifically, NN provide intuitive reasoning explanations to humans based on the class-specific structural concepts graph (C-SCG). On the other hand, humans can modify the biases present in the C-SCG through their prior knowledge and reasoning ability, and thus provide direct knowledge guidance to NN through this interface. Through experimental validation, based on this interaction interface, NN can provide humans with easily understandable explanations of the reasoning process. Furthermore, human involvement and prior knowledge can directly and effectively contribute to enhancing the performance of NN.