Here we present a new Explainable AI (XAI) method to probe the functional partition in AI models by comparing features attended to at different layers with human attention driven by diverse task demands. We applied this method to explain an object detector Yolo-v5s in multi-category and single-category object detection tasks. We found that the model’s neck showed higher similarity to human attention during object detection, indicating a reliance on diagnostic features in the neck, whereas its backbone showed higher similarity to attention during passive viewing, indicating salient local features encoded. With this understanding of its functional partition, using Yolo-v5s as a model for human cognition, our comparative analysis against human attention when providing explanations for object detection revealed that humans attended to a combination of diagnostic and salient features during explaining multi-category general object detection but attended to mainly diagnostic features when explaining single-category human/vehicle detection in driving scenarios.

Authors:

Jinhan Zhang: The University of Hong Kong; Guoyang Liu: Shandong University; Yunke Chen: The University of Hong Kong; Antoni B. Chan: City University of Hong Kong; Janet Hsiao: Hong Kong University of Science & Technology