Grad-CAM vs HiResCAM : a comparative study via quantitative evaluation metrics

Summary

In this study we utilize the Grad-CAM and HiResCAM attribution map methods and consider a setting where the HiResCAM algorithm provably produces faithful explanations while Grad-CAM does not. This theoretical result motivates us to investigate the quality of their attribution maps in terms of quantitative evaluation metrics and examine if faithfulness aligns with the metrics results. Our evaluation scheme implements the well-established AOPC and Max-Sensitivity scores along with the recently introduced HAAS score and utilizes ResNet and VGG pre-trained architectures trained on four medical image datasets. The experimental results suggest that Max-Sensitivity and AOPC favour faithfulness. On the other hand, HAAS does not contribute meaningful values to our comparison, but rather inspires further study about its nature.