Neurο-symbolic answer set programming for human activity recognition in videos

Summary

Machine / Deep Learning and Machine Reasoning are considered two different subfields of Artificial Intelligence. With machine learning methods we can build models with low level perceptual capabilities and with logic based methods we can extract information and perform reasoning at a higher level. Combining neural learning methods with logic-based techniques could help create systems that are able to perceive their environment and infer the data given as input.

In this thesis, we will focus on neurosymbolic computation, where the combination of deep learning and reasoning is achieved through an existing framework called NeurAsp. We will go through simple examples that demonstrate NeurAsp’s capabilities and show how it works and integrates internally with traditional deep learning methods. The main goal of this thesis is to apply this method to the task of detecting human activity in videos with the usage of Complex Event Recognition (CER) techniques. Finally, we will show the benefits of integrating logic-based techniques with neural methods by presenting three different experimental setups in which we compare the performances of pure traditional deep learning methods and those proposed by the NeurAsp framework.