HUMAN ACTION RECOGNITION USING ATTENTION BASED SPATIOTEMPORAL GRAPH CONVOLUTIONAL NETWORK

Abstract

Human skeleton modelling has gained popularity in recent years. As skeleton data successfully handles dynamic settings and complicated backdrops, the human skeleton dynamics include essential information for the identification of human actions. GCNs have shown substantial effectiveness in modelling the non-Euclidean character of human skeleton structures. Human skeleton structures are present in the form of spatiotemporal graphs depicting sequences of body skeletons during an action. We present an attention based human action recognition model that uses the mechanism of temporal and spatial attention modules to improve identification. The temporal attention module captures the most informative frames from a sequence of skeletons. The spatial attention mechanism then emphasizes the most informative joints from the frames highlighted. Frame selection is then performed to select the skeletons with the highest attention scores. Spatial and temporal modules are incorporated into the graph convolutional network. Both attention modules improve the model's effectiveness and the efficiency of skeleton-based human action identification when used together. The model is evaluated on two benchmarks of the NTURGB+D dataset, i.e., cross-view benchmark and cross-subject benchmark. The top-1 accuracy of both models is compared with existing benchmark techniques. The experimental findings show that our model exceeded the current benchmark methodologies, providing a considerable improvement over the baseline technique.