Spatio-Temporal Networks for Human Activity Recognition based on Optical Flow in Omnidirectional Image Scenes

This dissertation explores human activity recognition (HAR) by leveraging human motion perception through artificial neural networks. It investigates the effectiveness of RGB images, optical flow, and human keypoints for HAR in omnidirectional data, using a synthetically generated dataset called OmniFlow. The study validates this dataset with Test-Time Augmentation and demonstrates that fine-tuning with approximately 1000 images significantly reduces error rates. Two advanced convolutional neural networks, TSN and PoseC3D, are employed for performance evaluation, with detailed accuracy metrics provided for various modalities.