Human Activities Detection Using Social Robotic Platform
- Post by: Sohar University
- June 14, 2022
- Comments off
A large amount of computer-generated multimedia data (e.g., text, image and video) available on digital resources has created the potential for a substantial amount of useful information buried in these resources. This in turn has led to the need for automated techniques of discovering previously unknown, and potentially useful pattern or knowledge with its interrelationships and actions in those available data resources. One important topic within this area is the issue of human activities detection. This topic is widely related to computer vision area and it dates back to the 1960s, which was originally inspired by the capability of the human being to detect and recognize objects, emotions and other humanoid activities.
The human activities detection process is the process of recognizing instances of human activities such as happy, sad, walk, play, look, and etc. in the several of multimedia data. Human activities detection and recognition approaches commonly apply feature extraction and automatic learning processes to detect and recognize instances of an object, emotion or any other activity within multimedia data streams belonging to an object domain or class. Human activity detection deals with localizing a particular activity of interest in data streams, whereas activity recognition aims to classify objects into a specific class or domain. Each activity-class and activity-instance has its own specific features that describe and differentiate it from other activities and leading to recognize similar activities in other classes in the target data streams. Human activity detection and recognition are also used in several applications of computer vision, including self-driving cars, face recognition, social distance control, surveillance, image retrieval, distance learning and other variety of multimedia-related tasks.
The activities founded in multimedia data streams can be detected and recognized automatically using robotics-related techniques known as deep and machine learning. The main aim of this study, therefore, is to investigate if verbal, vocal, and facial information (i.e., emotional information) can be used to identify the academic performance of the students in robot-led teaching practice using deep and machine learning techniques. We employed social robot called Furhat to investigate how teacher characters (i.e., charisma and behavior) can affect the students’ performance by applying all available Furhat’s predefined faces and characters. There are many interesting questions we have explored here: How can natural language processing and sentiment and emotional analysis techniques be combined with social robotic platforms to achieve better performance than can be achieved by either of these approaches alone in evaluating students’ performance? In the empirical part, however, we have also explored how possible traditional human activity detection and recognition methods can be implemented in practice over social robotic platform, evaluate how well a used detection and recognition methods are experimentally learned on benchmark datasets in various real-world scenarios and tasks.
Dr. Khaled Abdalgader Omar is Assistant Professor in Computer Science, Faculty of Computing and Information Technology, Sohar University, Oman