Learning group interaction for sports video understanding from a perspective of athlete

Learning activities interactions between small groups is a key step in understanding team sports videos.Recent research focusing on team sports videos can be strictly regarded from the perspective of the audience rather than the athlete.For team sports videos such as volleyball and basketball videos,there are plenty of intra-team and inter-team relations.In this paper,a new task named Group Scene Graph Generation is introduced to better understand intra-team relations and inter-team relations in sports videos.To tackle this problem,a novel Hierarchical Relation Network is proposed.After all players in a video are finely divided into two teams,the feature of the two teams’activities and interactions will be enhanced by Graph Convolutional Networks,which are finally recognized to generate Group Scene Graph.For evaluation,built on Volleyball dataset with additional 9660 team activity labels,a Volleyball+dataset is proposed.A baseline is set for better comparison and our experimental results demonstrate the effectiveness of our method.Moreover,the idea of our method can be directly utilized in another video-based task,Group Activity Recognition.Experiments show the priority of our method and display the link between the two tasks.Finally,from the athlete’s view,we elaborately present an interpretation that shows how to utilize Group Scene Graph to analyze teams’activities and provide professional gaming suggestions.

Real-time distance field acceleration based free-viewpoint video synthesis for large sports fields

Free-viewpoint video allows the user to view objects from any virtual perspective,creating an immersive visual experience.This technology enhances the interactivity and freedom of multimedia performances.However,many free-viewpoint video synthesis methods hardly satisfy the requirement to work in real time with high precision,particularly for sports fields having large areas and numerous moving objects.To address these issues,we propose a freeviewpoint video synthesis method based on distance field acceleration.The central idea is to fuse multiview distance field information and use it to adjust the search step size adaptively.Adaptive step size search is used in two ways:for fast estimation of multiobject three-dimensional surfaces,and synthetic view rendering based on global occlusion judgement.We have implemented our ideas using parallel computing for interactive display,using CUDA and OpenGL frameworks,and have used real-world and simulated experimental datasets for evaluation.The results show that the proposed method can render free-viewpoint videos with multiple objects on large sports fields at 25 fps.Furthermore,the visual quality of our synthetic novel viewpoint images exceeds that of state-of-the-art neural-rendering-based methods.

Digit Recognition in Natural Scene Texts

Digit recognition from a natural scene text in video surveillance/broadcasting applications is a challenging research task due to blurred, font variations, twisted, and non-uniform color distribution issues with a digit in a natural scene to be recognized. In this paper, to solve the digit number recognition problem, a principal-axis based topology contour descriptor with support vector machine （SVM） classification is proposed. The contributions of this paper include： a） a local descriptor with SVM classification for digit recognition, b） higher accuracy than the state-of-the art methods, and c） low computational power （0.03 second/digit recognition）, which make this method adoptable to real-time applications.

Semantic and Structural Analysis of TV Diving Programs

Automatic content analysis of sports videos is a valuable and challenging task. Motivated by analogies between a class of sports videos and languages, the authors propose a novel approach for sports video analysis based on compiler principles. It integrates both semantic analysis and syntactic analysis to automatically create an index and a table of contents for a sports video. Each shot of the video sequence is first annotated and indexed with semantic labels through detection of events using domain knowledge. A grammar-based parser is then constructed to identify the tree structure of the video content based on the labels. Meanwhile, the grammar can be used to detect and recover errors during the analysis. As a case study, a sports video parsing system is presented in the particular domain of diving. Experimental results indicate the proposed approach is effective. Keywords sports video - event detection - grammar - video retrieval - content analysis and index This work was supported in part by the State Physical Culture Administration of China under Grant No.02005.Fei Wang was born in 1977. He is a Ph.D. candidate at Institute of Computing Technology (ICT), the Chinese Academy of Sciences (CAS). He received the B.S. degree in electrical engineering from Zhejiang University in 1999 and the M.S degree in computer science from Graduate School of the Chinese Academy of Sciences in 2001. His current research interests include content-based video analysis and retrieval.Jin-Tao Li was born in 1962. He is a professor and Ph.D. supervisor at ICT, CAS. His main research areas include multimedia data compression, virtual reality, and home network.Yong-Dong Zhang was born in 1973. He is an associate professor at ICT, CAS. His main research areas include multimedia data compression and multimedia information retrieval.Shou-Xun Lin was born in 1948. He is a professor and Ph.D. supervisor at ICT, CAS. His main research areas include multimedia technology and systems.