Fine-grained action recognition by motion saliency and mid-level patches
Article
Liu, F., Zhao, L., Cheng, X., Dai, Q., Shi, X. and Qiao, J. 2020. Fine-grained action recognition by motion saliency and mid-level patches. Applied Sciences. 10 (8). https://doi.org/10.3390/app10082811
Type | Article |
---|---|
Title | Fine-grained action recognition by motion saliency and mid-level patches |
Authors | Liu, F., Zhao, L., Cheng, X., Dai, Q., Shi, X. and Qiao, J. |
Abstract | Effective extraction of human body parts and operated objects participating in action is the key issue of fine-grained action recognition. However, most of the existing methods require intensive manual annotation to train the detectors of these interaction components. In this paper, we represent videos by mid-level patches to avoid the manual annotation, where each patch corresponds to an action-related interaction component. In order to capture mid-level patches more exactly and rapidly, candidate motion regions are extracted by motion saliency. Firstly, the motion regions containing interaction components are segmented by a threshold adaptively calculated according to the saliency histogram of the motion saliency map. Secondly, we introduce a mid-level patch mining algorithm for interaction component detection, with object proposal generation and mid-level patch detection. The object proposal generation algorithm is used to obtain multi-granularity object proposals inspired by the idea of the Huffman algorithm. Based on these object proposals, the mid-level patch detectors are trained by K-means clustering and SVM. Finally, we build a fine-grained action recognition model using a graph structure to describe relationships between the mid-level patches. To recognize actions, the proposed model calculates the appearance and motion features of mid-level patches and the binary motion cooperation relationships between adjacent patches in the graph. Extensive experiments on the MPII cooking database demonstrate that the proposed method gains better results on fine-grained action recognition. |
Keywords | fine-grained action recognition, motion saliency, mid-level patch, object proposal |
Publisher | MDPI AG |
Journal | Applied Sciences |
ISSN | 2076-3417 |
Publication dates | |
18 Apr 2020 | |
Online | 18 Apr 2020 |
Publication process dates | |
Deposited | 20 Apr 2020 |
Accepted | 15 Apr 2020 |
Output status | Published |
Publisher's version | License |
Copyright Statement | © 2020 by the authors. |
Additional information | This article belongs to the Special Issue Intelligent Processing on Image and Optical Information - Ⅱ. |
Digital Object Identifier (DOI) | https://doi.org/10.3390/app10082811 |
Language | English |
https://repository.mdx.ac.uk/item/88y1z
Download files
56
total views10
total downloads2
views this month0
downloads this month