proposed to connect the appearance and motion streams with multiplicative connections at several layers, as opposed to previous models which would only interact at the prediction layer. We use the ResNet-50-based model proposed in as baseline architecture for each stream block of our model. All these features are extracted from objects detected and tracked in video sequence after spatiotemporal segmentation. (1998) uses quantized CIE–LUV space as the color feature, three Tamura texture measures (coarseness, contrast, and orientation) as texture feature, and shape components and motion vectors. A similar approach proposed by Shih-Fu Chang et al. Texture is represented by gray-level co-occurrence matrices at four orientations.
The three histograms corresponding to the three channels (hue, saturation, and value) are then combined into one vector of dimension 36. The HSV color space is used because it is perceptually closer to human vision as compared to the RGB space. A linearized HSV histogram is used that has 12 bins per channel as the color feature. According to their experiments, each region is then processed for feature extraction. (1998) first segmented the video spatiotemporally obtaining regions in each shot. Thus, features used to represent video data have conventionally been the same ones used for images, extracted from keyframes of the video sequence, with additional motion features used to capture temporal aspects of video data. The obvious candidates for feature space are color, texture, and shape. At this level, any of the techniques from representation of image indexing schemes can be used. Low-level image representation features can be extracted from keyframes in an effort to efficiently model the visual content. Dan Schonfeld, in The Electrical Engineering Handbook, 2005 Spatial Image Features For motion feature, we also use BoFs to increase effectiveness of recognition as SIFT feature from image features.įaisal Bashir. Lastly, we calculate MBH feature for each cell and join them together.
Then, each block is divided into n σ × n σ × n t cells. It means scaling each optical flow matrix to size N × M and each block containing L optical flow.
To describe motion feature, each video is separated to many blocks, which are N × M × L-size blocks. Next, we track in optical flows to find out trajectories in a sequence of 15 continuous frames. To compute the optical flow from above dense samples, we use FarneBack algorithm because it is one of the fastest algorithm to compute a dense optical flow. Thus, it is completely appropriate to be applied in this step for action representation. Besides, MBH descriptor expresses only boundary of foreground motion and eliminates background and camera motion. Moreover, there are many fine motions in cooking videos, so that we use dense trajectories feature that is the best feature for representing even the fine motions. The main reason for choosing this motion feature is every cooking actions are characterized by different simple motions, such as cutting action is related to vertical motions while mixing action is almost described by turn around motions. In our method, we use dense trajectories and motion boundary histogram (MBH) description for action representation. Therefore, in our research, we use one of fastest and most density motion feature which was studied by Wang et al.īeing parallel with the image features extraction, we extract motion features from videos. However, this feature requires an enormous computation and is much more complex than image feature. Jin Young Kim, in Emerging Trends in Image Processing, Computer Vision and Pattern Recognition, 2015 3.4 Motion Feature Extractionįor solving our problem, motion feature is indispensable because of their efficient in motion representation.
0 kommentar(er)
