Block matching motion estimation techniques have been widely used in video coding applications. However, they also show their deficiency in the coherence of motion vectors and antinoise ability. This paper proposes a...Block matching motion estimation techniques have been widely used in video coding applications. However, they also show their deficiency in the coherence of motion vectors and antinoise ability. This paper proposes a modified algorithm which can adopt any one of existing search algorithms and pays more attention to the correlation of neighboring blocks.It will be shown that the proposed algorithm is simple and significantly reduces the computational complexity. Simulation results also show that this algorithm improves the smoothness of the motion field, hence reducing the cost to code the motion vectors while keeping good performance comparable with the conventional block matching motion estimation algorithm.展开更多
This paper presented an object-based fast motion estimation (ME) algorithm for object-based texture coding in moving picture experts group four (MPEG-4), which takes full advantage of the shape information of video ob...This paper presented an object-based fast motion estimation (ME) algorithm for object-based texture coding in moving picture experts group four (MPEG-4), which takes full advantage of the shape information of video object. Compared with the full search (FS) algorithm, the proposed algorithm can significantly speed the ME process. The speed of ME using the proposed algorithm is faster than that using new three-step search (NTSS), four-step search (4SS), diamond search (DS), and block-based gradient descent search (BBGDS) algorithms with similar motion compensation (MC) errors. The proposed algorithm can be combined with other fast ME algorithm to make the ME process faster.展开更多
Current mainstream unsupervised video object segmentation(UVOS) approaches typically incorporate optical flow as motion information to locate the primary objects in coherent video frames. However, they fuse appearance...Current mainstream unsupervised video object segmentation(UVOS) approaches typically incorporate optical flow as motion information to locate the primary objects in coherent video frames. However, they fuse appearance and motion information without evaluating the quality of the optical flow. When poor-quality optical flow is used for the interaction with the appearance information, it introduces significant noise and leads to a decline in overall performance. To alleviate this issue, we first employ a quality evaluation module(QEM) to evaluate the optical flow. Then, we select high-quality optical flow as motion cues to fuse with the appearance information, which can prevent poor-quality optical flow from diverting the network's attention. Moreover, we design an appearance-guided fusion module(AGFM) to better integrate appearance and motion information. Extensive experiments on several widely utilized datasets, including DAVIS-16, FBMS-59, and You Tube-Objects, demonstrate that the proposed method outperforms existing methods.展开更多
Block matching has been used for motion estimation and motion compensation in the Au- dio and Video Standard (AVS) for years. While having an acceptable performance in describing mo- tion between frames, it requires q...Block matching has been used for motion estimation and motion compensation in the Au- dio and Video Standard (AVS) for years. While having an acceptable performance in describing mo- tion between frames, it requires quite a few bits to represent the motion vectors. In certain circum- stances, the use of global motion estimation and compensation would perform equally well or even better than the block matching in terms of motion accuracy, while it results in the coding of global motion model parameters. In this letter, we modify an AVS coder by adding (1) six global motion model parameters to the frame header, and (2) mode selection among INTRA, SKIP, INTER-16×16, INTER-16×8, INTER-8×16, INTER-8×8, and Global Motion Compensation (GMC) modes by La- grange optimal rate-distortion criteria. Simulation results demonstrate that over 0.1dB improvement in PSNR is obtained compared to the AVS coder for an average coded P-frame with the same bit- rate.展开更多
Aiming at the higher bit-rate occupation of motion vector encoding and more time load of full-searching strategies, a multi-resolution motion estimation and compensation algorithm based on adjacent prediction of frame...Aiming at the higher bit-rate occupation of motion vector encoding and more time load of full-searching strategies, a multi-resolution motion estimation and compensation algorithm based on adjacent prediction of frame difference was proposed.Differential motion detection was employed to image sequences and proper threshold was adopted to identify the connected region.Then the motion region was extracted to carry out motion estimation and motion compensation on it.The experiment results show that the encoding efficiency of motion vector is promoted, the complexity of motion estimation is reduced and the quality of the reconstruction image at the same bit-rate as Multi-Resolution Motion Estimation(MRME) is improved.展开更多
The new features of H. 264 video coding standard make the motion estimation module much more time consuming than before. Especially, the motion search is required for each of the 4 modes for inter prediction. In order...The new features of H. 264 video coding standard make the motion estimation module much more time consuming than before. Especially, the motion search is required for each of the 4 modes for inter prediction. In order to reduce the computational complexity, we analyze the statistics of results of motion estimation, such as the continuity of best modes of blocks in successive frames and the chance to give up a sub-partition mode （smaller than 16 × 16） after integer-pixel motion estimation, from which we suggest to make mode prediction based on the motion information of the previous frame and skip sub-pixel motion estimation in subpartition mode selectively. According to the experimental result, the proposed algorithm can save 75 % of the computational time with a slight degradation （0.03 dB） on PSNR compared with the pseudocode of fast search motion estimation in JM12.2.展开更多
Scalable video quality enhancement refers to the process of enhancing low quality frames using high quality ones in scalable video bitstreams with time-varying qualities. A key problem in the enhancement is how to sea...Scalable video quality enhancement refers to the process of enhancing low quality frames using high quality ones in scalable video bitstreams with time-varying qualities. A key problem in the enhancement is how to search for correspondence between high quality and low quality frames. Previous algorithms usually use block-based motion estimation to search for correspondences. Such an approach can hardly estimate scale and rotation transforms and always in- troduces outliers to the motion estimation results. In this paper, we propose a pixel-based outlier-free motion estimation algorithm to solve this problem. In our algorithm, the motion vector for each pixel is calculated with respect to estimate translation, scale, and rotation transforms. The motion relationships between neighboring pixels are considered via the Markov random field model to improve the motion estimation accuracy. Outliers are detected and avoided by taking both blocking effects and matching percentage in scale- invariant feature transform field into consideration. Experiments are conducted in two scenarios that exhibit spatial scalability and quality scalability, respectively. Experimental results demonstrate that, in comparison with previous algorithms, the proposed algorithm achieves better correspondence and avoids the simultaneous introduction of outliers, especially for videos with scale and rotation transforms.展开更多
With the development of general-purpose processors (GPP) and video signal processing algorithms, it is possible to implement a software-based real-time video encoder on GPP, and its low cost and easy upgrade attract d...With the development of general-purpose processors (GPP) and video signal processing algorithms, it is possible to implement a software-based real-time video encoder on GPP, and its low cost and easy upgrade attract developers' interests to transfer video encoding from specialized hardware to more flexible software. In this paper, the encoding structure is set up first to support complexity scalability; then a lot of high performance algorithms are used on the key time-consuming modules in coding process; finally, at programming level, processor characteristics are considered to improve data access efficiency and processing parallelism. Other programming methods such as lookup table are adopted to reduce the computational complexity. Simulation results showed that these ideas could not only improve the global performance of video coding, but also provide great flexibility in complexity regulation.展开更多
A novel moving objects segmentation method is proposed in this paper. A modified three dimensional recursive search （3DRS） algorithm is used in order to obtain motion information accurately. A motion feature descrip...A novel moving objects segmentation method is proposed in this paper. A modified three dimensional recursive search （3DRS） algorithm is used in order to obtain motion information accurately. A motion feature descriptor （MFD） is designed to describe motion feature of each block in a picture based on motion intensity, motion in occlusion areas, and motion correlation among neighbouring blocks. Then, a fuzzy C-means clustering algorithm （FCM） is implemented based on those MFDs so as to segment moving objects. Moreover, a new parameter named as gathering degree is used to distinguish foreground moving objects and background motion. Experimental results demonstrate the effectiveness of the proposed method.展开更多
The basic search algorithm to implement Motion Estimation ( ME ) in the H .263 encoder is a full search. It is simple but time consuming. Traditional search algorithms are fast, but may cause a fall in image qua...The basic search algorithm to implement Motion Estimation ( ME ) in the H .263 encoder is a full search. It is simple but time consuming. Traditional search algorithms are fast, but may cause a fall in image quality or an increase in bit rate in low bit rate applications. A fast search algorithm for ME with consideration on image content is proposed in this paper. Experiments show that the proposed algorithm can offer up to 70 percent savings in execution time with almost no sacrifice in PSNR and bit rate, compared with the full search.展开更多
This paper proposes a motion-based region growing segmentation scheme for the object-based video coding, which segments an image into homogeneous regions characterized by a coherent motion. It adopts a block matching ...This paper proposes a motion-based region growing segmentation scheme for the object-based video coding, which segments an image into homogeneous regions characterized by a coherent motion. It adopts a block matching algorithm to estimate motion vectors and uses morphological tools such as open-close by reconstruction and the region-growing version of the watershed algorithm for spatial segmentation to improve the temporal segmentation. In order to determine the reliable motion vectors, this paper also proposes a change detection algorithm and a multi-candidate pro- screening motion estimation method. Preliminary simulation results demonstrate that the proposed scheme is feasible. The main advantage of the scheme is its low computational load.展开更多
In order to reduce the encoding complexity of macroblock coding mode decision in H.264/AVC, a selective smaller block-size searching algorithm and a selective intra coding mode searching algorithm are proposed by usin...In order to reduce the encoding complexity of macroblock coding mode decision in H.264/AVC, a selective smaller block-size searching algorithm and a selective intra coding mode searching algorithm are proposed by using the high correlation among coding modes and in spatial and temporal domains of video sequence. Simulation results demonstrate that the proposed algorithm can provide significant improvement in computational requirement, with negligible small picture quality degradation and slight bit rate increase.展开更多
The past two decades witnessed a broad-increase in web technology and on-line gaming.Enhancing the broadband confinements is viewed as one of the most significant variables that prompted new gaming technology.The imme...The past two decades witnessed a broad-increase in web technology and on-line gaming.Enhancing the broadband confinements is viewed as one of the most significant variables that prompted new gaming technology.The immense utilization of web applications and games additionally prompted growth in the handled devices and moving the limited gaming experience from user devices to online cloud servers.As internet capabilities are enhanced new ways of gaming are being used to improve the gaming experience.In cloud-based video gaming,game engines are hosted in cloud gaming data centers,and compressed gaming scenes are rendered to the players over the internet with updated controls.In such systems,the task of transferring games and video compression imposes huge computational complexity is required on cloud servers.The basic problems in cloud gaming in particular are high encoding time,latency,and low frame rates which require a new methodology for a better solution.To improve the bandwidth issue in cloud games,the compression of video sequences requires an alternative mechanism to improve gaming adaption without input delay.In this paper,the proposed improved methodology is used for automatic unnecessary scene detection,scene removing and bit rate reduction using an adaptive algorithm for object detection in a game scene.As a result,simulations showed without much impact on the players’quality experience,the selective object encoding method and object adaption technique decrease the network latency issue,reduce the game streaming bitrate at a remarkable scale on different games.The proposed algorithm was evaluated for three video game scenes.In this paper,achieved 14.6%decrease in encoding and 45.6%decrease in bit rate for the first video game scene.展开更多
This paper proposes a motion-based region growing segmentation scheme, which incorporatesluminance and motion information simultaneously and uses morphological tools such as open-close byreconstruction and the region-...This paper proposes a motion-based region growing segmentation scheme, which incorporatesluminance and motion information simultaneously and uses morphological tools such as open-close byreconstruction and the region-growing version of the watershed algorithm. The main advantage of this scheme is thatthe resultant objects ore characterized by a coherent motion and foe moving object boundaries are precisely located.Simulation results demonstrate the effiency of the Proposed scheme.展开更多
文摘Block matching motion estimation techniques have been widely used in video coding applications. However, they also show their deficiency in the coherence of motion vectors and antinoise ability. This paper proposes a modified algorithm which can adopt any one of existing search algorithms and pays more attention to the correlation of neighboring blocks.It will be shown that the proposed algorithm is simple and significantly reduces the computational complexity. Simulation results also show that this algorithm improves the smoothness of the motion field, hence reducing the cost to code the motion vectors while keeping good performance comparable with the conventional block matching motion estimation algorithm.
基金National High Technology Research and De-velopment Program of China (863 Program)(No.2003AA103810)
文摘This paper presented an object-based fast motion estimation (ME) algorithm for object-based texture coding in moving picture experts group four (MPEG-4), which takes full advantage of the shape information of video object. Compared with the full search (FS) algorithm, the proposed algorithm can significantly speed the ME process. The speed of ME using the proposed algorithm is faster than that using new three-step search (NTSS), four-step search (4SS), diamond search (DS), and block-based gradient descent search (BBGDS) algorithms with similar motion compensation (MC) errors. The proposed algorithm can be combined with other fast ME algorithm to make the ME process faster.
基金supported by the National Natural Science Foundation of China (No.61872189)。
文摘Current mainstream unsupervised video object segmentation(UVOS) approaches typically incorporate optical flow as motion information to locate the primary objects in coherent video frames. However, they fuse appearance and motion information without evaluating the quality of the optical flow. When poor-quality optical flow is used for the interaction with the appearance information, it introduces significant noise and leads to a decline in overall performance. To alleviate this issue, we first employ a quality evaluation module(QEM) to evaluate the optical flow. Then, we select high-quality optical flow as motion cues to fuse with the appearance information, which can prevent poor-quality optical flow from diverting the network's attention. Moreover, we design an appearance-guided fusion module(AGFM) to better integrate appearance and motion information. Extensive experiments on several widely utilized datasets, including DAVIS-16, FBMS-59, and You Tube-Objects, demonstrate that the proposed method outperforms existing methods.
基金Supported by the National Natural Science Foundation of China (60673092)Natural science Foundation of Ji-angsu Province (No.BK2003029)the Key Laboratory Project of Information Science & Engineering of Railway of the Ministry of Railways, China (No.TDXX0501).
文摘Block matching has been used for motion estimation and motion compensation in the Au- dio and Video Standard (AVS) for years. While having an acceptable performance in describing mo- tion between frames, it requires quite a few bits to represent the motion vectors. In certain circum- stances, the use of global motion estimation and compensation would perform equally well or even better than the block matching in terms of motion accuracy, while it results in the coding of global motion model parameters. In this letter, we modify an AVS coder by adding (1) six global motion model parameters to the frame header, and (2) mode selection among INTRA, SKIP, INTER-16×16, INTER-16×8, INTER-8×16, INTER-8×8, and Global Motion Compensation (GMC) modes by La- grange optimal rate-distortion criteria. Simulation results demonstrate that over 0.1dB improvement in PSNR is obtained compared to the AVS coder for an average coded P-frame with the same bit- rate.
基金Supported by the National Natural Science Foundation of China (No. 60803036)the Scientific Research Fund of Heilongjiang Provincial Education Department (No.11531013)
文摘Aiming at the higher bit-rate occupation of motion vector encoding and more time load of full-searching strategies, a multi-resolution motion estimation and compensation algorithm based on adjacent prediction of frame difference was proposed.Differential motion detection was employed to image sequences and proper threshold was adopted to identify the connected region.Then the motion region was extracted to carry out motion estimation and motion compensation on it.The experiment results show that the encoding efficiency of motion vector is promoted, the complexity of motion estimation is reduced and the quality of the reconstruction image at the same bit-rate as Multi-Resolution Motion Estimation(MRME) is improved.
基金Sponsored by the National Natural Science Foundation of China(60772066)
文摘The new features of H. 264 video coding standard make the motion estimation module much more time consuming than before. Especially, the motion search is required for each of the 4 modes for inter prediction. In order to reduce the computational complexity, we analyze the statistics of results of motion estimation, such as the continuity of best modes of blocks in successive frames and the chance to give up a sub-partition mode （smaller than 16 × 16） after integer-pixel motion estimation, from which we suggest to make mode prediction based on the motion information of the previous frame and skip sub-pixel motion estimation in subpartition mode selectively. According to the experimental result, the proposed algorithm can save 75 % of the computational time with a slight degradation （0.03 dB） on PSNR compared with the pseudocode of fast search motion estimation in JM12.2.
基金Acknowledgements This work was supported by the National Science Fund for Distinguished Young Scholars of China （61125102）, and the State Key Program of National Natural Science Foundation of China （Grant No. 61133008）.
文摘Scalable video quality enhancement refers to the process of enhancing low quality frames using high quality ones in scalable video bitstreams with time-varying qualities. A key problem in the enhancement is how to search for correspondence between high quality and low quality frames. Previous algorithms usually use block-based motion estimation to search for correspondences. Such an approach can hardly estimate scale and rotation transforms and always in- troduces outliers to the motion estimation results. In this paper, we propose a pixel-based outlier-free motion estimation algorithm to solve this problem. In our algorithm, the motion vector for each pixel is calculated with respect to estimate translation, scale, and rotation transforms. The motion relationships between neighboring pixels are considered via the Markov random field model to improve the motion estimation accuracy. Outliers are detected and avoided by taking both blocking effects and matching percentage in scale- invariant feature transform field into consideration. Experiments are conducted in two scenarios that exhibit spatial scalability and quality scalability, respectively. Experimental results demonstrate that, in comparison with previous algorithms, the proposed algorithm achieves better correspondence and avoids the simultaneous introduction of outliers, especially for videos with scale and rotation transforms.
文摘With the development of general-purpose processors (GPP) and video signal processing algorithms, it is possible to implement a software-based real-time video encoder on GPP, and its low cost and easy upgrade attract developers' interests to transfer video encoding from specialized hardware to more flexible software. In this paper, the encoding structure is set up first to support complexity scalability; then a lot of high performance algorithms are used on the key time-consuming modules in coding process; finally, at programming level, processor characteristics are considered to improve data access efficiency and processing parallelism. Other programming methods such as lookup table are adopted to reduce the computational complexity. Simulation results showed that these ideas could not only improve the global performance of video coding, but also provide great flexibility in complexity regulation.
基金Supported by the National Natural Science Foundation of China （No. 60772134, 60902081, 60902052） the 111 Project （No.B08038） the Fundamental Research Funds for the Central Universities（No.72105457）.
文摘A novel moving objects segmentation method is proposed in this paper. A modified three dimensional recursive search （3DRS） algorithm is used in order to obtain motion information accurately. A motion feature descriptor （MFD） is designed to describe motion feature of each block in a picture based on motion intensity, motion in occlusion areas, and motion correlation among neighbouring blocks. Then, a fuzzy C-means clustering algorithm （FCM） is implemented based on those MFDs so as to segment moving objects. Moreover, a new parameter named as gathering degree is used to distinguish foreground moving objects and background motion. Experimental results demonstrate the effectiveness of the proposed method.
文摘The basic search algorithm to implement Motion Estimation ( ME ) in the H .263 encoder is a full search. It is simple but time consuming. Traditional search algorithms are fast, but may cause a fall in image quality or an increase in bit rate in low bit rate applications. A fast search algorithm for ME with consideration on image content is proposed in this paper. Experiments show that the proposed algorithm can offer up to 70 percent savings in execution time with almost no sacrifice in PSNR and bit rate, compared with the full search.
文摘This paper proposes a motion-based region growing segmentation scheme for the object-based video coding, which segments an image into homogeneous regions characterized by a coherent motion. It adopts a block matching algorithm to estimate motion vectors and uses morphological tools such as open-close by reconstruction and the region-growing version of the watershed algorithm for spatial segmentation to improve the temporal segmentation. In order to determine the reliable motion vectors, this paper also proposes a change detection algorithm and a multi-candidate pro- screening motion estimation method. Preliminary simulation results demonstrate that the proposed scheme is feasible. The main advantage of the scheme is its low computational load.
基金National Natural Science Foundation of China (60372018)
文摘In order to reduce the encoding complexity of macroblock coding mode decision in H.264/AVC, a selective smaller block-size searching algorithm and a selective intra coding mode searching algorithm are proposed by using the high correlation among coding modes and in spatial and temporal domains of video sequence. Simulation results demonstrate that the proposed algorithm can provide significant improvement in computational requirement, with negligible small picture quality degradation and slight bit rate increase.
文摘The past two decades witnessed a broad-increase in web technology and on-line gaming.Enhancing the broadband confinements is viewed as one of the most significant variables that prompted new gaming technology.The immense utilization of web applications and games additionally prompted growth in the handled devices and moving the limited gaming experience from user devices to online cloud servers.As internet capabilities are enhanced new ways of gaming are being used to improve the gaming experience.In cloud-based video gaming,game engines are hosted in cloud gaming data centers,and compressed gaming scenes are rendered to the players over the internet with updated controls.In such systems,the task of transferring games and video compression imposes huge computational complexity is required on cloud servers.The basic problems in cloud gaming in particular are high encoding time,latency,and low frame rates which require a new methodology for a better solution.To improve the bandwidth issue in cloud games,the compression of video sequences requires an alternative mechanism to improve gaming adaption without input delay.In this paper,the proposed improved methodology is used for automatic unnecessary scene detection,scene removing and bit rate reduction using an adaptive algorithm for object detection in a game scene.As a result,simulations showed without much impact on the players’quality experience,the selective object encoding method and object adaption technique decrease the network latency issue,reduce the game streaming bitrate at a remarkable scale on different games.The proposed algorithm was evaluated for three video game scenes.In this paper,achieved 14.6%decrease in encoding and 45.6%decrease in bit rate for the first video game scene.
文摘This paper proposes a motion-based region growing segmentation scheme, which incorporatesluminance and motion information simultaneously and uses morphological tools such as open-close byreconstruction and the region-growing version of the watershed algorithm. The main advantage of this scheme is thatthe resultant objects ore characterized by a coherent motion and foe moving object boundaries are precisely located.Simulation results demonstrate the effiency of the Proposed scheme.
