为了节省360°全景视频的编码时间,对通用视频编码标准中的编码单元划分决策过程进行了研究,提出了一种面向360°全景视频的帧内预测编码的快速算法。通过优化编码树单元(Coding Tree Unit,CTU)的编码深度范围和编码单元的划分...为了节省360°全景视频的编码时间,对通用视频编码标准中的编码单元划分决策过程进行了研究,提出了一种面向360°全景视频的帧内预测编码的快速算法。通过优化编码树单元(Coding Tree Unit,CTU)的编码深度范围和编码单元的划分模式的选择过程,减少编码时间。实验结果表明,在全帧内模式下,所提算法比原始算法平均可以节省34.33%的时间复杂度,同时带来的BDBR平均增量仅为1.665%,BDPSNR的平均降低量仅为0.076 dB。展开更多
Video synopsis is an effective way to easily summarize long-recorded surveillance videos.The omnidirectional view allows the observer to select the desired fields of view(FoV)from the different FoVavailable for spheri...Video synopsis is an effective way to easily summarize long-recorded surveillance videos.The omnidirectional view allows the observer to select the desired fields of view(FoV)from the different FoVavailable for spherical surveillance video.By choosing to watch one portion,the observer misses out on the events occurring somewhere else in the spherical scene.This causes the observer to experience fear of missing out(FOMO).Hence,a novel personalized video synopsis approach for the generation of non-spherical videos has been introduced to address this issue.It also includes an action recognition module that makes it easy to display necessary actions by prioritizing them.This work minimizes and maximizes multiple goals such as loss of activity,collision,temporal consistency,length,show,and important action cost respectively.The performance of the proposed framework is evaluated through extensive simulation and compared with the state-of-art video synopsis optimization algorithms.Experimental results suggest that some constraints are better optimized by using the latest metaheuristic optimization algorithms to generate compact personalized synopsis videos from spherical surveillance videos.展开更多
360 video streaming services over the network are becoming popular. In particular, it is easy to experience 360 video through the already popular smartphone. However, due to the nature of 360 video, it is difficult to...360 video streaming services over the network are becoming popular. In particular, it is easy to experience 360 video through the already popular smartphone. However, due to the nature of 360 video, it is difficult to provide stable streaming service in general network environment because the size of data to send is larger than that of conventional video. Also, the real user's viewing area is very small compared to the sending amount. In this paper, we propose a system that can provide high quality 360 video streaming services to the users more efficiently in the cloud. In particular, we propose a streaming system focused on using a head mount display (HMD).展开更多
随着虚拟现实技术不断发展,360度视频编码开始成为研究热点。应用于虚拟现实的360度视频与传统的视频相比分辨率更高,编码数据量更大,在实际应用中面临传输带宽有限的瓶颈,编码效率问题有待解决。归纳分析了国际标准组织联合视频研究组(...随着虚拟现实技术不断发展,360度视频编码开始成为研究热点。应用于虚拟现实的360度视频与传统的视频相比分辨率更高,编码数据量更大,在实际应用中面临传输带宽有限的瓶颈,编码效率问题有待解决。归纳分析了国际标准组织联合视频研究组(Joint Video Exploration Team,JVET)正在制定的360度视频编码投影变换技术及其编码优化方法,综合对比了各变换技术的编码性能。根据当前最新研究成果,对下一步研究需要解决的问题进行了探讨总结,给出未来的研究方向和思路。展开更多
We describe four fundamental challenges that complex real-life Virtual Reality (VR) productions are facing today (such as multi-camera management, quality control, automatic annotation with cinematography and 360&...We describe four fundamental challenges that complex real-life Virtual Reality (VR) productions are facing today (such as multi-camera management, quality control, automatic annotation with cinematography and 360˚?depth estimation) and describe an integrated solution, called Hyper 360, to address them. We demonstrate our solution and its evaluation in the context of practical productions and present related results.展开更多
随着虚拟现实技术的发展,360度视频越来越受欢迎。这些视频在使用标准编码器进行编码之前,要先将其转换为2D图像平面格式。为了提高编码效率,专家们提出了新一代视频编码标准H.266/VVC(Versatile Video Coding),然而,VVC分区模式的多样...随着虚拟现实技术的发展,360度视频越来越受欢迎。这些视频在使用标准编码器进行编码之前,要先将其转换为2D图像平面格式。为了提高编码效率,专家们提出了新一代视频编码标准H.266/VVC(Versatile Video Coding),然而,VVC分区模式的多样性导致编码360度高分辨率视频耗时过长。针对上述问题,设计一种CU划分早期决策算法。通过对ERP(Equirectangular projection)视频的统计实验,发现这类视频采用水平分区的概率大于垂直分区。利用经验变差函数设计算法衡量纹理方向差异度,再根据编码单元水平与垂直2个方向的差异程度选择不同的分区。实验结果表明:在全帧内模式下,与VVC测试模型VTM4.0相比,该算法节省了35.42%的编码时间,BD-rate仅增加0.70%。展开更多
360° video has been becoming one of the major media in recent years, providing immersive experience for viewers with more interactions compared with traditional videos. Most of today's implementations rely on...360° video has been becoming one of the major media in recent years, providing immersive experience for viewers with more interactions compared with traditional videos. Most of today's implementations rely on bulky Head-Mounted Displays (HMDs) or require touch screen operations for interactive display, which are not only expensive but also inconvenient for viewers. In this paper, we demonstrate that interactive 360° video streaming can be done with hints from gaze movement detected by the front camera of today's mobile devices (e.g., a smartphone). We design a lightweight real-time gaze point tracking method for this purpose. We integrate it with streaming module and apply a dynamic margin adaption algorithm to minimize the overall energy consumption for battery-constrained mobile devices. Our experiments on state-of-the-art smartphones show the feasibility of our solution and its energy efficiency toward cost-effective real-time 360° video streaming.展开更多
Improving the danger prediction during driving can significantly reduce the risk of accidents.However,previous danger prediction training systems had not been sufficiently effective owing to the lack of realism.In thi...Improving the danger prediction during driving can significantly reduce the risk of accidents.However,previous danger prediction training systems had not been sufficiently effective owing to the lack of realism.In this study,we propose an immersive training system for danger prediction training using virtual reality(VR)technology.This system provides drivers with a highly realistic training environment with 360°videos viewed with VR goggles.Users can practice various dangerous scenarios in an environment that simulates a real-driving situation.In addition,we introduced a system to select dangerous spots with a controller and implement training schemes on a voluntary basis.This setup enables them to train in a highly interactive state.In addition,we proposed a method to express multiple indices numerically so that users can understand the training effect.We tested the effect of the system on the danger prediction abilities of various users with two experiments by using this approach.These results show that our system was more effective in improving the driver’s danger prediction ability than previous systems.展开更多
为了提高对360°全景视频的编码效率,联合探索专家组研发了基于HEVC的下一代视频编码标准——多功能视频编码标准(Versatile Video Coding,VVC)。相对于HEVC,VVC具有更高的编码效率,但是也引入了更高的时间复杂度。因此为了降低其...为了提高对360°全景视频的编码效率,联合探索专家组研发了基于HEVC的下一代视频编码标准——多功能视频编码标准(Versatile Video Coding,VVC)。相对于HEVC,VVC具有更高的编码效率,但是也引入了更高的时间复杂度。因此为了降低其编码的计算复杂度,提出了一种针对VVC的帧内模式快速决策算法。通过分析图像块的纹理特性来减少帧内编码的候选模式数量从而减少模式选择中的冗余计算。实验结果表明,提出的算法可以节省24. 08%的编码时间同时只有0. 80%的BDrate损失。展开更多
文摘为了节省360°全景视频的编码时间,对通用视频编码标准中的编码单元划分决策过程进行了研究,提出了一种面向360°全景视频的帧内预测编码的快速算法。通过优化编码树单元(Coding Tree Unit,CTU)的编码深度范围和编码单元的划分模式的选择过程,减少编码时间。实验结果表明,在全帧内模式下,所提算法比原始算法平均可以节省34.33%的时间复杂度,同时带来的BDBR平均增量仅为1.665%,BDPSNR的平均降低量仅为0.076 dB。
文摘Video synopsis is an effective way to easily summarize long-recorded surveillance videos.The omnidirectional view allows the observer to select the desired fields of view(FoV)from the different FoVavailable for spherical surveillance video.By choosing to watch one portion,the observer misses out on the events occurring somewhere else in the spherical scene.This causes the observer to experience fear of missing out(FOMO).Hence,a novel personalized video synopsis approach for the generation of non-spherical videos has been introduced to address this issue.It also includes an action recognition module that makes it easy to display necessary actions by prioritizing them.This work minimizes and maximizes multiple goals such as loss of activity,collision,temporal consistency,length,show,and important action cost respectively.The performance of the proposed framework is evaluated through extensive simulation and compared with the state-of-art video synopsis optimization algorithms.Experimental results suggest that some constraints are better optimized by using the latest metaheuristic optimization algorithms to generate compact personalized synopsis videos from spherical surveillance videos.
文摘360 video streaming services over the network are becoming popular. In particular, it is easy to experience 360 video through the already popular smartphone. However, due to the nature of 360 video, it is difficult to provide stable streaming service in general network environment because the size of data to send is larger than that of conventional video. Also, the real user's viewing area is very small compared to the sending amount. In this paper, we propose a system that can provide high quality 360 video streaming services to the users more efficiently in the cloud. In particular, we propose a streaming system focused on using a head mount display (HMD).
文摘随着虚拟现实技术不断发展,360度视频编码开始成为研究热点。应用于虚拟现实的360度视频与传统的视频相比分辨率更高,编码数据量更大,在实际应用中面临传输带宽有限的瓶颈,编码效率问题有待解决。归纳分析了国际标准组织联合视频研究组(Joint Video Exploration Team,JVET)正在制定的360度视频编码投影变换技术及其编码优化方法,综合对比了各变换技术的编码性能。根据当前最新研究成果,对下一步研究需要解决的问题进行了探讨总结,给出未来的研究方向和思路。
基金funding from the European Union’s Horizon 2020 research and innovation programme,grant n°761934,Hyper 360(“Enriching 360 media with 3D storytelling and personalisation elements”).
文摘We describe four fundamental challenges that complex real-life Virtual Reality (VR) productions are facing today (such as multi-camera management, quality control, automatic annotation with cinematography and 360˚?depth estimation) and describe an integrated solution, called Hyper 360, to address them. We demonstrate our solution and its evaluation in the context of practical productions and present related results.
文摘随着虚拟现实技术的发展,360度视频越来越受欢迎。这些视频在使用标准编码器进行编码之前,要先将其转换为2D图像平面格式。为了提高编码效率,专家们提出了新一代视频编码标准H.266/VVC(Versatile Video Coding),然而,VVC分区模式的多样性导致编码360度高分辨率视频耗时过长。针对上述问题,设计一种CU划分早期决策算法。通过对ERP(Equirectangular projection)视频的统计实验,发现这类视频采用水平分区的概率大于垂直分区。利用经验变差函数设计算法衡量纹理方向差异度,再根据编码单元水平与垂直2个方向的差异程度选择不同的分区。实验结果表明:在全帧内模式下,与VVC测试模型VTM4.0相比,该算法节省了35.42%的编码时间,BD-rate仅增加0.70%。
文摘360° video has been becoming one of the major media in recent years, providing immersive experience for viewers with more interactions compared with traditional videos. Most of today's implementations rely on bulky Head-Mounted Displays (HMDs) or require touch screen operations for interactive display, which are not only expensive but also inconvenient for viewers. In this paper, we demonstrate that interactive 360° video streaming can be done with hints from gaze movement detected by the front camera of today's mobile devices (e.g., a smartphone). We design a lightweight real-time gaze point tracking method for this purpose. We integrate it with streaming module and apply a dynamic margin adaption algorithm to minimize the overall energy consumption for battery-constrained mobile devices. Our experiments on state-of-the-art smartphones show the feasibility of our solution and its energy efficiency toward cost-effective real-time 360° video streaming.
文摘Improving the danger prediction during driving can significantly reduce the risk of accidents.However,previous danger prediction training systems had not been sufficiently effective owing to the lack of realism.In this study,we propose an immersive training system for danger prediction training using virtual reality(VR)technology.This system provides drivers with a highly realistic training environment with 360°videos viewed with VR goggles.Users can practice various dangerous scenarios in an environment that simulates a real-driving situation.In addition,we introduced a system to select dangerous spots with a controller and implement training schemes on a voluntary basis.This setup enables them to train in a highly interactive state.In addition,we proposed a method to express multiple indices numerically so that users can understand the training effect.We tested the effect of the system on the danger prediction abilities of various users with two experiments by using this approach.These results show that our system was more effective in improving the driver’s danger prediction ability than previous systems.
文摘为了提高对360°全景视频的编码效率,联合探索专家组研发了基于HEVC的下一代视频编码标准——多功能视频编码标准(Versatile Video Coding,VVC)。相对于HEVC,VVC具有更高的编码效率,但是也引入了更高的时间复杂度。因此为了降低其编码的计算复杂度,提出了一种针对VVC的帧内模式快速决策算法。通过分析图像块的纹理特性来减少帧内编码的候选模式数量从而减少模式选择中的冗余计算。实验结果表明,提出的算法可以节省24. 08%的编码时间同时只有0. 80%的BDrate损失。
