Soft transmission of 3D video for low power and low complexity scenario

Thanks to the rapid development of naked-eye 3D and wireless communication technology,3D video related applications on mobile devices have attracted a lot of attention.Nevertheless,the time-varying characteristics of the wireless channel is very challenging for conventional source-channel coding based transmission strategy.Also,the high complexity of source-channel coding based transmission scheme is undesired for low power mobile terminals.An advanced transmission scheme named Softcast was proposed to achieve efficient transmission performance for 2D image/video.Unfortunately,it cannot be directly applied to wireless 3D video transmission with high efficiency.This paper proposes a more efficient soft transmission scheme for 3D video with a graceful quality adaptation within a wide range of channel Signal-to-Noise Ratio(SNR).The proposed method first extends the linear transform to 4 dimensions with additional view dimension to eliminate the view redundancy,and then metadata optimization and chunk interleaving are designed to further improve the transmission performance.Meanwhile,a synthesis distortion based chunk discard strategy is developed to improve the overall 3D video quality under the condition of limited bandwidth.The experimental results demonstrate that the proposed method significantly improves the 3D video transmission performance over the wireless channel for low power and low complexity scenarios.

AVS 3D Video Coding Technology and System

Following the success of the audio video standard （AVS） for 2D video coding, in 2008, the China AVS workgroup started developing 3D video （3DV） coding techniques. In this paper, we discuss the background, technical features, and applications of AVS 3DV coding technology. We introduce two core techniques used in AVS 3DV coding： inter-view prediction and enhanced stereo packing coding. We elaborate on these techniques, which are used in the AVS real-time 3DV encoder. An application of the AVS 3DV coding system is presented to show the great practical value of this system. Simulation results show that the advanced techniques used in AVS 3DV coding provide remarkable coding gain compared with techniques used in a simulcast scheme.

3D Perception Algorithms: Towards Perceptually Driven Compression of 3D Video

In this paper, we summarize 3D perception-oriented algorithms for perceptually driven 3D video coding. Several perceptual ef- fects have been exploited for 2D video viewing; however, this is not yet the case for 3D video viewing. 3D video requires depth perception, which implies binocular effects such as con fl icts, fusion, and rivalry. A better understanding of these effects is necessary for 3D perceptual compression, which provides users with a more comfortable visual experience for video that is de- livered over a channel with limited bandwidth. We present state-of-the-art of 3D visual attention models, 3D just-notice- able difference models, and 3D texture-synthesis models that address 3D human vision issues in 3D video coding and trans-mission.

Dynamic Hyperlinker: Innovative Solution for 3D Video Content Search and Retrieval

Recently, 3D display technology, and content creation tools have been undergone rigorous development and as a result they have been widely adopted by home and professional users. 3D digital repositories are increasing and becoming available ubiquitously. However, searching and visualizing 3D content remains a great challenge. In this paper, we propose and present the development of a novel approach for creating hypervideos, which ease the 3D content search and retrieval. It is called the dynamic hyperlinker for 3D content search and retrieval process. It advances 3D multimedia navigability and searchability by creating dynamic links for selectable and clickable objects in the video scene whilst the user consumes the 3D video clip. The proposed system involves 3D video processing, such as detecting/tracking clickable objects, annotating objects, and metadata engineering including 3D content descriptive protocol. Such system attracts the attention from both home and professional users and more specifically broadcasters and digital content providers. The experiment is conducted on full parallax holoscopic 3D videos “also known as integral images”.

Web-based interactive visualization of 3D video mosaics using X3D standard

We present a method of 3D image mosaicing for real 3D representation of roadside buildings, and implement a Web-based interactive visualization environment for the 3D video mosaics created by 3D image mosaicing. The 3D image mo- saicing technique developed in our previous work is a very powerful method for creating textured 3D-GIS data without excessive data processing like the laser or stereo system. For the Web-based open access to the 3D video mosaics, we build an interactive visualization environment using X3D, the emerging standard of Web 3D. We conduct the data preprocessing for 3D video mosaics and the X3D modeling for textured 3D data. The data preprocessing includes the conversion of each frame of 3D video mosaics into concatenated image files that can be hyperlinked on the Web. The X3D modeling handles the representation of concatenated images using necessary X3D nodes. By employing X3D as the data format for 3D image mosaics, the real 3D representation of roadside buildings is extended to the Web and mobile service systems.

基于智能3D视频教学模式在骨外科学实践教学的应用研究

目的:探索和评价基于智能3D视频教学模式下骨外科学临床实践教学模式的理论基础及应用效果。方法:以右江民族医学院临床2015级1~5班248名实习本科生为试验组,以同年级6~10班242名实习临床本科生为对照组,试验组采用智能3D视频教学授课模式进行临床实践教学,对照组采用常规多媒体教学+模拟手术教学模式;通过同一套试题进行理论考试、技能考核和问卷调查对比两组学生的理论考核成绩、临床思维和实践能力考核和课堂满意度测评的差异,探索和评估两种教学模式在骨科实践教学的应用效果。结果:试验组明显优于对照组,差异比较有统计学意义(P<0.05)。结论:基于智能3D视频教学模式对于提高学生理解和实际临床技能有较好的实践效果,在骨科临床实践教学具有较好的应用前景。

A retargeting method for stereoscopic 3D video

We propose a disparity-constrained retargeting method for stereoscopic 3D video, which simultaneously resizes a binocular video to a new aspect ratio and remaps the depth to the perceptual comfort zone. First, we model distortion energies to prevent important video contents from deforming. Then, to maintain depth mapping stability, we model disparity variation energies to constraint the disparity range both in spatial and temporal domains. The last component of our method is a non-uniform, pixel-wise warp to the target resolution based on these energy models. Using this method, we can process the original stereoscopic video to generate new, high-perceptual-quality versions at different display resolutions. For evaluation, we conduct a user study; we also discuss the performance of our method.

BOPPPS教学模式联合3D胸腔镜培训在胸外科实习教学中的应用研究

目的探讨BOPPPS教学模式联合3D胸腔镜培训在胸外科临床教学中的应用效果。方法纳入2020年3月—2021年3月在西安交通大学第一附属医院胸外科实习的临床医学五年制学生共128人为研究对象,随机分为BOPPPS教学+3D胸腔镜组(试验组)和传统教学+2D胸腔镜组(对照组),每组64人;教学培训3周后,每人完成2份大病历,并行技能考核和教学问卷调查,统计其考核成绩及问卷结果。结果试验组考核总分为(82.0±3.4),高于对照组的(74.0±2.2)分,差异具有统计学意义(P=0.001);试验组在影像读片、外科技能考试得分均高于对照组(P=0.001),但试验组病历书写评分为(14.0±2.9),对照组为(14.0±3.1),差异无统计学意义(P=0.732)。问卷调查结果显示,试验组有93.7%的学生认为BOPPPS+3D胸腔镜教学培训可“提高学习主观能动性”,87.5%的学生认为能“加深理论知识的理解记忆”,而对照组中分别有75.0%和68.7%的学生认为传统教学+2D胸腔镜教学培训也有此作用,试验组认可率高于对照组,差异具有统计学意义(分别为P=0.003和P=0.010)。试验组48.4%的学生认为BOPPPS+3D胸腔镜教学培训“课堂时间充足”,70.3%的学生认为“学习压力大”,而对照组中分别有78.1%和51.5%的学生认为传统教学+2D胸腔镜教学培训也有此作用,对照组比率高于试验组,差异具有统计学意义(分别为P=0.001和P=0.030)。在“学习目标明确”、“学习重点难点突出”和“提高临床分析能力”方面,两组之间的差异无统计学意义(分别为P=0.552、P=0.516和P=0.057)。结论BOPPPS教学模式联合3D胸腔镜培训能激发学生的学习热情,提高教学效率,也能帮助教师改进教学过程,值得推广应用。

Computational approaches for fast generation of digital 3D video holograms(Invited Paper)

Several approaches for fast generation of digital holograms of a three-dimensional （3D） object have been discussed. Among them, the novel look-up table （N-LUT） method is analyzed to dramatically reduce the number of pre-calculated fringe patterns required for computation of digital holograms of a 3D object by employing a new concept of principal fringe patterns, so that problems of computational complexity and huge memory size of the conventional ray-tracing and look-up table methods have been considerably alleviated. Meanwhile, as the 3D video images have a lot of temporally or spatially redundant data in their inter- and intra-frames, computation time of the 3D video holograms could be also reduced just by removing these redundant data. Thus, a couple of computational methods for generation of 3D video holograms by combined use of the N-LUT method and data compression algorithms are also presented and discussed. Some experimental results finally reveal that by using this approach a great reduction of computation time of 3D video holograms could be achieved.

3D视频在医学教育中应用的范围综述

目的3D视频在医学教育中的应用方兴未艾。本研究旨在综述3D视频在医学教育中的应用以及对课程活动的影响,为未来该领域的研究提供参考。方法遵循Arksey和O'malley的范围综述指南,采用系统检索策略检索9个英文文献数据库,仅纳人原始文献。2名作者根据纳人排除标准独立筛选了所有文献,并对最终纳入的原始研究文献进行了分析。结果共检索到1302篇文献,最终纳入了23篇。从文献中获得的3个主题包括:在医学教育中使用3D视频的优势,使用3D视频对学生学业成绩和能力的影响,以及学生关于医学教育中使用3D视频的体验。结论在医学教育中使用3D视频得到了广大学生和教育工作者的支持。然而,3D视频在医学教育中的应用效果仍存在争议。医学教育工作者应结合课程特点、学生学习情况和现有教育资源合理使用。

3D-CTBA联合手术视频在胸外科教学中的应用研究

目的对比三维计算机断层扫描支气管血管成像(three-dimensional computed tomography bronchography and angiography,3D-CTBA)联合胸腔镜手术视频与单纯影像归档和通信系统(picture archiving and communication systems,PACS)联合胸腔镜手术视频在胸外科临床教学中的应用效果。方法选择2021年5—10月在苏州大学附属第一医院胸外科临床实习的102名实习生作为研究对象,将其随机分为教改组及传统组各51名。教改组实施3D-CTBA联合胸腔镜手术视频教学方法,传统组则实施单纯PACS系统联合胸腔镜手术视频教学方法。实习结束后比较两组学生的教学满意度、理论考核和实践操作考核成绩。结果两组实习生理论考核成绩和操作考核成绩及格率比较,教改组的理论考核成绩为(84.14±5.70)分,传统组的理论考核成绩为(83.95±5.26)分,差异无统计学意义(P>0.05)。教改组的实践考核成绩(89.66±4.88)分明显高于传统组(75.19±5.65)分,差异有统计学意义(P<0.05)。结论对比单纯PACS联合胸腔镜手术视频的教学方式,3D-CTBA联合胸腔镜手术视频在胸外科临床实践教学中能更明显提高学生对于肺部解剖的理解,提高其实践操作能力,教学效果满意。

基于注意力机制的3D卷积双波段烟火识别方法

为提高复杂环境下烟火识别的精度,提出一种基于3D卷积和时空注意力机制的双波段烟火识别方法,该方法融合近红外和可见光双波段图像数据,使用视频流中基于时间的动态特征和基于空间的静态特征降低漏报率、误报率。实验结果表明,该算法在双波段数据集上的烟火识别精度达到99.90%,优于其他基于3D卷积的烟火识别算法,同时,模型具有较小的参数量,能够满足实时推理需求。因此,使用双波段特征的同时,结合注意力机制充分利用视频的动态信息,可以有效提高烟火识别精度。

水中目标回声特性测试3D微视频虚拟仿真实验

水声测量技术被广泛应用于海防、油气勘探、地貌成像、探寻沉船等领域,同时是海洋技术、海洋测绘、海洋科学等方向学科人才培养的关键理论和实验教学内容之一。运用SolidWorks软件设计实验系统所需的接收和发射换能,以及不同种类3D虚拟模型,结合V-Ray虚拟仿真渲染、AI配音等多媒体技术搭建虚拟实验场景,以板块元算法和COMSOL声固耦合−频域有限元法理论为基础,完成不同目标散射声场数值仿真计算,模拟3D场景的水中目标声特性测试虚拟仿真实验。虚拟仿真实验通过3D微视频方式展示,不仅可以用来辅助教学,提高学生对水中目标测试技术的深度认知,同时可对普通民众进行声呐技术的科普教育。系统集水声工程科学研究、培育人才和服务社会功能集于一体。研究成果为扩大水声技术影响力和覆盖面发挥作用,为水下目标探测技术创新发展提供参考。

3DQoE评价方法及其模型研究进展综述

随着3D视频技术的逐步发展,3D视频网络服务的用户体验质量(3DQoE)评价逐步成为学术界和工业界都极为关注的热点问题之一.本文通过分析端到端的3DQoE影响因素,总结基于3D视频与图像质量的3DQoE评价方法,综述3DQoE模型的研究进展.通过3DQoE评价与模型研究现状的分析,本文给出了目前3DQoE评价与模型研究过程中存在的精确性,可靠性以及通用性问题,并指出针对大规模网络服务的3DQoE建模是未来可能的研究方向.

3D视频资源:数字化教育资源的新形态

3D视频能够创设最自然直观的学习情景,不仅可以提升学生的学习兴趣,降低学生的认知负荷,而且能促进学生对知识的记忆和理解。文章在阐述3D视频内涵、成像原理及其特征的基础上,探讨了国内外3D视频在教育教学领域中的典型应用以及对学习效果的影响。智慧时代需要提升智慧学习资源,而3D视频为学习者提供逼真的3D内容,使之获得沉浸式的学习体验。在我国推广3D视频资源在教育教学中的应用,具有良好的开拓性和前瞻性。3D视频作为一种新型的学习资源,将成为数字化教育资源的新形态,对于提升教育教学质量大有裨益。

基于3D计算机视觉的鱼类行为分析研究

分析了现有鱼类生物监测系统不能全面精准地检测和描述鱼类的空间位置的缺陷,设计了一种基于3D计算机视觉的鱼类行为分析系统,以实现从三维视角对鱼类进行监测并分析鱼类的运动行为。该系统首先通过自组织背景建模方法对图像中鱼的前景进行提取,然后计算出鱼的质心位置,并由3D视觉装置的几何关系计算鱼在实际三维空间中的位置,通过跟踪得到鱼类运动的三维轨迹,最后根据鱼类运动轨迹数据计算出鱼的游动的速度和角速度,得到鱼类运动行为模式曲线。实验结果表明利用3D计算机视觉对鱼类行为进行跟踪与描述,能为鱼类活动量水平变化及鱼类活动模式改变的定量分析奠定坚实的基础。

双视点3D视频文件的裸眼立体组合投影实时显示算法

基于Client/Server结构和sort-last并行绘制策略,提出双视点3D视频文件的裸眼立体组合投影实时显示算法.首先在服务端全屏播放左右或上下格式的3D视频文件,以不低于25帧/s的速率在线截屏并按JPEG格式压缩后转发给12个Client PC.每个Client PC接收每一帧截屏图像后,利用Fragment Shader和多渲染目标(MRT)通过一遍绘制完成2个单视点子图像的裁剪、缩放、奇偶条纹倾斜绘制,经几何和亮度校正,并将2个子图像交织后再向前投影到光栅显示屏幕.该屏幕的投影表面为3.6 m×1.6 m,单台投影仪分辨率为1024×768,投影系统的分辨率为3584×1536.实验结果表明,该算法的显示帧率≥24帧/s,且当条纹倾斜角度为10°时裸眼立体显示效果最好.

3D单孔腹腔镜在泌尿外科手术中应用的单中心研究

目的报道单中心开展的20例3D单孔腹腔镜手术,探讨该手术的可行性和安全性,总结操作经验,并评价其临床应用特点和价值。方法收集2013年8月至2014年11月长海医院泌尿外科完成的20例3D单孔腹腔镜手术患者的临床资料和围手术期数据。20例患者中男性9例,女性11例。年龄31岁至69岁,平均55.8岁,平均体质指数24.64 kg/m^2。其中经腹腔入路14例,经腹膜后入路6例。9例肾上腺切除术,5例肾癌根治术,4例无功能肾切除术,1例肾部分切除术,1例腹膜后囊肿切除术。分析患者围手术期情况,手术时间、出血量、手术并发症和手术中转情况。结果 20例患者均在不增加任何额外切口的情况下顺利完成手术,无中转开放。在3D单孔腹腔镜下共完成5种类型手术,其中有肾上腺切除术、肾癌根治术和无功能肾切除术分别进行经腹腔镜入路和经腹膜后入路进行。本组20例3D单孔腹腔镜手术均未出现外科并发症。20例手术平均时间为156.3±56.1 min,患者术后(4.5±1.6)d出院。术后第3天,15例患者疼痛评分为0分,3例为1分,2例为2分。结论初步经验表明:3D单孔腹腔镜下手术安全、可行、有效。3D腹腔镜由于手术图像立体感强,手术操作精确度高,手眼协调难度明显降低,能降低单孔腹腔镜手术操作的难度,具有良好的临床应用前景。目前3D单孔腹腔镜完成的手术例数较少,其长期临床疗效尚需前瞻性大样本多中心临床随机对照研究进一步证实。

裸眼3D视频的教育应用研究

在介绍裸眼3D视频相关概念的基础上,简介了4种常见3D显示技术的教育应用特点,并通过设计实验和分析美国两个典型的裸眼3D视频教学实验案例,发现裸眼3D视频在创设身临其境的学习环境和提高学生在教学过程中的参与度方面有较好的成效,进而总结了裸眼3D视频应用于教育的优势,论证了裸眼3D视频在教育中应用的价值所在。

基于跨层码率适配和差错控制的3D 视频无线传输方法

针对纹理视频加深度序列的3D视频无线传输,本文提出一种基于跨层优化的码率适配和差错控制方法.通过最小化端到端3D视频失真,均衡调整和配置应用层3D视频编码的码率和帧内编码更新比例,以及物理层的调制和编码模式,达到信源码率适配信道带宽以及应用层差错控制和物理层信道保护强度相互平衡的目的,进而提高接收端的3D虚拟视点视频质量.实验结果表明,提出的方法能有效的提高3D视频无线传输的性能.