End-to-End Paired Ambisonic-Binaural Audio Rendering

Binaural rendering is of great interest to virtual reality and immersive media. Although humans can naturally use their two ears to perceive the spatial information contained in sounds, it is a challenging task for machines to achieve binaural rendering since the description of a sound field often requires multiple channels and even the metadata of the sound sources. In addition, the perceived sound varies from person to person even in the same sound field. Previous methods generally rely on individual-dependent head-related transferred function(HRTF)datasets and optimization algorithms that act on HRTFs. In practical applications, there are two major drawbacks to existing methods. The first is a high personalization cost, as traditional methods achieve personalized needs by measuring HRTFs. The second is insufficient accuracy because the optimization goal of traditional methods is to retain another part of information that is more important in perception at the cost of discarding a part of the information. Therefore, it is desirable to develop novel techniques to achieve personalization and accuracy at a low cost. To this end, we focus on the binaural rendering of ambisonic and propose 1) channel-shared encoder and channel-compared attention integrated into neural networks and 2) a loss function quantifying interaural level differences to deal with spatial information. To verify the proposed method, we collect and release the first paired ambisonic-binaural dataset and introduce three metrics to evaluate the content information and spatial information accuracy of the end-to-end methods. Extensive experimental results on the collected dataset demonstrate the superior performance of the proposed method and the shortcomings of previous methods.

经阴道超声联合3D-TVS Render成像诊断子宫内膜病变的临床价值分析

目的分析经阴道超声联合三维超声(3D-TVS)Render成像诊断子宫内膜病变的临床价值。方法回顾性分析2019年11月至2022年11月我院收治的123例子宫内膜病变患者的临床资料,术前或诊刮前均行经阴道超声、3D-TVS Render成像检查,以病理检查结果作为“金标准”,分析经阴道超声联合3D-TVS Render成像对子宫内膜病变的诊断价值。结果经病理检查结果显示,123例患者中子宫内膜癌24例,子宫内膜良性病变99例;子宫内膜癌患者经阴道超声检查的子宫内膜厚度(ET)高于良性病变患者,经3D-TVS Render成像检查的子宫内膜血管指数(VI)、血流指数(FI)、血管血流指数(VFI)高于良性病变患者(P<0.05);绘制ROC曲线结果显示,ET、VI、FI、VFI联合诊断子宫内膜病变的AUC为0.968,高于各指标单独诊断。结论经阴道超声联合3D-TVS Render成像对子宫内膜病变的诊断价值较高,值得临床推广应用。

Research and Application of Caideng Model Rendering Technology for Virtual Reality

With the development of virtual reality (VR) technology, more and more industries are beginning to integrate with VR technology. In response to the problem of not being able to directly render the lighting effect of Caideng in digital Caideng scenes, this article analyzes the lighting model. It combines it with the lighting effect of Caideng scenes to design an optimized lighting model algorithm that fuses the bidirectional transmission distribution function (BTDF) model. This algorithm can efficiently render the lighting effect of Caideng models in a virtual environment. And using image optimization processing methods, the immersive experience effect on the VR is enhanced. Finally, a Caideng roaming interactive system was designed based on this method. The results show that the frame rate of the system is stable during operation, maintained above 60 fps, and has a good immersive experience.

3D-TVS Render成像联合OmniView技术诊断宫腔粘连的价值探究

目的:探究经阴道三维超声(3D-TVS)三维渲染(Render)成像技术与自由解剖(OmniView)成像技术联合指导宫腔粘连(IUA)诊断的价值。方法:将2019年4月-2022年4月宜春市妇幼保健院收治的临床疑诊为IUA,且均接受阴道超声检查的76例患者所有资料进行回顾性分析,检查结果以宫腔镜为准,与3D-TVS Render、OmniView对比,评价两种技术联合使用对IUA的诊断效能。结果:Render技术漏诊18例(18/63,28.57%),误诊3例(3/13,23.08%),其诊断的敏感度为71.43%,特异度为76.92%,阳性预测值为93.75%,阴性预测值为35.71%,准确率为72.37%。OmniView技术漏诊14例(14/63,22.22%),误诊2例(2/13,15.38%),其诊断的敏感度为77.78%,特异度为84.62%,阳性预测值为96.08%,阴性预测值为44.00%,准确率为78.95%。Render联合OmniView技术漏诊6例(6/63,9.52%),误诊1例(1/13,7.69%),其诊断的敏感度为90.48%,特异度为92.31%,阳性预测值为98.28%,阴性预测值为66.67%,准确率为90.79%。Render、OmniView技术与宫腔镜检查的一致性检验的Kappa值分别为0.332、0.457;联合诊断与宫腔镜检查的一致性检验的Kappa值为0.718,一致性水平好。结论:3D-TVS Render成像联合OmniView成像技术诊断IUA准确率较高,且操作简单、无创,具有临床诊断价值。

A survey of real-time rendering on Web3D application

Background In recent years, with the rapid development of mobile Internet and Web3D technologies, a large number of web-based online 3D visualization applications have emerged. Web3D applications, including Web3D online tourism, Web3D online architecture, Web3D online education environment, Web3D online medical care, and Web3D online shopping are examples of these applications that leverage 3D rendering on the web. These applications have pushed the boundaries of traditional web applications that use text, sound, image, video, and 2D animation as their main communication media, and resorted to 3D virtual scenes as the main interaction object, enabling a user experience that delivers a strong sense of immersion. This paper approached the emerging Web3D applications that generate stronger impacts on people's lives through “real-time rendering technology”, which is the core technology of Web3D. This paper discusses all the major 3D graphics APIs of Web3D and the well-known Web3D engines at home and abroad and classify the real-time rendering frameworks of Web3D applications into different categories. Results Finally, this study analyzed the specific demand posed by different fields to Web3D applications by referring to the representative Web3D applications in each particular field. Conclusions Our survey results show that Web3D applications based on real-time rendering have in-depth sectors of society and even family, which is a trend that has influence on every line of industry.

Perceptual Optimization for Point-Based Point Cloud Rendering

Point-based rendering is a common method widely used in point cloud rendering.It realizes rendering by turning the points into the base geometry.The critical step in point-based rendering is to set an appropriate rendering radius for the base geometry,usually calculated using the average Euclidean distance of the N nearest neighboring points to the rendered point.This method effectively reduces the appearance of empty spaces between points in rendering.However,it also causes the problem that the rendering radius of outlier points far away from the central region of the point cloud sequence could be large,which impacts the perceptual quality.To solve the above problem,we propose an algorithm for point-based point cloud rendering through outlier detection to optimize the perceptual quality of rendering.The algorithm determines whether the detected points are outliers using a combination of local and global geometric features.For the detected outliers,the minimum radius is used for rendering.We examine the performance of the proposed method in terms of both objective quality and perceptual quality.The experimental results show that the peak signal-to-noise ratio(PSNR)of the point cloud sequences is improved under all geometric quantization,and the PSNR improvement ratio is more evident in dense point clouds.Specifically,the PSNR of the point cloud sequences is improved by 3.6%on average compared with the original algorithm.The proposed method significantly improves the perceptual quality of the rendered point clouds and the results of ablation studies prove the feasibility and effectiveness of the proposed method.

An Improved HVQ Algorithm for Compression and Rendering of Space Environment Volume Data with Multi-correlated Variables

Volume visualization can not only illustrate overall distribution but also inner structure and it is an important approach for space environment research.Space environment simulation can produce several correlated variables at the same time.However,existing compressed volume rendering methods only consider reducing the redundant information in a single volume of a specific variable,not dealing with the redundant information among these variables.For space environment volume data with multi-correlated variables,based on the HVQ-1d method we propose a further improved HVQ method by compositing variable-specific levels to reduce the redundant information among these variables.The volume data associated with each variable is divided into disjoint blocks of size 43 initially.The blocks are represented as two levels,a mean level and a detail level.The variable-specific mean levels and detail levels are combined respectively to form a larger global mean level and a larger global detail level.To both global levels,a splitting based on a principal component analysis is applied to compute initial codebooks.Then,LBG algorithm is conducted for codebook refinement and quantization.We further take advantage of progressive rendering based on GPU for real-time interactive visualization.Our method has been tested along with HVQ and HVQ-1d on high-energy proton flux volume data,including>5,>10,>30 and>50 MeV integrated proton flux.The results of our experiments prove that the method proposed in this paper pays the least cost of quality at compression,achieves a higher decompression and rendering speed compared with HVQ and provides satisficed fidelity while ensuring interactive rendering speed.

Web-based mixed reality video fusion with remote rendering

Background Mixed reality(MR)video fusion systems merge video imagery with 3D scenes to make the scene more realistic and help users understand the video content and temporal–spatial correlation between them,reducing the user′s cognitive load.MR video fusion are used in various applications;however,video fusion systems require powerful client machines because video streaming delivery,stitching,and rendering are computationally intensive.Moreover,huge bandwidth usage is another critical factor that affects the scalability of video-fusion systems.Methods Our framework proposes a fusion method for dynamically projecting video images into 3D models as textures.Results Several experiments on different metrics demonstrate the effectiveness of the proposed framework.Conclusions The framework proposed in this study can overcome client limitations by utilizing remote rendering.Furthermore,the framework we built is based on browsers.Therefore,the user can test the MR video fusion system with a laptop or tablet without installing any additional plug-ins or application programs.

Image Color Rendering Based on Hinge-Cross-Entropy GAN in Internet of Medical Things

Computer-aided diagnosis based on image color rendering promotes medical image analysis and doctor-patient communication by highlighting important information of medical diagnosis.To overcome the limitations of the color rendering method based on deep learning,such as poor model stability,poor rendering quality,fuzzy boundaries and crossed color boundaries,we propose a novel hinge-cross-entropy generative adversarial network(HCEGAN).The self-attention mechanism was added and improved to focus on the important information of the image.And the hinge-cross-entropy loss function was used to stabilize the training process of GAN models.In this study,we implement the HCEGAN model for image color rendering based on DIV2K and COCO datasets,and evaluate the results using SSIM and PSNR.The experimental results show that the proposed HCEGAN automatically re-renders images,significantly improves the quality of color rendering and greatly improves the stability of prior GAN models.

Fabrication of High Color Rendering Index White Light Emitting Diodes from Gold Nanoclusters

We demonstrated gold nanoclusters as color tunable emissive light converters for the application of white light emitting diodes (WLEDs). A blue LED providing 460 nm to excite gold nanoclusters mixed with UV curable material generates broad bandwidth emission at the visible range. Increasing the amount of gold nanoclusters, the correlated color temperature of WLEDs tuned from cold white to warm white, and also results in the variation of color rendering index (CRI). The highest CRI in the experiment is 92.

A Comparative Study of YīJiàn Xiǎo Shìand A Small Incident Rendered by the Yangs

Lu Xun’s short stories play a prominent role in the history of modern Chinese literature,and a number of Sinologists have studied them and published their English translations.This paper selects“YīJiàn Xiǎo Shì”in Lu Xun’s collection“Call to Arms”as the object of study,taking hypotaxis and parataxis as the starting point,and compares the translation of Yang Hsien-yi and Gladys Yang with the original version,studying the differences between the them in the process of C-E translation.It is hoped that some flexible conversion strategies in English-Chinese translation can be learned from the translation research.

孔子“以直报怨”思想研究——兼论孔子是否反对“以德报怨”

据《论语·宪问》,孔子主张“以直报怨”,而对“以德报怨”则并未明确表示赞成或反对。通过考察《论语》中关于“直”、“怨”的各种论述,结合历代学者的相关观点,可以得出:“以直报怨”中的“直”,应该包含诚实、无私、符合通权达变原则的正直、等值、合乎礼的直率等丰富的含义;“以德报怨”的情况比较复杂,其中有的体现了人们不计前嫌、公而忘私的高尚品德,有的则反映了一些人的虚伪和充满私心的算计,因此,对于“以德报怨”,我们不能简单地予以肯定或否定;“以直报怨”既包含“以德报怨”中值得肯定或提倡的内容,又包含“以德报怨”中所没有的对怨恨对象采取相应报复措施的内容,因此是一种更为全面、合理的报怨方式。

冬奥气象图形产品可视化模型研究

高质量图形产品是冬奥气象服务信息的重要载体。基于冬奥气象数据高时空分辨率特征,研究了由多维气象信息向制图要素转换的冬奥气象图形产品可视化概念模型;设计了由基于数据引擎的结构映射、赛事服务的空间映射、组件应用的元素映射和派生思想的属性映射组成的冬奥气象图形产品可视化过程模型。针对差异化、准实时气象服务制图需求,设计了基于DVDL可视化描述语言的制图表达框架,对可视化模型进行业务组织,研发了冬奥气象图形产品加工系统,提供50余种基础产品样式和所见即所得的制图编辑功能,支持任意位置点长时间序列气象产品、山地赛场三维气象产品和多赛区综合服务气象产品的制图渲染。成果在2022年北京冬奥会气象服务中应用,生产了200余种不用服务主题气象图形产品,后续也可为其他冬季赛事和重大活动气象服务保障提供精细化产品制作解决方案。

一种带高光处理的无缝纹理映射方法

纹理映射作为三维重建的重要步骤,直接关系到生成模型的视觉效果。传统的纹理映射方法通常使用简单的混合方法来获取表面纹理。通过将模型表面投影到每个纹理图像上来获取相应的纹理区域,然后将它们混合在一起以获取表面纹理。然而,由于相机位姿的不准确性,纹理映射结果存在明显的模糊和鬼影等问题。此外,在高光环境下获取的纹理图像很容易包含高光区域,最终导致纹理颜色的丢失,降低纹理的真实性。为了解决这些问题,提出了一种无缝纹理映射方法,可以有效消除高光反射。通过衡量纹理图像的质量,为每个模型表面选择最佳的纹理图像,并利用色度一致性约束优化的相机位姿以消除明显的纹理错位。针对高光问题,提出了一个高光处理模块,使用多视图图像信息并基于双色反射模型定位和处理高光纹理。最后,对纹理图进行纹理颜色一致性调整,以处理纹理之间的颜色差异。实验结果表明,与现有方法相比,提出的算法可以有效消除高光反射的影响,从而获得更好的纹理映射结果。

注意力机制与神经渲染的多视图三维重建算法

针对多视图立体网络在弱纹理或非朗伯曲面等挑战性区域重建效果差的问题,首先提出一个基于3个并行扩展卷积和注意力机制的多尺度特征提取模块,在增加感受野的同时捕获特征之间的依赖关系以获取全局上下文信息,从而提升多视图立体网络在挑战性区域特征的表征能力以进行鲁棒的特征匹配。其次在代价体正则化3D CNN部分引入注意力机制,使网络注意于代价体中的重要区域以进行平滑处理。另外建立一个神经渲染网络,该网络利用渲染参考损失精确地解析辐射场景表达的几何外观信息,并引入深度一致性损失保持多视图立体网络与神经渲染网络之间的几何一致性,有效地缓解有噪声代价体对多视图立体网络的不利影响。该算法在室内DTU数据集中测试,点云重建的完整性和整体性指标分别为0.289和0.326,与基准方法CasMVSNet相比,分别提升24.9%和8.2%,即使在挑战性区域也得到高质量的重建效果;在室外Tanks and Temples中级数据集中,点云重建的平均F-score为60.31,与方法UCS-Net相比提升9.9%,体现出较强的泛化能力。

结合上下文特征融合的虚拟视点图像空洞填充

由于参考纹理视图的前景遮挡和不同视点间的视角差异,基于深度图的虚拟视点合成会产生大量空洞,先前的空洞填充方法耗时较长且填充区域与合成图像缺乏纹理一致性。该文首先对深度图进行预处理来减少空洞填充时的前景渗透;然后,针对经3D-warping后输出合成图像中的空洞,设计了一种基于生成对抗网络(GAN)架构的图像生成网络来填充空洞。该网络模型由2级子网络构成,第1级网络生成空洞区域的纹理结构信息,第2级网络采用了一种结合上下文特征融合的注意力模块来提升空洞填充质量。提出的网络模型能有效解决当虚拟视点图像中的前景对象存在快速运动时,空洞填充区易产生伪影的问题。在多视点深度序列上的实验结果表明,提出方法在主客观质量上均优于已有的虚拟视点图像空洞填充方法。

基于计算机视觉的舰船模型三维重建方法

为保证舰船模型三维重建的完整性以及细节程度,提出基于计算机视觉的舰船模型三维重建方法。基于双目立体视觉计算全局舰船目标点云坐标,生成舰船全局的三维点云数据;多视角点云配准方法对该数据进行配准后,输入多尺度特征递归卷积的稠密点云重建网络模型中,通过该模型生成舰船模型三维深度图,在此基础上,利用运动结构法完成舰船深度图中的三维曲线重建,对舰船模型进行颜色渲染,输出舰船模型三维重建结果。测试结果显示,该方法能够完成不同图像之间对应点的可靠匹配,确定各点的坐标位置;三维重建后模型的空间偏差均在0.015以下;能够较好地完成舰船结构的重建,重建后舰船模型的完整性较好,清晰呈现舰船的结构细节。

基于虚拟现实的虚拟仿真建模及渲染技术

虚拟现实技术能够创造出逼真的虚拟环境,为能够更加快速地建立模型,该文结合相关工具和Unity3D平台开发仿真建模及渲染技术。通过工具收集真实地形的数据,然后进行相关的数据处理并生成数据集合。通过使用Unity3D中地形工具进行数据读取和快速制作三维场景模型,为提高模型的渲染逼真度使用高清渲染管线对场景模型进行光照等渲染处理。通过对该技术的研究,该技术能够使三维场景的逼真度提高及渲染程度非常好,模型细节也能观察,同时该技术也能被运用到毁伤评估仿真、虚拟训练等方面。

虚实共生视域下的职业教育内容呈现模式研究

当今职业院校的学生成长于信息技术高速发展的时代,是同时拥有虚拟空间和现实存在两大生活空间的年轻人。对于他们,职业教育进行虚实共生的内容呈现是可行并必要的。从虚实共生和职业教育的关系出发,分析了以往常见的内容呈现方式的不足,比较了虚实环境各自的优点和缺点,提炼了虚实融合共生的意义所在,并在此基础上分析了几种虚实共生下的有效内容呈现模式,并提出多元化的模式结合呈现模式实现全方位教育,展望了虚实共生下的职业教育内容呈现场景,为虚实融合的教育内容与环境建设提供了思路。

CT用于非人体目标几何体的三维重建

当被扫描物体存在镂空、遮挡和狭小空间时,需要大量图像拍摄或激光扫描工作;利用三维点云构建三角网格时,需要大量人工交互操作,且三维面绘制容易丢失特征信息。为解决上述问题,提出了采用CT对非人体目标几何体进行三维重建,再用体绘制进行显示的方法。基于医用CT对瓷器、陶器、木器和青铜器等材质的工艺品进行三维扫描,直接基于体绘制算法进行三维显示,同时提取CT图像轮廓点实现面绘制建模。结果表明,CT扫描重建精度高、体绘制效果好且人工工作量小、速度快。因此CT扫描及体绘制在非人体目标的三维重建中具有可行性,并且可以提高效率和精度。