Spatio-Temporal Context-Guided Algorithm for Lossless Point Cloud Geometry Compression

Point cloud compression is critical to deploy 3D representation of the physical world such as 3D immersive telepresence,autonomous driving,and cultural heritage preservation.However,point cloud data are distributed irregularly and discontinuously in spatial and temporal domains,where redundant unoccupied voxels and weak correlations in 3D space make achieving efficient compression a challenging problem.In this paper,we propose a spatio-temporal context-guided algorithm for lossless point cloud geometry compression.The proposed scheme starts with dividing the point cloud into sliced layers of unit thickness along the longest axis.Then,it introduces a prediction method where both intraframe and inter-frame point clouds are available,by determining correspondences between adjacent layers and estimating the shortest path using the travelling salesman algorithm.Finally,the few prediction residual is efficiently compressed with optimal context-guided and adaptive fastmode arithmetic coding techniques.Experiments prove that the proposed method can effectively achieve low bit rate lossless compression of point cloud geometric information,and is suitable for 3D point cloud compression applicable to various types of scenes.

Lossy Point Cloud Attribute Compression with Subnode-Based Prediction

Recent years have witnessed that 3D point cloud compression(PCC)has become a research hotspot both in academia and industry.Especially in industry,the Moving Picture Expert Group(MPEG)has actively initiated the development of PCC standards.One of the adopted frameworks called geometry-based PCC(G-PCC)follows the architecture of coding geometry first and then coding attributes,where the region adaptive hierarchical transform(RAHT)method is introduced for the lossy attribute compression.The upsampled transform domain prediction in RAHT does not sufficiently explore the attribute correlations between neighbor nodes and thus fails to further reduce the attribute redundancy between neighbor nodes.In this paper,we propose a subnode-based prediction method,where the spatial position relationship between neighbor nodes is fully considered and prediction precision is further promoted.We utilize some already-encoded neighbor nodes to facilitate the upsampled transform domain prediction in RAHT by means of a weighted average strategy.Experimental results have illustrated that our proposed attribute compression method shows better rate-distortion(R-D)performance than the latest MPEG G-PCC(both on reference software TMC13-v22.0 and GeS-TM-v2.0).

Introduction to Point Cloud Compression

Characterized by geometry and photometry attributes, point cloud has been widely applied in the immersive services of various 3Dobjects and scenes. The development of even more precise capture devices and the increasing requirements for vivid rendering in-evitably induce huge point capacity, thus making the point cloud compression a demanding issue. In this paper, we introduce sev-eral well-known compression algorithms in the research area as well as the boosting industry standardization works. Specifically,based on various applications of this 3D data, we summarize the static and dynamic point cloud compression, both including irreg-ular geometry and photometry information that represent the spatial structure information and corresponding attributes, respective-ly. In the end, we conclude the point cloud compression as a promising topic and discuss trends for future works.

Portable Perceptron Network-Based Fast Mode Decision for Video-Based Point Cloud Compression

In Video-based Point Cloud Compression(V-PCC),2D videos to be encoded are generated by 3D point cloud projection,and compressed by High Efficiency Video Coding(HEVC).In the process of 2D video compression,the best mode of Coding Unit(CU)is searched by brute-force strategy,which greatly increases the complexity of the encoding process.To address this issue,we first propose a simple and effective Portable Perceptron Network(PPN)-based fast mode decision method for V-PCC under Random Access(RA)configuration.Second,we extract seven simple hand-extracted features for input into the PPN network.Third,we design an adaptive loss function,which can calculate the loss by allocating different weights according to different Rate-Distortion(RD)costs,to train our PPN network.Finally,experimental results show that the proposed method can save encoding complexity of 43.13%with almost no encoding efficiency loss under RA configuration,which is superior to the state-of-the-art methods.The source code is available at https://github.com/Mesks/PPNforV-PCC.

基于场景流的可变速率动态点云压缩

针对现有的动态点云压缩神经网络需要训练多个网络模型的问题,提出基于场景流的可变速率动态点云压缩网络框架.网络以原始动态点云为输入,利用场景流网络进行运动向量估计,在压缩运动向量和残差的同时,引入通道增益模块对隐向量通道进行评估和缩放,实现可变速率控制.通过综合考虑运动向量损失和率失真损失,设计新的联合训练损失函数,用来端到端地训练整个网络框架.为了解决动态点云数据集缺少真实运动信息标签的问题,基于AMASS数据集制作带有运动向量标签的人体数据集,用于网络的训练.实验结果显示,与现有的基于深度学习动态点云压缩方法相比,该方法的压缩比特率下降了几个数量级,与静态压缩网络单独处理每帧的重构效果相比,该方法有5%~10%的提升.

基于三维点云的光栅三维显示景深优化方法

对三维显示内容进行深度压缩处理,是一种针对裸眼光栅三维显示器景深范围不足问题的有效解决方法,但常用的压缩方法会使场景中的主体不可避免地产生几何形变。本文面向密集视点光栅三维显示,提出了一种基于三维点云的景深优化方法,根据三维场景的双目视差图重建出三维点云,将场景主体点云分割,仅对超出显示器景深范围的主体进行深度位置调整,保持场景主体的几何结构不变,从而实现三维场景深度的整体压缩。对本文方法进行关于视觉体验的多项统计实验,对于每个参与者,本文方法处理后结果得票率均超过77%;对于每个实验场景,本文方法得票率均超过80%。实验结果证明了本文方法对观众主观感受有明显提升,可以在压缩场景整体深度的同时保证场景主体没有形变,让观众在观看三维场景时没有不自然感且对场景主体保留较强的深度感知。

基于Transformer的点云几何有损压缩方法

点云被广泛地用于三维物体表达,不过真实世界采集到的点云往往数据庞大,不利于传输与储存,针对点云数据冗余性问题,引入基于注意力机制的Transformer模块,提出一种基于Transformer的端到端多尺度点云几何压缩方法。将点云进行体素化,在编码端利用稀疏卷积提取特征,进行多尺度的逐步下采样,结合Transformer模块加强点空间特征感知与提取;在解码端进行对应的多尺度上采样重建,同样采用Transformer模块对有用特征进行加强与恢复,逐步细化并重建点云。与2种点云标准编码方法对比,所提方法平均获得80%和75%的BD-Rate增益;与基于深度学习的点云压缩方法对比,平均获得16%的BD-Rate增益,在相同码率点有约0.6的PSNR提升。实验结果表明:Transformer在点云压缩领域的可行性与有效性;在主观质量方面,所提方法也有明显的主观效果提升,重建的点云更接近原始点云。

基于加权损失的点云占用图视频上采样

基于视频的点云压缩标准(Video-based Point Cloud Compression,V-PCC)中,3D点云会被分成数百个块并投影到2D平面中,形成记录点云纹理信息的纹理视频和记录点云空间信息的几何视频。同时,还需要生成一个占用图视频(Occupancy Map Video),以记录纹理视频和几何视频中每一个像素点是否对应重建点云中的某个点。因此,占用图视频质量与重建点云质量直接相关。为了节约编码比特数,占用图视频在编码端会先被下采样,然后在解码端通过简单的上采样恢复到原分辨率。文中的基本思路是引入深度学习来代替V-PCC中的简单上采样方法,使得上采样后的占用图视频质量更高,从而提高点云的重建质量。在网络训练阶段提出使用加权损失函数,使得在重建点云时能尽可能少地移除正常点并尽可能多地移除噪声点。实验结果证明,所提方法可以大幅提升V-PCC的主客观性能。

基于多尺度特征和注意力机制的深度学习点云压缩

三维点云广泛应用于无人驾驶、实景三维等领域,然而复杂场景的海量点云对存储、处理和传输等带来极大挑战。提出一种基于多尺度特征和注意力机制的深度变分自编码点云几何信息压缩算法MSA-GPCC,通过加入多尺度模型提取特征、变分自编码器构建熵模型,继而结合尺度注意力模块和多尺度特征,实现基于熵编码的点云几何信息高码率、低失真压缩。在MPEG数据集上进行的实验表明,相比G-PCC算法和基于深度学习的D-PCC算法,MSA-GPCC算法在点间等比特率下平均质量增益分别提升7.72和4.91 dB,点到面等比特率下平均质量增益分别提升5.56和3.09 dB。

A Fast Compression Framework Based on 3D Point Cloud Data for Telepresence

In this paper,a novel compression framework based on 3D point cloud data is proposed for telepresence,which consists of two parts.One is implemented to remove the spatial redundancy,i.e.,a robust Bayesian framework is designed to track the human motion and the 3D point cloud data of the human body is acquired by using the tracking 2D box.The other part is applied to remove the temporal redundancy of the 3D point cloud data.The temporal redundancy between point clouds is removed by using the motion vector,i.e.,the most similar cluster in the previous frame is found for the cluster in the current frame by comparing the cluster feature and the cluster in the current frame is replaced by the motion vector for compressing the current frame.The hrst,the B-SHOT(binary signatures of histograms orientation)descriptor is applied to represent the point feature for matching the corresponding point between two frames.The second,the K-mean algorithm is used to generate the cluster because there are a lot of unsuccessfully matched points in the current frame.The matching operation is exploited to find the corresponding clusters between the point cloud data of two frames.Finally,the cluster information in the current frame is replaced by the motion vector for compressing the current frame and the unsuccessfully matched clusters in the curren t and the motion vectors are transmit ted into the rem ote end.In order to reduce calculation time of the B-SHOT descriptor,we introduce an octree structure into the B-SHOT descriptor.In particular,in order to improve the robustness of the matching operation,we design the cluster feature to estimate the similarity bet ween two clusters.Experimen tai results have shown the bet ter performance of the proposed method due to the lower calculation time and the higher compression ratio.The proposed met hod achieves the compression ratio of 8.42 and the delay time of 1228 ms compared with the compression ratio of 5.99 and the delay time of 2163 ms in the octree-based compression method under conditions of similar distortion rate.

自适应分割的视频点云多模式帧间编码方法

基于视频的点云压缩(Video based point cloud compression, V-PCC)为压缩动态点云提供了高效的解决方案,但V-PCC从三维到二维的投影使得三维帧间运动的相关性被破坏,降低了帧间编码性能.针对这一问题,提出一种基于V-PCC改进的自适应分割的视频点云多模式帧间编码方法,并依此设计了一种新型动态点云帧间编码框架.首先,为实现更精准的块预测,提出区域自适应分割的块匹配方法以寻找最佳匹配块;其次,为进一步提高帧间编码性能,提出基于联合属性率失真优化(Rate distortion optimization, RDO)的多模式帧间编码方法,以更好地提高预测精度和降低码率消耗.实验结果表明,提出的改进算法相较于V-PCC实现了-22.57%的BD-BR (Bjontegaard delta bit rate)增益.该算法特别适用于视频监控和视频会议等帧间变化不大的动态点云场景.

基于激光点云的预制混凝土箱梁外观尺寸检测方法研究

为精确评估复杂施工环境下具有弯扭变形的预制混凝土箱梁的外观尺寸制作精度,基于三维激光点云提出预制混凝土箱梁的外观尺寸和整体变形全流程检测方法。该方法针对扫描点云的非均匀性,提出了点云混合压缩算法,较好地保护了曲率特征和低密度区点云。通过点云边界识别、点云面域分割提取特征线和特征面;对实际扫描中点云缺失面进行填充;采用切片投影法提取构件局部断面点云以进行尺寸计算。以某单跨高架桥8片预制混凝土箱梁为检测对象,针对7个典型尺寸检测项目,计算分析了其点云计算值和现场测量值的绝对差值,依据《公路工程质量检验评定标准第一册土建工程》(JTG F80/1—2017)评估了箱梁整体尺寸的制作精度,运用统计学方法评估整体变形形态。结果表明:箱梁7个尺寸检测项目共132组对比数据中80.3%的绝对差值保持在10 mm以内,验证了本文点云特征提取及尺寸计算方法具有较高精度;箱梁整体尺寸制作质量基本合格;箱梁上拱线形更接近三次多项式曲线,腹板相对扭转角保持在2°以内,表明箱梁并未产生过大的扭转变形。

一种基于动态点云的三维监控与压缩系统

动态点云能准确表达三维空间位置关系,相较于二维影像,在目标检测、人脸识别以及可视化等方面具有更好的表现,因此动态点云在视频监控领域具有较大应用前景。基于所提出的改进的动态点云编解码框架,实现一种基于动态点云的三维实时监控与压缩系统。首先,通过ZED 2i双目相机进行点云视频获取,以Jetson Nano作为数据处理器,应用基于统计学的滤波算法实现离群点与噪声的去除。其次,依据监控场景的静动特性进行前后景分割,分别应用提出的改进算法和PCL库的压缩算法进行编码。实验表明,在监控场景下,获取的动态点云序列取得了较好的主观效果的同时,实现了点云数据的高效压缩。

采用SJND模型的动态点云感知编码方法

动态点云能有效描述自然场景与3D对象,提供沉浸式视觉体验;但其数据量庞大。需对其进行有效压缩。提出了采用显著性引导的恰可察觉失真(Saliency-guided Just Noticeable Distortion,SJND)模型的动态点云感知编码方法。针对纹理图感知冗余,构建了基于离散余弦变换域的SJND模型,应用于纹理图编码过程中的DCT系数抑制;考虑到相同失真等级下显著区域的几何失真更易被察觉,提出使用投影显著图将几何图进行分层;最后,为不同层级的编码树单元进行自适应量化参数选择和编码。与V-PCC标准方法相比,在保证动态点云视觉质量的前提下,所提出方法提升了动态点云的编码效率。

八叉树结构下动态点云的无损几何编码方法

为进一步提高动态点云无损压缩的性能,提出了一种八叉树结构下的几何信息熵编码方法。针对帧内空间相关性,利用当前八叉树节点的已编解码邻域信息,建立帧内邻居节点上下文和帧内邻居父节点上下文。针对帧间时间相关性,将已编解码的上一帧点云作为参考帧,并将参考帧中与当前八叉树节点同位置的八叉树节点作为参考节点。使用参考节点及其父节点进行帧间上下文建模。为充分利用已建模的上下文,并准确地估计不同上下文下当前节点为非空的条件概率,提出了一种基于指数移动平均的二级概率估计方法:分别在帧内邻居节点上下文和帧内邻居父节点上下文下进行概率估计;利用概率估计结果和帧间上下文建模新的二级上下文,并在二级上下文下再次进行概率估计;采用二进制算术编码器实现无损压缩。选取常用的微软体素化人物上半身和8i体素化人物全身数据集进行性能测试,实验结果表明:与近年的方法相比,所提方法的无损压缩性能更高,平均编码增益达到2.2%~28.7%。

一种面向环境扫描的彩色点云流实时压缩算法

随着数字孪生技术的不断发展,人们面临将大规模、高维度的点云数据进行压缩编码的问题,以提高传输效率。然而大多点云编码方式存在压缩实时性不强、压缩效率低、点云格式要求过高等问题。针对解决这些问题,提出了一种基于谷歌Draco几何压缩库的实时彩色点云流压缩(Real-time Color Stream Draco,RCS-Draco)算法。将算法集成到ROS框架内,借助ROS消息流,对点云流实时地进行编码和解码,提高了算法的实时性;通过建立优化裁剪模型,对点云进行裁剪和滤波,去除了漂移和离群点云,提高了压缩算法的压缩效率。建立量化预测模型,对点云的RGB颜色信息进行编码,解决了大多数点云压缩算法无法处理颜色信息的问题。对比试验通过调整压缩等级和量化参数,证明RCS-Draco算法的平均压缩率最高能够达到77%、平均压缩和解压时间小于0.035 s、位置平均误差小于0.05 m、属性平均误差小于35;并通过相融试验证明RCS-Draco算法在各项指标上优于Draco算法。试验结果表明,RCS-Draco压缩算法在压缩实时性、效率、点云格式方面均表现良好,能够有效地提升传输效率。

基于混合上下文熵模型的点云几何编码算法

针对三维点云存在的大量空域冗余信息,提出一种基于混合上下文熵模型的点云几何编码算法框架.通过多层感知机与Resnet网络分别对基于八叉树结构的点云和基于体素结构的点云特征进行上下文特征提取,并使用选择单元对上下文信息进行裁剪、选择和融合,使网络能够针对当前编码体素建立更加准确的概率模型,从而提高三维点云的压缩效果.同时,针对模型复杂度高的问题提出并行多尺度自回归进行概率估计的方案,大大降低了编解码时间.实验结果表明:点云几何编码算法能够有效降低每个体素所占的比特数并且整个编码过程无损;与G-PCC编码算法相比,压缩后比特率下降了14.27%.

改进的多尺度点云自编码器网络

三维数据的应用场景持续发展,对点云数据高效传输的需求不断增加。然而,如何压缩稀疏、非结构化、高精度的三维点以实现高效通信,仍是一个具有挑战性的问题。为此,通过改进多尺度自编码网络框架以优化局部特征提取,控制模型复杂度和计算开销;通过引入多尺度特征提取块,将潜在表征自适应地分解为多个不同尺度的局部特征;通过残差级联结构减小全局特征损失,实现特征的多尺度融合。

一种融合曲度的自适应八叉树点云压缩算法

针对传统点云压缩算法压缩精度和压缩速度不均衡的问题,提出了一种点云曲度与八叉树结合的点云压缩算法,利用点云曲度能够精确表示点云表面特征的特性和八叉树快速索引的特性,在保证点云压缩精度的同时提高压缩速度,较好地解决了点云压缩精度和速度不均衡问题。采用兔子点云数据进行实验,结果表明:本算法既在保证压缩精度的同时,又能够在一定程度上缩短压缩时间,特别是针对海量点云数据的压缩更能够体现本算法压缩精度和压缩速度的优势。

激光测距前沿回波大数据点云压缩方法

点云数据压缩时,外边界点易丢失,影响后续点云压缩品质。为此,提出激光测距前沿回波大数据点云压缩方法。首先通过主回波匹配、回波信号识别、多项式拟合滤波、时间误差修正等操作预处理激光测距前沿回波大数据;然后采用主成分分析法获取数据法矢,再分别提取边界特征点和尖锐特征点,保留点云模型完整边界;最后利用SIFT算法压缩点云数据并保存,完成激光测距前沿回波大数据的点云压缩。实验结果表明,所提方法能够提高压缩率、缩短压缩时间、降低压缩前后表面积比。