Transforming Education with Photogrammetry:Creating Realistic 3D Objects for Augmented Reality Applications

Augmented reality(AR)is an emerging dynamic technology that effectively supports education across different levels.The increased use of mobile devices has an even greater impact.As the demand for AR applications in education continues to increase,educators actively seek innovative and immersive methods to engage students in learning.However,exploring these possibilities also entails identifying and overcoming existing barriers to optimal educational integration.Concurrently,this surge in demand has prompted the identification of specific barriers,one of which is three-dimensional(3D)modeling.Creating 3D objects for augmented reality education applications can be challenging and time-consuming for the educators.To address this,we have developed a pipeline that creates realistic 3D objects from the two-dimensional(2D)photograph.Applications for augmented and virtual reality can then utilize these created 3D objects.We evaluated the proposed pipeline based on the usability of the 3D object and performance metrics.Quantitatively,with 117 respondents,the co-creation team was surveyed with openended questions to evaluate the precision of the 3D object created by the proposed photogrammetry pipeline.We analyzed the survey data using descriptive-analytical methods and found that the proposed pipeline produces 3D models that are positively accurate when compared to real-world objects,with an average mean score above 8.This study adds new knowledge in creating 3D objects for augmented reality applications by using the photogrammetry technique;finally,it discusses potential problems and future research directions for 3D objects in the education sector.

Rail-Pillar Net:A 3D Detection Network for Railway Foreign Object Based on LiDAR

Aiming at the limitations of the existing railway foreign object detection methods based on two-dimensional(2D)images,such as short detection distance,strong influence of environment and lack of distance information,we propose Rail-PillarNet,a three-dimensional(3D)LIDAR(Light Detection and Ranging)railway foreign object detection method based on the improvement of PointPillars.Firstly,the parallel attention pillar encoder(PAPE)is designed to fully extract the features of the pillars and alleviate the problem of local fine-grained information loss in PointPillars pillars encoder.Secondly,a fine backbone network is designed to improve the feature extraction capability of the network by combining the coding characteristics of LIDAR point cloud feature and residual structure.Finally,the initial weight parameters of the model were optimised by the transfer learning training method to further improve accuracy.The experimental results on the OSDaR23 dataset show that the average accuracy of Rail-PillarNet reaches 58.51%,which is higher than most mainstream models,and the number of parameters is 5.49 M.Compared with PointPillars,the accuracy of each target is improved by 10.94%,3.53%,16.96%and 19.90%,respectively,and the number of parameters only increases by 0.64M,which achieves a balance between the number of parameters and accuracy.

Depth-Guided Vision Transformer With Normalizing Flows for Monocular 3D Object Detection

Monocular 3D object detection is challenging due to the lack of accurate depth information.Some methods estimate the pixel-wise depth maps from off-the-shelf depth estimators and then use them as an additional input to augment the RGB images.Depth-based methods attempt to convert estimated depth maps to pseudo-LiDAR and then use LiDAR-based object detectors or focus on the perspective of image and depth fusion learning.However,they demonstrate limited performance and efficiency as a result of depth inaccuracy and complex fusion mode with convolutions.Different from these approaches,our proposed depth-guided vision transformer with a normalizing flows(NF-DVT)network uses normalizing flows to build priors in depth maps to achieve more accurate depth information.Then we develop a novel Swin-Transformer-based backbone with a fusion module to process RGB image patches and depth map patches with two separate branches and fuse them using cross-attention to exchange information with each other.Furthermore,with the help of pixel-wise relative depth values in depth maps,we develop new relative position embeddings in the cross-attention mechanism to capture more accurate sequence ordering of input tokens.Our method is the first Swin-Transformer-based backbone architecture for monocular 3D object detection.The experimental results on the KITTI and the challenging Waymo Open datasets show the effectiveness of our proposed method and superior performance over previous counterparts.

A new method of mesh simplification for 3-Dimension terrain using Laplace operator

This paper proposes a new method to simplify mesh in 3D terrain. The 3D terrain is presented by digital elevation model. First, Laplace operator is introduced to calculate sharp degree of mesh point, which indicates the variation trend of the terrain. Through setting a critical value of sharp degree, feature points are selected. Second, critical mesh points are extracted by an recursive process, and constitute the simplified mesh. Third, the algorithm of linear-square interpolation is employed to restore the characteris- tics of the terrain. Last, the terrain is rendered with color and texture. The experimental results demonstrate that this method can compress data by 16% and the error is lower than 10%.

Algorithm and System of Scanning Color 3D Objects

This paper presents a complete system for scanning the geometry and texture of a large 3D object, then the automatic registration is performed to obtain a whole realistic 3D model. This system is composed of one line strip laser and one color CCD camera. The scanned object is pictured twice by a color CCD camera. First, the texture of the scanned object is taken by a color CCD camera. Then the 3D information of the scanned object is obtained from laser plane equations. This paper presents a practical way to implement the three dimensional measuring method and the automatic registration of a large 3D object and a pretty good result is obtained after experiment verification.

基于Stingray Objective Studio的防空C^3I显控系统界面开发

文中提出了用 Stingray Objective Studio 工具集来进行防空显控系统界面开发,使得开发人员将大量的时间和精力用于系统结构和数据处理算法上,这样不光降低了开发难度,也增强了系统的功能和稳定性。

Part-level 3-D object classification with improved interpretation tree

For classifying unknown 3-D objects into a set of predetermined object classes, a part-level object classification method based on the improved interpretation tree is presented. The part-level representation is implemented, which enables a more compact shape description of 3-D objects. The proposed classification method consists of two key processing stages： the improved constrained search on an interpretation tree and the following shape similarity measure computation. By the classification method, both whole match and partial match with shape similarity ranks are achieved; especially, focus match can be accomplished, where different key parts may be labeled and all the matched models containing corresponding key parts may be obtained. A series of experiments show the effectiveness of the presented 3-D object classification method.

NUMERICAL MODELLING OF THREE-DIMENSION CHARACTERISTICS OF WIND-DRIVEN CURRENT IN THE BOHAI SEA

Three- dimension (3-D) wind-driven currents in the Bohai Sea in both winter and summer are calculated by using a 3- D barotropic steady model, and the results are consistent with observed flow char -acteristics. Based on the results, 3- D characteristics of flow, currents at different depths, compensated flow in the lower layer , long and narrow alongshore current, the area of upwelling and downwelling, main circulation in vertical profile, and the current in Bohai Strait are discussed.

CurveNet:Curvature-Based Multitask Learning Deep Networks for 3D Object Recognition

In computer vision fields,3D object recognition is one of the most important tasks for many real-world applications.Three-dimensional convolutional neural networks(CNNs)have demonstrated their advantages in 3D object recognition.In this paper,we propose to use the principal curvature directions of 3D objects(using a CAD model)to represent the geometric features as inputs for the 3D CNN.Our framework,namely CurveNet,learns perceptually relevant salient features and predicts object class labels.Curvature directions incorporate complex surface information of a 3D object,which helps our framework to produce more precise and discriminative features for object recognition.Multitask learning is inspired by sharing features between two related tasks,where we consider pose classification as an auxiliary task to enable our CurveNet to better generalize object label classification.Experimental results show that our proposed framework using curvature vectors performs better than voxels as an input for 3D object classification.We further improved the performance of CurveNet by combining two networks with both curvature direction and voxels of a 3D object as the inputs.A Cross-Stitch module was adopted to learn effective shared features across multiple representations.We evaluated our methods using three publicly available datasets and achieved competitive performance in the 3D object recognition task.

General and robust voxel feature learning with Transformer for 3D object detection

The self-attention networks and Transformer have dominated machine translation and natural language processing fields,and shown great potential in image vision tasks such as image classification and object detection.Inspired by the great progress of Transformer,we propose a novel general and robust voxel feature encoder for 3D object detection based on the traditional Transformer.We first investigate the permutation invariance of sequence data of the self-attention and apply it to point cloud processing.Then we construct a voxel feature layer based on the self-attention to adaptively learn local and robust context of a voxel according to the spatial relationship and context information exchanging between all points within the voxel.Lastly,we construct a general voxel feature learning framework with the voxel feature layer as the core for 3D object detection.The voxel feature with Transformer(VFT)can be plugged into any other voxel-based 3D object detection framework easily,and serves as the backbone for voxel feature extractor.Experiments results on the KITTI dataset demonstrate that our method achieves the state-of-the-art performance on 3D object detection.

THREE-IMAGE MATCHING FOR 3-D LINEAR OBJECT TRACKING

This paper will discuss strategies for trinocular image rectification and matching for linear object tracking.It is well known that a pair of stereo images generates two epipolar images.Three overlapped images can yield six epipolar images in situations where any two are required to be rectified for the purpose of image matching.In this case,the search for feature correspondences is computationally intensive and matching complexity increases.A special epipolar image rectification for three stereo images,which simplifies the image matching process,is therefore proposed.This method generates only three rectified images,with the result that the search for matching features becomes more straightforward.With the three rectified images,a particular line_segment_based correspondence strategy is suggested.The primary characteristics of the feature correspondence strategy include application of specific epipolar geometric constraints and reference to three_ray triangulation residuals in object space.

Exploring Local Regularities for 3D Object Recognition

In order to find better simplicity measurements for 3D object recognition, a new set of local regularities is developed and tested in a stepwise 3D reconstruction method, including localized minimizing standard deviation of angles（L-MSDA）, localized minimizing standard deviation of segment magnitudes（L-MSDSM）, localized minimum standard deviation of areas of child faces （L-MSDAF）, localized minimum sum of segment magnitudes of common edges （L-MSSM）, and localized minimum sum of areas of child face （L-MSAF）. Based on their effectiveness measurements in terms of form and size distortions, it is found that when two local regularities： L-MSDA and L-MSDSM are combined together, they can produce better performance. In addition, the best weightings for them to work together are identified as 10% for L-MSDSM and 90% for L-MSDA. The test results show that the combined usage of L-MSDA and L-MSDSM with identified weightings has a potential to be applied in other optimization based 3D recognition methods to improve their efficacy and robustness.

SQL3 Object Model and Its Extension

As the latest version of relational database standard, SQL3 not only has been extended with many new relational features but also added with the object-oriented technologies. This paper introduces the object-oriented features of SQL3 and then extends it with object deputy model to support object view mechanisms. Key words SQL3 - object-relational database - object deputy model - inheritance CLC number TP 392 Foundation item: Supported by the National Natural Science Foundation of China (60273072), 863 Foundation of China (2002AA4Z3450), Research Fund for the Doctoral Program of Higher Education(20010486029) and Hubei Natural Science Fund for Distinguished Youth(2002AC003)Biography: ZHUANG Ji-feng (1979-), male, Master, research direction: database theory.

Securing Copyright Using 3D Objects Blind Watermarking Scheme

Recently,securing Copyright has become a hot research topic due to rapidly advancing information technology.As a host cover,watermarking methods are used to conceal or embed sensitive information messages in such a manner that it was undetectable to a human observer in contemporary times.Digital media covers may often take any form,including audio,video,photos,even DNA data sequences.In this work,we present a new methodology for watermarking to hide secret data into 3-D objects.The technique of blind extraction based on reversing the steps of the data embedding process is used.The implemented technique uses the features of the 3-D object vertex’discrete cosine transform to embed a grayscale image with high capacity.The coefficient of vertex and the encrypted picture pixels are used in the watermarking procedure.Additionally,the extraction approach is fully blind and is dependent on the backward steps of the encoding procedure to get the hidden data.Correlation distance,Euclidean distance,Manhattan distance,and the Cosine distance are used to evaluate and test the performance of the proposed approach.The visibility and imperceptibility of the proposed method are assessed to show the efficiency of our work compared to previous corresponding methods.

Several Strategies on 3D Modeling of Manmade Objects

Several different strategies of 3D modeling are a do pted for different kinds of manmade objects. Firstly, for those manmade objects with regular structure, if 2D information is available and elevation information can be obtained conveniently, then 3D modeling of them can be executed directly . Secondly, for those manmade objects with complicated structure comparatively a nd related stereo images pair can be acquired, in the light of topology-based 3 D model we finish 3D modeling of them by integrating automatic and semi-automat ic object extraction. Thirdly, for the most complicated objects whose geometrica l information cannot be got from stereo images pair completely, we turn to topol ogical 3D model based on CAD.

Application of 3D Scanned Big Data of Large-scale Cultural Heritage Objects Based on Noise-robust Transparent Visualization

Three-dimensional(3D) scanning technology has undergone remarkable developments in recent years.Data acquired by 3D scanning have the form of 3D point clouds.The 3D scanned point clouds have data sizes that can be considered big data.They also contain measurement noise inherent in measurement data.These properties of 3D scanned point clouds make many traditional CG/visualization techniques difficult.This paper reviewed our recent achievements in developing varieties of high-quality visualizations suitable for the visual analysis of 3D scanned point clouds.We demonstrated the effectiveness of the method by applying the visualizations to various cultural heritage objects.The main visualization targets used in this paper are the floats in the Gion Festival in Kyoto(the float parade is on the UNESCO Intangible Cultural Heritage List) and Borobudur Temple in Indonesia(a UNESCO World Heritage Site).

MFF-Net: Multimodal Feature Fusion Network for 3D Object Detection

In complex traffic environment scenarios,it is very important for autonomous vehicles to accurately perceive the dynamic information of other vehicles around the vehicle in advance.The accuracy of 3D object detection will be affected by problems such as illumination changes,object occlusion,and object detection distance.To this purpose,we face these challenges by proposing a multimodal feature fusion network for 3D object detection(MFF-Net).In this research,this paper first uses the spatial transformation projection algorithm to map the image features into the feature space,so that the image features are in the same spatial dimension when fused with the point cloud features.Then,feature channel weighting is performed using an adaptive expression augmentation fusion network to enhance important network features,suppress useless features,and increase the directionality of the network to features.Finally,this paper increases the probability of false detection and missed detection in the non-maximum suppression algo-rithm by increasing the one-dimensional threshold.So far,this paper has constructed a complete 3D target detection network based on multimodal feature fusion.The experimental results show that the proposed achieves an average accuracy of 82.60%on the Karlsruhe Institute of Technology and Toyota Technological Institute(KITTI)dataset,outperforming previous state-of-the-art multimodal fusion networks.In Easy,Moderate,and hard evaluation indicators,the accuracy rate of this paper reaches 90.96%,81.46%,and 75.39%.This shows that the MFF-Net network has good performance in 3D object detection.

Monocular 3D object detection with Pseudo-LiDAR confidence sampling and hierarchical geometric feature extraction in 6G network

The high bandwidth and low latency of 6G network technology enable the successful application of monocular 3D object detection on vehicle platforms.Monocular 3D-object-detection-based Pseudo-LiDAR is a low-cost,lowpower solution compared to LiDAR solutions in the field of autonomous driving.However,this technique has some problems,i.e.,(1)the poor quality of generated Pseudo-LiDAR point clouds resulting from the nonlinear error distribution of monocular depth estimation and(2)the weak representation capability of point cloud features due to the neglected global geometric structure features of point clouds existing in LiDAR-based 3D detection networks.Therefore,we proposed a Pseudo-LiDAR confidence sampling strategy and a hierarchical geometric feature extraction module for monocular 3D object detection.We first designed a point cloud confidence sampling strategy based on a 3D Gaussian distribution to assign small confidence to the points with great error in depth estimation and filter them out according to the confidence.Then,we present a hierarchical geometric feature extraction module by aggregating the local neighborhood features and a dual transformer to capture the global geometric features in the point cloud.Finally,our detection framework is based on Point-Voxel-RCNN(PV-RCNN)with high-quality Pseudo-LiDAR and enriched geometric features as input.From the experimental results,our method achieves satisfactory results in monocular 3D object detection.

3D Object Detection with Attention:Shell-Based Modeling

LIDAR point cloud-based 3D object detection aims to sense the surrounding environment by anchoring objects with the Bounding Box(BBox).However,under the three-dimensional space of autonomous driving scenes,the previous object detection methods,due to the pre-processing of the original LIDAR point cloud into voxels or pillars,lose the coordinate information of the original point cloud,slow detection speed,and gain inaccurate bounding box positioning.To address the issues above,this study proposes a new two-stage network structure to extract point cloud features directly by PointNet++,which effectively preserves the original point cloud coordinate information.To improve the detection accuracy,a shell-based modeling method is proposed.It roughly determines which spherical shell the coordinates belong to.Then,the results are refined to ground truth,thereby narrowing the localization range and improving the detection accuracy.To improve the recall of 3D object detection with bounding boxes,this paper designs a self-attention module for 3D object detection with a skip connection structure.Some of these features are highlighted by weighting them on the feature dimensions.After training,it makes the feature weights that are favorable for object detection get larger.Thus,the extracted features are more adapted to the object detection task.Extensive comparison experiments and ablation experiments conducted on the KITTI dataset verify the effectiveness of our proposed method in improving recall and precision.