3D scene modeling has long been a fundamental problem in computer graphics and computer vision. With the popularity of consumer-level RGB-D cameras,there is a growing interest in digitizing real-world indoor 3D scenes...3D scene modeling has long been a fundamental problem in computer graphics and computer vision. With the popularity of consumer-level RGB-D cameras,there is a growing interest in digitizing real-world indoor 3D scenes. However,modeling indoor3 D scenes remains a challenging problem because of the complex structure of interior objects and poor quality of RGB-D data acquired by consumer-level sensors.Various methods have been proposed to tackle these challenges. In this survey,we provide an overview of recent advances in indoor scene modeling techniques,as well as public datasets and code libraries which can facilitate experiments and evaluation.展开更多
Well-designed indoor scenes incorporate interior design knowledge,which has been an essential prior for most indoor scene modeling methods.However,the layout qualities of indoor scene datasets are often uneven,and mos...Well-designed indoor scenes incorporate interior design knowledge,which has been an essential prior for most indoor scene modeling methods.However,the layout qualities of indoor scene datasets are often uneven,and most existing data-driven methods do not differentiate indoor scene examples in terms of quality.In this work,we aim to explore an approach that leverages datasets with differentiated indoor scene examples for indoor scene modeling.Our solution conducts subjective evaluations on lightweight datasets having various room configurations and furniture layouts,via pairwise comparisons based on fuzzy set theory.We also develop a system to use such examples to guide indoor scene modeling using user-specified objects.Specifically,we focus on object groups associated with certain human activities,and define room features to encode the relations between the position and direction of an object group and the room configuration.To perform indoor scene modeling,given an empty room,our system first assesses it in terms of the user-specified object groups,and then places associated objects in the room guided by the assessment results.A series of experimental results and comparisons to state-of-the-art indoor scene synthesis methods are presented to validate the usefulness and effectiveness of our approach.展开更多
Simultaneous localisation and mapping(SLAM)are the basis for many robotic applications.As the front end of SLAM,visual odometry is mainly used to estimate camera pose.In dynamic scenes,classical methods are deteriorat...Simultaneous localisation and mapping(SLAM)are the basis for many robotic applications.As the front end of SLAM,visual odometry is mainly used to estimate camera pose.In dynamic scenes,classical methods are deteriorated by dynamic objects and cannot achieve satisfactory results.In order to improve the robustness of visual odometry in dynamic scenes,this paper proposed a dynamic region detection method based on RGBD images.Firstly,all feature points on the RGB image are classified as dynamic and static using a triangle constraint and the epipolar geometric constraint successively.Meanwhile,the depth image is clustered using the K-Means method.The classified feature points are mapped to the clustered depth image,and a dynamic or static label is assigned to each cluster according to the number of dynamic feature points.Subsequently,a dynamic region mask for the RGB image is generated based on the dynamic clusters in the depth image,and the feature points covered by the mask are all removed.The remaining static feature points are applied to estimate the camera pose.Finally,some experimental results are provided to demonstrate the feasibility and performance.展开更多
Illumination harmonization is an important but challenging task that aims to achieve illumination compatibility between the foreground and background under different illumination conditions.Most current studies mainly...Illumination harmonization is an important but challenging task that aims to achieve illumination compatibility between the foreground and background under different illumination conditions.Most current studies mainly focus on achieving seamless integration between the appearance(illumination or visual style)of the foreground object itself and the background scene or producing the foreground shadow.They rarely considered global illumination consistency(i.e.,the illumination and shadow of the foreground object).In our work,we introduce“Illuminator”,an image-based illumination editing technique.This method aims to achieve more realistic global illumination harmonization,ensuring consistent illumination and plausible shadows in complex indoor environments.The Illuminator contains a shadow residual generation branch and an object illumination transfer branch.The shadow residual generation branch introduces a novel attention-aware graph convolutional mechanism to achieve reasonable foreground shadow generation.The object illumination transfer branch primarily transfers background illumination to the foreground region.In addition,we construct a real-world indoor illumination harmonization dataset called RIH,which consists of various foreground objects and background scenes captured under diverse illumination conditions for training and evaluating our Illuminator.Our comprehensive experiments,conducted on the RIH dataset and a collection of real-world everyday life photos,validate the effectiveness of our method.展开更多
基金supported by the National Natural Science Foundation of China(Project No.61120106007)Research Grant of Beijing Higher Institution Engineering Research CenterTsinghua University Initiative Scientific Research Program
文摘3D scene modeling has long been a fundamental problem in computer graphics and computer vision. With the popularity of consumer-level RGB-D cameras,there is a growing interest in digitizing real-world indoor 3D scenes. However,modeling indoor3 D scenes remains a challenging problem because of the complex structure of interior objects and poor quality of RGB-D data acquired by consumer-level sensors.Various methods have been proposed to tackle these challenges. In this survey,we provide an overview of recent advances in indoor scene modeling techniques,as well as public datasets and code libraries which can facilitate experiments and evaluation.
基金This work was partially supported by grants from the National Natural Science Foundation of China(61902032)Research Grants Council of the Hong Kong Special Administrative Region,China(CityU 11237116)City University of Hong Kong(7004915).
文摘Well-designed indoor scenes incorporate interior design knowledge,which has been an essential prior for most indoor scene modeling methods.However,the layout qualities of indoor scene datasets are often uneven,and most existing data-driven methods do not differentiate indoor scene examples in terms of quality.In this work,we aim to explore an approach that leverages datasets with differentiated indoor scene examples for indoor scene modeling.Our solution conducts subjective evaluations on lightweight datasets having various room configurations and furniture layouts,via pairwise comparisons based on fuzzy set theory.We also develop a system to use such examples to guide indoor scene modeling using user-specified objects.Specifically,we focus on object groups associated with certain human activities,and define room features to encode the relations between the position and direction of an object group and the room configuration.To perform indoor scene modeling,given an empty room,our system first assesses it in terms of the user-specified object groups,and then places associated objects in the room guided by the assessment results.A series of experimental results and comparisons to state-of-the-art indoor scene synthesis methods are presented to validate the usefulness and effectiveness of our approach.
基金supported in part by the National Natural Science Foundation of China(Grant No.U1913201,U22B2041)Natural Science Foundation of Liaoning Province(Grant No.2019-ZD-0169).
文摘Simultaneous localisation and mapping(SLAM)are the basis for many robotic applications.As the front end of SLAM,visual odometry is mainly used to estimate camera pose.In dynamic scenes,classical methods are deteriorated by dynamic objects and cannot achieve satisfactory results.In order to improve the robustness of visual odometry in dynamic scenes,this paper proposed a dynamic region detection method based on RGBD images.Firstly,all feature points on the RGB image are classified as dynamic and static using a triangle constraint and the epipolar geometric constraint successively.Meanwhile,the depth image is clustered using the K-Means method.The classified feature points are mapped to the clustered depth image,and a dynamic or static label is assigned to each cluster according to the number of dynamic feature points.Subsequently,a dynamic region mask for the RGB image is generated based on the dynamic clusters in the depth image,and the feature points covered by the mask are all removed.The remaining static feature points are applied to estimate the camera pose.Finally,some experimental results are provided to demonstrate the feasibility and performance.
基金supported by the National Natural Science Foundation of China(61972298 and 62372336),CAAI–Huawei MindSpore Open Fund,and Wuhan University–Huawei GeoInformatices Innovation Lab.
文摘Illumination harmonization is an important but challenging task that aims to achieve illumination compatibility between the foreground and background under different illumination conditions.Most current studies mainly focus on achieving seamless integration between the appearance(illumination or visual style)of the foreground object itself and the background scene or producing the foreground shadow.They rarely considered global illumination consistency(i.e.,the illumination and shadow of the foreground object).In our work,we introduce“Illuminator”,an image-based illumination editing technique.This method aims to achieve more realistic global illumination harmonization,ensuring consistent illumination and plausible shadows in complex indoor environments.The Illuminator contains a shadow residual generation branch and an object illumination transfer branch.The shadow residual generation branch introduces a novel attention-aware graph convolutional mechanism to achieve reasonable foreground shadow generation.The object illumination transfer branch primarily transfers background illumination to the foreground region.In addition,we construct a real-world indoor illumination harmonization dataset called RIH,which consists of various foreground objects and background scenes captured under diverse illumination conditions for training and evaluating our Illuminator.Our comprehensive experiments,conducted on the RIH dataset and a collection of real-world everyday life photos,validate the effectiveness of our method.
