A great challenge for 21 cm intensity mapping experiments is the strong foreground radiation which is orders of magnitude brighter than the 21 cm signal.Removal of the foreground takes advantage of the fact that its f...A great challenge for 21 cm intensity mapping experiments is the strong foreground radiation which is orders of magnitude brighter than the 21 cm signal.Removal of the foreground takes advantage of the fact that its frequency spectrum is smooth while the redshifted 21 cm signal spectrum is stochastic.However,a complication is the nonsmoothness of the instrument response.This paper describes the electromagnetic simulation of the Tianlai cylinder array,a pathfinder for 21 cm intensity mapping experiments.Due to the vast scales involved,a direct simulation requires a large amount of computing resources.We have made the simulation practical by using a combination of methods:first simulate a single feed,then an array of feed units,finally with the feed array and a cylindrical reflector together,obtain the response for a single cylinder.We studied its radiation pattern,bandpass response and the effects of mutual coupling between feed units,and compared the results with observation.Many features seen in the measurement result are reproduced well in the simulation,especially the oscillatory features which are associated with the standing waves on the reflector.The mutual coupling between feed units is quantified with Sparameters,which decrease as the distance between the two feeds increases.Based on the simulated S-parameters,we estimate the correlated noise which has been seen in the visibility data,and the results show very good agreement with the data in both magnitude and frequency structures.These results provide useful insights on the problem of 21 cm signal extraction for real instruments.展开更多
Optical flow estimation in human facial video,which provides 2D correspondences between adjacent frames,is a fundamental pre-processing step for many applications,like facial expression capture and recognition.However...Optical flow estimation in human facial video,which provides 2D correspondences between adjacent frames,is a fundamental pre-processing step for many applications,like facial expression capture and recognition.However,it is quite challenging as human facial images contain large areas of similar textures,rich expressions,and large rotations.These characteristics also result in the scarcity of large,annotated realworld datasets.We propose a robust and accurate method to learn facial optical flow in a self-supervised manner.Specifically,we utilize various shape priors,including face depth,landmarks,and parsing,to guide the self-supervised learning task via a differentiable nonrigid registration framework.Extensive experiments demonstrate that our method achieves remarkable improvements for facial optical flow estimation in the presence of significant expressions and large rotations.展开更多
The traditional pipeline for non-rigid registration is to iteratively update the correspondence and alignment such that the transformed source surface aligns well with the target surface.Among the pipeline,the corresp...The traditional pipeline for non-rigid registration is to iteratively update the correspondence and alignment such that the transformed source surface aligns well with the target surface.Among the pipeline,the correspondence construction and iterative manner are key to the results,while existing strategies might result in local optima.In this paper,we adopt the widely used deformation graph-based representation,while replacing some key modules with neural learning-based strategies.Specifically,we design a neural network to predict the correspondence and its reliability confidence rather than the strategies like nearest neighbor search and pair rejection.Besides,we adopt the GRU-based recurrent network for iterative refinement,which is more robust than the traditional strategy.The model is trained in a self-supervised manner and thus can be used for arbitrary datasets without ground-truth.Extensive experiments demonstrate that our proposed method outperforms the state-of-the-art methods by a large margin.展开更多
This paper presents a joint head pose and facial landmark regression method with input from depth images for realtime application. Our main contributions are: firstly, a joint optimization method to estimate head pose...This paper presents a joint head pose and facial landmark regression method with input from depth images for realtime application. Our main contributions are: firstly, a joint optimization method to estimate head pose and facial landmarks, i.e., the pose regression result provides supervised initialization for cascaded facial landmark regression, while the regression result for the facial landmarks can also help to further refine the head pose at each stage. Secondly,we classify the head pose space into 9 sub-spaces, and then use a cascaded random forest with a global shape constraint for training facial landmarks in each specific space. This classification-guided method can effectively handle the problem of large pose changes and occlusion.Lastly, we have built a 3D face database containing 73 subjects, each with 14 expressions in various head poses. Experiments on challenging databases show our method achieves state-of-the-art performance on both head pose estimation and facial landmark regression.展开更多
In the past ten years,deep learning technology has achieved a great success in many fields,like computer vision and speech recognition.Recently,large-scale geometry data become more and more available,and the learned ...In the past ten years,deep learning technology has achieved a great success in many fields,like computer vision and speech recognition.Recently,large-scale geometry data become more and more available,and the learned geometry priors have been successfully applied to 3D computer vision and computer graphics fields.Different from the regular representation of images,surface meshes have irregular structures with different vertex numbers and topologies.Therefore,the traditional convolution neural networks used for images cannot be directly used to handle surface meshes,and thus,many methods have been proposed to solve this problem.In this paper,we provide a comprehensive survey of existing geometric deep learning methods formesh processing.We first introduce the relevant knowledge and theoretical background of geometric deep learning and some basic mesh data knowledge,including some commonly used mesh datasets.Then,we review various deep learning models for mesh data with two different types:graph-based methods and mesh structure-based methods.We also review the deep learning-based applications for mesh data.In the final,we give some potential research directions in this field.展开更多
The Tianlai Cylinder Pathfinder is a radio interferometer array designed to test techniques for 21 cm intensity mapping in the post-reionization Universe,with the ultimate aim of mapping the large scale structure and ...The Tianlai Cylinder Pathfinder is a radio interferometer array designed to test techniques for 21 cm intensity mapping in the post-reionization Universe,with the ultimate aim of mapping the large scale structure and measuring cosmological parameters such as the dark energy equation of state.Each of its three parallel cylinder reflectors is oriented in the north-south direction,and the array has a large field of view.As the Earth rotates,the northern sky is observed by drift scanning.The array is located in Hongliuxia,a radio-quiet site in Xinjiang,and saw its first light in September 2016.In this first data analysis paper for the Tianlai cylinder array,we discuss the sub-system qualification tests,and present basic system performance obtained from preliminary analysis of the commissioning observations during 2016-2018.We show typical interferometric visibility data,from which we derive the actual beam profile in the east-west direction and the frequency band-pass response.We describe also the calibration process to determine the complex gains for the array elements,either using bright astronomical point sources,or an artificial on site calibrator source,and discuss the instrument response stability,crucial for transit interferometry.Based on this analysis,we find a system temperature of about 90 K,and we also estimate the sensitivity of the array.展开更多
Face views are particularly important in person-to-person communication.Differenes between the camera location and the face orientation can result in undesirable facial appearances of the participants during video con...Face views are particularly important in person-to-person communication.Differenes between the camera location and the face orientation can result in undesirable facial appearances of the participants during video conferencing.This phenomenon is particularly noticeable when using devices where the frontfacing camera is placed in unconventional locations such as below the display or within the keyboard.In this paper,we take a video stream from a single RGB camera as input,and generate a video stream that emulates the view from a virtual camera at a designated location.The most challenging issue in this problem is that the corrected view often needs out-of-plane head rotations.To address this challenge,we reconstruct the 3D face shape and re-render it into synthesized frames according to the virtual camera location.To output the corrected video stream with natural appearance in real time,we propose several novel techniques including accurate eyebrow reconstruction,high-quality blending between the corrected face image and background,and template-based 3D reconstruction of glasses.Our system works well for different lighting conditions and skin tones,and can handle users wearing glasses.Extensive experiments and user studies demonstrate that our method provides high-quality results.展开更多
基金supported by the Ministry of Science and Technology(MOST)-BRICS Flagship Project 2018YFE0120800National SKA Program of China No.2020SKA0110401+6 种基金the National Key R&D Program 2017YFA0402603the National Natural Science Foundation of China(NSFC,Grant Nos.11973047,11633004 and U1631118)the Chinese Academy of Sciences(CAS)Strategic Priority Research Program XDA15020200the CAS Frontier Science Key Project QYZDJ–SSW–SLH017the CAS Inter-disciplinary Innovation Team(JCTD-2019-05)the CAS Key Instruments project ZDKYYQ20200008the Hebei Key Laboratory of Radio Astronomy Technology(HKLRAT)。
文摘A great challenge for 21 cm intensity mapping experiments is the strong foreground radiation which is orders of magnitude brighter than the 21 cm signal.Removal of the foreground takes advantage of the fact that its frequency spectrum is smooth while the redshifted 21 cm signal spectrum is stochastic.However,a complication is the nonsmoothness of the instrument response.This paper describes the electromagnetic simulation of the Tianlai cylinder array,a pathfinder for 21 cm intensity mapping experiments.Due to the vast scales involved,a direct simulation requires a large amount of computing resources.We have made the simulation practical by using a combination of methods:first simulate a single feed,then an array of feed units,finally with the feed array and a cylindrical reflector together,obtain the response for a single cylinder.We studied its radiation pattern,bandpass response and the effects of mutual coupling between feed units,and compared the results with observation.Many features seen in the measurement result are reproduced well in the simulation,especially the oscillatory features which are associated with the standing waves on the reflector.The mutual coupling between feed units is quantified with Sparameters,which decrease as the distance between the two feeds increases.Based on the simulated S-parameters,we estimate the correlated noise which has been seen in the visibility data,and the results show very good agreement with the data in both magnitude and frequency structures.These results provide useful insights on the problem of 21 cm signal extraction for real instruments.
基金This work was supported by National Natural Science Foundation of China(No.62122071)the Youth Innovation Promotion Association CAS(No.2018495)+1 种基金the Fundamental Research Funds for the Central Universities(No.WK3470000021)through the Alibaba Innovation Research Program(AIR).
文摘Optical flow estimation in human facial video,which provides 2D correspondences between adjacent frames,is a fundamental pre-processing step for many applications,like facial expression capture and recognition.However,it is quite challenging as human facial images contain large areas of similar textures,rich expressions,and large rotations.These characteristics also result in the scarcity of large,annotated realworld datasets.We propose a robust and accurate method to learn facial optical flow in a self-supervised manner.Specifically,we utilize various shape priors,including face depth,landmarks,and parsing,to guide the self-supervised learning task via a differentiable nonrigid registration framework.Extensive experiments demonstrate that our method achieves remarkable improvements for facial optical flow estimation in the presence of significant expressions and large rotations.
基金supported by National Natural Science Foundation of China(No.62122071)the Youth Innovation Promotion Association CAS(No.2018495)“the Fundamental Research Funds for the Central Universities”(No.WK3470000021).
文摘The traditional pipeline for non-rigid registration is to iteratively update the correspondence and alignment such that the transformed source surface aligns well with the target surface.Among the pipeline,the correspondence construction and iterative manner are key to the results,while existing strategies might result in local optima.In this paper,we adopt the widely used deformation graph-based representation,while replacing some key modules with neural learning-based strategies.Specifically,we design a neural network to predict the correspondence and its reliability confidence rather than the strategies like nearest neighbor search and pair rejection.Besides,we adopt the GRU-based recurrent network for iterative refinement,which is more robust than the traditional strategy.The model is trained in a self-supervised manner and thus can be used for arbitrary datasets without ground-truth.Extensive experiments demonstrate that our proposed method outperforms the state-of-the-art methods by a large margin.
基金supported by the National Key Technologies R&D Program of China (No. 2016YFC0800501)the National Natural Science Foundation of China (No. 61672481)
文摘This paper presents a joint head pose and facial landmark regression method with input from depth images for realtime application. Our main contributions are: firstly, a joint optimization method to estimate head pose and facial landmarks, i.e., the pose regression result provides supervised initialization for cascaded facial landmark regression, while the regression result for the facial landmarks can also help to further refine the head pose at each stage. Secondly,we classify the head pose space into 9 sub-spaces, and then use a cascaded random forest with a global shape constraint for training facial landmarks in each specific space. This classification-guided method can effectively handle the problem of large pose changes and occlusion.Lastly, we have built a 3D face database containing 73 subjects, each with 14 expressions in various head poses. Experiments on challenging databases show our method achieves state-of-the-art performance on both head pose estimation and facial landmark regression.
文摘In the past ten years,deep learning technology has achieved a great success in many fields,like computer vision and speech recognition.Recently,large-scale geometry data become more and more available,and the learned geometry priors have been successfully applied to 3D computer vision and computer graphics fields.Different from the regular representation of images,surface meshes have irregular structures with different vertex numbers and topologies.Therefore,the traditional convolution neural networks used for images cannot be directly used to handle surface meshes,and thus,many methods have been proposed to solve this problem.In this paper,we provide a comprehensive survey of existing geometric deep learning methods formesh processing.We first introduce the relevant knowledge and theoretical background of geometric deep learning and some basic mesh data knowledge,including some commonly used mesh datasets.Then,we review various deep learning models for mesh data with two different types:graph-based methods and mesh structure-based methods.We also review the deep learning-based applications for mesh data.In the final,we give some potential research directions in this field.
基金support of the Astronomical Technology Center of National Astronomical Observatories of China(NAOC)support of Ministry of Science and Technology(MOST)(Grant No.2012AA121701)+10 种基金supported by MOST(Grant Nos.2016YFE0100300,and 2018YFE0120800)the National Natural Science Foundation of China(NSFC)(Grant Nos.11633004,11473044,and 11653003)the Chinese Academy of Sciences(CAS)(Grant No.QYZDJ-SSW-SLH017)support of the NSFC-CAS Joint Fund of Astronomy(Grant No.U1631118)partially supported by the National Key R&D Program(Grant No.2017YFA0402603)the CAS Interdisciplinary Innovation Team(Grant No.JCTD-2019-05)support of NSFC(Grant No.U1501501)the Tianhe-1 supercomputerpartially supported by the US National Science Foundation(NSF)Award(Grant No.AST-1616554)partial support from Centre National de la Recherche Scientifique(CNRS)via IN2P3&INSU,Observatoire de ParisIrfu/CEA。
文摘The Tianlai Cylinder Pathfinder is a radio interferometer array designed to test techniques for 21 cm intensity mapping in the post-reionization Universe,with the ultimate aim of mapping the large scale structure and measuring cosmological parameters such as the dark energy equation of state.Each of its three parallel cylinder reflectors is oriented in the north-south direction,and the array has a large field of view.As the Earth rotates,the northern sky is observed by drift scanning.The array is located in Hongliuxia,a radio-quiet site in Xinjiang,and saw its first light in September 2016.In this first data analysis paper for the Tianlai cylinder array,we discuss the sub-system qualification tests,and present basic system performance obtained from preliminary analysis of the commissioning observations during 2016-2018.We show typical interferometric visibility data,from which we derive the actual beam profile in the east-west direction and the frequency band-pass response.We describe also the calibration process to determine the complex gains for the array elements,either using bright astronomical point sources,or an artificial on site calibrator source,and discuss the instrument response stability,crucial for transit interferometry.Based on this analysis,we find a system temperature of about 90 K,and we also estimate the sensitivity of the array.
文摘Face views are particularly important in person-to-person communication.Differenes between the camera location and the face orientation can result in undesirable facial appearances of the participants during video conferencing.This phenomenon is particularly noticeable when using devices where the frontfacing camera is placed in unconventional locations such as below the display or within the keyboard.In this paper,we take a video stream from a single RGB camera as input,and generate a video stream that emulates the view from a virtual camera at a designated location.The most challenging issue in this problem is that the corrected view often needs out-of-plane head rotations.To address this challenge,we reconstruct the 3D face shape and re-render it into synthesized frames according to the virtual camera location.To output the corrected video stream with natural appearance in real time,we propose several novel techniques including accurate eyebrow reconstruction,high-quality blending between the corrected face image and background,and template-based 3D reconstruction of glasses.Our system works well for different lighting conditions and skin tones,and can handle users wearing glasses.Extensive experiments and user studies demonstrate that our method provides high-quality results.