Deep appearance modeling:A survey

Appearance modeling is an essential task in computer graphics for capturing and reproducing rich appearance of real world materials under different lighting and viewing conditions.With recent advances of deep learning techniques,a set of deep learning based approaches have been proposed for improving the efficiency and result quality of appearance modeling.In this paper,we provide a survey of these deep appearance modeling techniques from both graphics and machine learning perspectives,and discuss the challenges and opportunities along this direction.

EFFECTIVE APPEARANCE MODEL FOR PROBABILISTIC OBJECT TRACKING

This paper presents a robust object tracking approach via a spatially constrained colour model. Local image patches of the object and spatial relation between these patches are informative and stable during object tracking. So, we propose to partition an object into patches and develop a Spatially Constrained Colour Model (SCCM) by combining the colour distributions and spatial configuration of these patches. The likelihood of the candidate object is given by estimating the confidences of the pixels in the candidate object region. The appearance model is learnt from the first frame and the tracking is carried out by particle filter. The experimental results show that the proposed tracking approach can accurately track the object with scale changes, pose variance and partial occlusion.

Scene-adaptive hierarchical data association and depth-invariant part-based appearance model for indoor multiple objects tracking

Indoor multi-tracking is more challenging compared with outdoor tasks due to frequent occlusion, view-truncation, severe scale change and pose variation, which may bring considerable unreliability and ambiguity to target representation and data association. So discriminative and reliable target representation is vital for accurate data association in multi-tracking. Pervious works always combine bunch of features to increase the discriminative power, but this is prone to error accumulation and unnecessary computational cost, which may increase ambiguity on the contrary. Moreover, reliability of a same feature in different scenes may vary a lot, especially for currently widespread network cameras, which are settled in various and complex indoor scenes, previous fixed feature selection schemes cannot meet general requirements. To properly handle these problems, first, we propose a scene-adaptive hierarchical data association scheme, which adaptively selects features with higher reliability on target representation in the applied scene, and gradually combines features to the minimum requirement of discriminating ambiguous targets; second, a novel depth-invariant part-based appearance model using RGB-D data is proposed which makes the appearance model robust to scale change, partial occlusion and view-truncation. The introduce of RGB-D data increases the diversity of features, which provides more types of features for feature selection in data association and enhances the final multi-tracking performance. We validate our method from several aspects including scene-adaptive feature selection scheme, hierarchical data association scheme and RGB-D based appearance modeling scheme in various indoor scenes, which demonstrates its effectiveness and efficiency on improving multi-tracking performances in various indoor scenes.

Artificial Neural Network Performing the Forward Operation of Color Appearance Model

A method of the forward operation of color appearance (from colorimetric attributes to color appearance attributes) using an artificial neural network (ANN) is presented The neural network model developed is a multilayer feedforward neural network model for predicting color appearance model (CAM). This method greatly decreased the mathematical computation in color appearance prediction. The error backed-propagation (BP) algorithm was applied in the training of the neural networks, and it was trained and tested by the LUTCHI color appearance datasets which are the most comprehensive one in testing color appearance model. CRT was selected as a typical example in experiment because it is usually used as self-luminous object in fact, and several ways for choosing training samples were included and compared each other. The testing results show that the color appearance prediction using artificial neural network is well consistent with visual evaluation.

Simulation of CIECAM02 color appearance model based on Chinese color system

Color appearance model （CAM） can be used to determine the required colors for repro- duction across changes in cross-media. CIECAM02 color appearance model prediction is implemen- ted by artificial neural networks in this paper, which includes forward and reversed prediction. 1333 color samples as training samples and other 1332 color samples as test samples are selected in the Chinese color system. In order to test the prediction accuracy of neural networks after simulation of CIECAM02 color appearance model, the color-difference formula can be used for the evaluation of forward and reversed models. Results have shown that BP neural-network has acceptable accuracy in simulation of CIECAM02 color appearance model for colors of Chinese color system.

Efficient model building in active appearance model for rotated face

This paper proposes the efficient model building in active appearance model(AAM) for the rotated face.Finding an exact region of the face is generally difficult due to different shapes and viewpoints.Unlike many papers about the fitting method of AAM,this paper treats how images are chosen for fitting of the rotated face in modelling process.To solve this problem,databases of facial rotation and expression are selected and models are built using Procrustes method and principal component analysis(PCA).These models are applied in fitting methods like basic AAM fitting,inverse compositional alignment(ICA),project-out ICA,normalization ICA,robust normalization inverse compositional algorithm(RNIC)and efficient robust normalization algorithm(ERN).RNIC and ERN can fit the rotated face in images efficiently.The efficiency of model building is checked using sequence images made by ourselves.

Facial model fitting algorithm based on active appearance model

Active appearance model（AAM） is an efficient useful for the subsequent work such as face detection and method for the localization of facial feature points, which is also facial expression recognition. In this paper, we mainly discuss the AAMs based on principal component analysis （PCA）. We also propose an efficient facial fitting algorithm, which is named inverse compositional image alignment （ICIA）, to eliminate a considerable amount of computation resulting from traditional gradient descent fitting algorithm. Finally, 3D facial curvature is used to initialize the location of facial feature, which helps select the parameters of initial state for the improved AAM.

Research on degradation mechanism of trichlorobenzene and Hg^(0) by nonthermal plasma catalysis

Aiming at mercury and dioxin in fire coal gas as research objects,nonthermal plasma(NTP)catalytic technology was used to investigate the degradation effect of operating condition parameters on mixed pollutants in mixed flue gas condition,and to explore the synergistic degradation of Hg0and TCB(1,2,3-trichlorobenzene,TCB)under mixed flue gas conditions.The research results showed that the conversion efficiency of mercury and TCB increased with the additional output of voltage,and decreased with the increase of the gas flow rate.Under optimal reaction conditions:voltage=17 k V,frequency=300 Hz,gas flow rate=21 min^(-1),the conversion efficiency of Hg^(0)and TCB reached the highest 91.4%and 84.98%,respectively.In the NTP catalytic system,active free radicals played an important role in the synergistic conversion of mercury and TCB,which have a competitive effect,to make the conversion efficiency of mixed pollutants lower than a single substance.In the mixed flue gas condition,the mixed gas has an inhibitory effect on the synergistic conversion of mercury and TCB.Kinetic modeling of NTP catalytic synergistic reaction was established.Under three conditions of TCB,mercury and TCB,mixed simulated flue gas,the NTP catalytic technology showed a quasi-firstorder kinetic reaction for the degradation of TCB.According to the synergistic effect of NTP and composites,the transformation and degradation of TCB mainly included two processes:TCB and ring opening,and Hg^(0)was finally oxidized to Hg^(2+).

3D fluid model analysis on the generation of negative hydrogen ions for negative ion source of NBI

A radio-frequency(RF) inductively coupled negative hydrogen ion source(NHIS) has been adopted in the China Fusion Engineering Test Reactor(CFETR) to generate negative hydrogen ions.By incorporating the level-lumping method into a three-dimensional fluid model,the volume production and transportation of H^(-) in the NHIS,which consists of a cylindrical driver region and a rectangular expansion chamber,are investigated self-consistently at a large input power(40 k W) and different pressures(0.3–2.0 Pa).The results indicate that with the increase of pressure,the H^(-) density at the bottom of the expansion region first increases and then decreases.In addition,the effect of the magnetic filter is examined.It is noteworthy that a significant increase in the H^(-) density is observed when the magnetic filter is introduced.As the permanent magnets move towards the driver region,the H^(-) density decreases monotonically and the asymmetry is enhanced.This study contributes to the understanding of H-distribution under various conditions and facilitates the optimization of volume production of negative hydrogen ions in the NHIS.

Human tracking in camera network with non-overlapping FOVs

An adaptive human tracking method across spatially separated surveillance cameras with non-overlapping fields of views （FOVs） is proposed. The method relies on the two cues of the human appearance model and spatio-temporal information between cameras. For the human appearance model, an HSV color histogram is extracted from different human body parts （head, torso, and legs）, then a weighted algorithm is used to compute the similarity distance of two people. Finally, a similarity sorting algorithm with two thresholds is exploited to find the correspondence. The spatio- temporal information is established in the learning phase and is updated incrementally according to the latest correspondence. The experimental results prove that the proposed human tracking method is effective without requiring camera calibration and it becomes more accurate over time as new observations are accumulated.

LOCALIZATION OF OBJECT (SPINE) IN MEDICAL IMAGE USING ACTIVE SHAPE MODELS

Active shape models (ASM), consisting of a shape model and a local gray-level appearance model, can be used to locate the objects in images. In original ASM scheme, the model of object′s gray-level variations is based on the assumption of one-dimensional sampling and searching method. In this work a new way to model the gray-level appearance of the objects is explored, using a two-dimensional sampling and searching technique in a rectangular area around each landmark of object shape. The ASM based on this improvement is compared with the original ASM on an identical medical image set for task of spine localization. Experiments demonstrate that the method produces significantly fast, effective, accurate results for spine localization in medical images.

Tracking facial features with occlusions

Facial expression recognition consists of determining what kind of emotional content is presented in a human face. The problem presents a complex area for exploration, since it encompasses face acquisition, facial feature tracking, facial ex- pression classification. Facial feature tracking is of the most interest. Active Appearance Model (AAM) enables accurate tracking of facial features in real-time, but lacks occlusions and self-occlusions. In this paper we propose a solution to improve the accuracy of fitting technique. The idea is to include occluded images into AAM training data. We demonstrate the results by running ex- periments using gradient descent algorithm for fitting the AAM. Our experiments show that using fitting algorithm with occluded training data improves the fitting quality of the algorithm.

Coherent Point Drift Registration Combined with Image Feature and its Application

A key step of constructing active appearance model is requiring a set of appropriate training shapes with well-defined correspondences.In this paper,we introduce a novel point correspondence method(FB-CPD),which can improve the accuracy of coherent point drift(CPD) by using the information of image feature.The objective function of the proposed method is defined by both of geometric spatial information and image feature information,and the origin Gaussian mixture model in CPD is modified according to the image feature of points.FB-CPD is tested on the 3D prostate and liver point sets through the simulation experiments.The registration error can be reduced efficiently by FB-CPD.Moreover,the active appearance model constructed by FB-CPD can obtain fine segmentation in 3D CT prostate image.Compared with the original CPD,the overlap ratio of voxels was improved from 88.7% to 90.2% by FB-CPD.

DiffMat:Latent diffusion models for image-guided material generation

Creating realistic materials is essential in the construction of immersive virtual environments.While existing techniques for material capture and conditional generation rely on flash-lit photos,they often produce artifacts when the illumination mismatches the training data.In this study,we introduce DiffMat,a novel diffusion model that integrates the CLIP image encoder and a multi-layer,crossattention denoising backbone to generate latent materials from images under various illuminations.Using a pre-trained StyleGAN-based material generator,our method converts these latent materials into high-resolution SVBRDF textures,a process that enables a seamless fit into the standard physically based rendering pipeline,reducing the requirements for vast computational resources and expansive datasets.DiffMat surpasses existing generative methods in terms of material quality and variety,and shows adaptability to a broader spectrum of lighting conditions in reference images.

Effectively Lossless Subspace Appearance Model Compression Using Prior Information

Subspace appearance models are widely used in computer vision and image processing tasks to compactly represent the appearance variations of target objects. In order to ensure algorithm performance, they are typically stored in high-precision formats; this results in a large storage footprint, rendering redistribution costly and difficult. Since for most image and vision applications, pixel values are quantized to 8 bits by the acquisition apparatuses, we show that it is possible to construct a fixed-width, effectively Iossless representation of the bases vectors, in the sense that reconstructions from the original bases and from the quantized bases never deviate by more than half of the quantization step-size. In addition to directly applying this result to Iosslessly compress individual models, we also propose an algorithm to compress appearance models by utilizing prior information on the modeled objects in the form of prior appearance subspaces. Experiments conducted on the compression of person-specific face appearance models demonstrate the effectiveness of the proposed algorithms.

Multiview SVBRDF capture from unified shape and illumination

This paper proposes a stable method for reconstructing spatially varying appearances (SVBRDFs) frommultiview images captured under casual lighting conditions. Unlike flat surface capture methods, ourscan be applied to surfaces with complex silhouettes. The proposed method takes multiview images asinputs and outputs a unified SVBRDF estimation. We generated a large-scale dataset containing themultiview images, SVBRDFs, and lighting appearance of vast synthetic objects to train a two-streamhierarchical U-Net for SVBRDF estimation that is integrated into a differentiable rendering networkfor surface appearance reconstruction. In comparison with state-of-the-art approaches, our methodproduces SVBRDFs with lower biases for more casually captured images.