CRD-CGAN:category-consistent and relativistic constraints for diverse text-to-image generation

Generating photo-realistic images from a text description is a challenging problem in computer vision.Previous works have shown promising performance to generate synthetic images conditional on text by Generative Adversarial Networks(GANs).In this paper,we focus on the category-consistent and relativistic diverse constraints to optimize the diversity of synthetic images.Based on those constraints,a category-consistent and relativistic diverse conditional GAN(CRD-CGAN)is proposed to synthesize K photo-realistic images simultaneously.We use the attention loss and diversity loss to improve the sensitivity of the GAN to word attention and noises.Then,we employ the relativistic conditional loss to estimate the probability of relatively real or fake for synthetic images,which can improve the performance of basic conditional loss.Finally,we introduce a category-consistent loss to alleviate the over-category issues between K synthetic images.We evaluate our approach using the Caltech-UCSD Birds-200-2011,Oxford 102 flower and MS COCO 2014 datasets,and the extensive experiments demonstrate superiority of the proposed method in comparison with state-of-the-art methods in terms of photorealistic and diversity of the generated synthetic images.

Sparse RGB-D images create a real thing:A flexible voxel based 3D reconstruction pipeline for single object

Reconstructing 3D models for single objects with complex backgrounds has wide applications like 3D printing,AR/VR,and so on.It is necessary to consider the tradeoff between capturing data at low cost and getting high-quality reconstruction results.In this work,we propose a voxel-based modeling pipeline with sparse RGB-D images to effectively and efficiently reconstruct a single real object without the geometrical post-processing operation on background removal.First,referring to the idea of VisualHull,useless and inconsistent voxels of a targeted object are clipped.It helps focus on the target object and rectify the voxel projection information.Second,a modified TSDF calculation and voxel filling operations are proposed to alleviate the problem of depth missing in the depth images.They can improve TSDF value completeness for voxels on the surface of the object.After the mesh is generated by the MarchingCube,texture mapping is optimized with view selection,color optimization,and camera parameters fine-tuning.Experiments on Kinect capturing dataset,TUM public dataset,and virtual environment dataset validate the effectiveness and flexibility of our proposed pipeline.

Multi-omics Approach Reveals Influenza-A Virus Target Genes Associated Genomic,Clinical and Immunological Characteristics in Cancers

Objective To examine the precise function of influenza A virus target genes(IATGs)in malignancy.Methods Using multi-omics data from the TCGA and TCPA datasets,33 tumor types were evaluated for IATGs.IATG expression in cancer cells was analyzed using transcriptome analysis.Copy number variation(CNV)was assessed using GISTICS 2.0.Spearman’s analysis was used to correlate mRNA expression with methylation levels.GSEA was used for the enrichment analysis.Pearson’s correlation analysis was used to examine the association between IATG mRNA expression and IC50.The ImmuCellAI algorithm was used to calculate the infiltration scores of 24 immune cell types.Results In 13 solid tumors,IATG mRNA levels were atypically expressed.Except for UCS,UVM,KICH,PCPG,THCA,CHOL,LAMI,and MESO,most cancers contained somatic IATG mutations.The main types of CNVs in IATGs are heterozygous amplifications and deletions.In most tumors,IATG mRNA expression is adversely associated with methylation.RT-PCR demonstrated that EGFR,ANXA5,CACNA1C,CD209,UVRAG were upregulated and CLEC4M was downregulated in KIRC cell lines,consistent with the TCGA and GTEx data.Conclusion Genomic changes and clinical characteristics of IATGs were identified,which may offer fresh perspectives linking the influenza A virus to cancer.

High glucose levels impact visual response properties of retinal ganglion cells in C57 mice—An in vitro physiological study

This study investigated visual response properties of retinal ganglion cells(RGCs) under high glucose levels. Extracellular single-unit responses of RGCs from mouse retinas were recorded. And the eyecup was prepared as a flat mount in a recording chamber and superfused with Ames medium. The averaged RF size of the ON RGCs(34.1±2.9, n=14) was significantly smaller than the OFF RGCs under the HG(49.3±0.3, n=12)(P<0.0001) conditions. The same reduction pattern was also observed in the osmotic control group(HM) between ON and OFF RGCs(P<0.0001). The averaged luminance threshold(LT) of ON RGCs increased significantly under HG or HM(HG: P<0.0001; HM: P<0.0002). OFF RGCs exhibited a similar response pattern under the same conditions(HG: P<0.01; HM: P<0.0002). The averaged contrast gain of ON cells was significantly lower than that of OFF cells with the HM treatment(P<0.015, unpaired Student's t test). The averaged contrast gain of ON cells was significantly higher than OFF cells with the HG treatment(P<0.0001). The present results suggest that HG reduced receptive field center size, suppressed luminance threshold, and attenuated contrast gain of RGCs. The impact of HG on ON and OFF RGCs may be mediated via different mechanisms.

Self-supervised coarse-to-fine monocular depth estimation using a lightweight attention module

Self-supervised monocular depth estimation has been widely investigated and applied in previous works.However,existing methods suffer from texture-copy,depth drift,and incomplete structure.It is difficult for normal CNN networks to completely understand the relationship between the object and its surrounding environment.Moreover,it is hard to design the depth smoothness loss to balance depth smoothness and sharpness.To address these issues,we propose a coarse-to-fine method with a normalized convolutional block attention module(NCBAM).In the coarse estimation stage,we incorporate the NCBAM into depth and pose networks to overcome the texture-copy and depth drift problems.Then,we use a new network to refine the coarse depth guided by the color image and produce a structure-preserving depth result in the refinement stage.Our method can produce results competitive with state-of-the-art methods.Comprehensive experiments prove the effectiveness of our two-stage method using the NCBAM.

Interactive lighting editing system for single indoor low-light scene images with corresponding depth maps

We propose a novel interactive lighting editing system for lighting a single indoor RGB image based on spherical harmonic lighting.It allows users to intuitively edit illumination and relight the complicated low-light indoor scene.Our method not only achieves plausible global relighting but also enhances the local details of the complicated scene according to the spatially-varying spherical harmonic lighting,which only requires a single RGB image along with a corresponding depth map.To this end,we first present a joint optimization algorithm,which is based on the geometric optimization of the depth map and intrinsic image decomposition avoiding texture-copy,for refining the depth map and obtaining the shading map.Then we propose a lighting estimation method based on spherical harmonic lighting,which not only achieves the global illumination estimation of the scene,but also further enhances local details of the complicated scene.Finally,we use a simple and intuitive interactive method to edit the environment lighting map to adjust lighting and relight the scene.Through extensive experimental results,we demonstrate that our proposed approach is simple and intuitive for relighting the low-light indoor scene,and achieve state-of-the-art results.