An Improved Encoder-Decoder CNN with Region-Based Filtering for Vibrant Colorization

Colorization is the practice of adding appropriate chromatic values to monochrome photographs or videos.A real-valued luminance image can be mapped to a three-dimensional color image.However,it is a severely ill-defined problem and not has a single solution.In this paper,an encoder-decoder Convolutional Neural Network(CNN)model is used for colorizing gray images where the encoder is a Densely Connected Convolutional Network(DenseNet)and the decoder is a conventional CNN.The DenseNet extracts image features from gray images and the conventional CNN outputs a^(*)b^(*)color channels.Due to a large number of desaturated color components compared to saturated color components in the training images,the saturated color components have a strong tendency towards desaturated color components in the predicted a^(*)b^(*)channel.To solve the problems,we rebalance the predicted a^(*)b^(*)color channel by smoothing every subregion individually using the average filter.2 stage k-means clustering technique is applied to divide the subregions.Then we apply Gamma transformation in the entire a^(*)b^(*)channel to saturate the image.We compare our proposed method with several existing methods.From the experimental results,we see that our proposed method has made some notable improvements over the existing methods and color representation of gray-scale images by our proposed method is more plausible to visualize.Additionally,our suggested approach beats other approaches in terms of Peak Signal-to-Noise Ratio(PSNR),Structural Similarity Index Measure(SSIM)and Histogram.

Colorization method based on the linear relationship assumption

A new local cost function is proposed in this paper based on the linear relationship assumption between the values of the color components and the intensity component in each local image window,then a new quadratic objective function is derived from it and the globally optimal chrominance values can be computed by solving a sparse linear system of equations.Through the colorization experiments on various test images,it is confirmed that the colorized images obtained by our proposed method have more vivid colors and sharper boundaries than those obtained by the traditional method.The peak signal to noise ratio(PSNR) of the colorized images and the average estimation error of the chrominance values relative to the original images also show that our proposed method gives more precise estimation than the traditional method.

Effect of Neodymium on the Crystallization, Microstructure and Colorization of Li₂O-Al₂O₃-SiO₂Glass Ceramics

The effect of single coloring agent Nd(NO3)3 on the crystallization, microstructure and colorization of Li2O-Al2O3-SiO2 (LAS) glass ceramics were investigated by differential thermal analysis (DTA), X-ray diffractometry (XRD) and scanning electron microscopy (SEM). The introduction of little neodymium has no effect on the crystallization manner and the formation of main crystallization phase, but more neodymium will weaken the crystallization of LAS glass. Little neodymium can increase the glossiness of LAS glass ceramic, while more neodymium can color LAS glass with light purple or red purple. The colorability of neodymium for LAS glass ceramic decreases with the increase of crystallization temperature.

Decolorization of Direct Black 22 by Aspergillus ficuum

The decolorization of Direct Black 22 by Aspergillus ficuum has been studied. It was found that Aspergillus ficuum could effectively decolorize Direct Black 22 especially when grown as pelleted mycelia. Results showed that the media containing Direct Black 22 at 50 mg/L could be decolorized by 98.05% of the initial color in 24 h. The optimum pH and temperature of decolorization are 4.0 and 33 °C respectively. Aeration was quite beneficial to decolorization. Medium composition and the concentration of Direct Black 22 could affect the rate of decolorization. The dye degraded products assayed by UV-visible spectrophotometer and macroscopic observation showed that the decolorization of Direct Black 22 by mycelial pellets includes two important processes: bioadsorption and biodegradation. The degradation experiment agree with the Michaelis-Menten kinetics equation.

Temporally consistent video colorization with deep feature propagation and self-regularization learning

Video colorization is a challenging and highly ill-posed problem.Although recent years have witnessed remarkable progress in single image colorization,there is relatively less research effort on video colorization,and existing methods always suffer from severe flickering artifacts(temporal inconsistency)or unsatisfactory colorization.We address this problem from a new perspective,by jointly considering colorization and temporal consistency in a unified framework.Specifically,we propose a novel temporally consistent video colorization(TCVC)framework.TCVC effectively propagates frame-level deep features in a bidirectional way to enhance the temporal consistency of colorization.Furthermore,TCVC introduces a self-regularization learning(SRL)scheme to minimize the differences in predictions obtained using different time steps.SRL does not require any ground-truth color videos for training and can further improve temporal consistency.Experiments demonstrate that our method can not only provide visually pleasing colorized video,but also with clearly better temporal consistency than state-of-the-art methods.A video demo is provided at https://www.youtube.com/watch?v=c7dczMs-olE,while code is available at https://github.com/lyh-18/TCVC-Tem porally-Consistent-Video-Colorization.

Video Colorization:A Survey

Video colorization aims to add color to grayscale or monochrome videos.Although existing methods have achieved substantial and noteworthy results in the field of image colorization,video colorization presents more formidable obstacles due to the additional necessity for temporal consistency.Moreover,there is rarely a systematic review of video colorization methods.In this paper,we aim to review existing state-of-the-art video colorization methods.In addition,maintaining spatial-temporal consistency is pivotal to the process of video colorization.To gain deeper insight into the evolution of existing methods in terms of spatial-temporal consistency,we further review video colorization methods from a novel perspective.Video colorization methods can be categorized into four main categories:optical-flow based methods,scribble-based methods,exemplar-based methods,and fully automatic methods.However,optical-flow based methods rely heavily on accurate optical-flow estimation,scribble-based methods require extensive user interaction and modifications,exemplar-based methods face challenges in obtaining suitable reference images,and fully automatic methods often struggle to meet specific colorization requirements.We also discuss the existing challenges and highlight several future research opportunities worth exploring.

A review of image and video colorization: From analogies to deep learning

Image colorization is a classic and important topic in computer graphics,where the aim is to add color to a monochromatic input image to produce a colorful result.In this survey,we present the history of colorization research in chronological order and summarize popular algorithms in this field.Early work on colorization mostly focused on developing techniques to improve the colorization quality.In the last few years,researchers have considered more possibilities such as combining colorization with NLP(natural language processing)and focused more on industrial applications.To better control the color,various types of color control are designed,such as providing reference images or color-scribbles.We have created a taxonomy of the colorization methods according to the input type,divided into grayscale,sketch-based and hybrid.The pros and cons are discussed for each algorithm,and they are compared according to their main characteristics.Finally,we discuss how deep learning,and in particular Generative Adversarial Networks(GANs),has changed this field.

Reference-guided structure-aware deep sketch colorization for cartoons

Digital cartoon production requires extensive manual labor to colorize sketches with visually pleasant color composition and color shading.During colorization,the artist usually takes an existing cartoon image as color guidance,particularly when colorizing related characters or an animation sequence.Reference-guided colorization is more intuitive than colorization with other hints,such as color points or scribbles,or text-based hints.Unfortunately,reference-guided colorization is challenging since the style of the colorized image should match the style of the reference image in terms of both global color composition and local color shading.In this paper,we propose a novel learning-based framework which colorizes a sketch based on a color style feature extracted from a reference color image.Our framework contains a color style extractor to extract the color feature from a color image,a colorization network to generate multi-scale output images by combining a sketch and a color feature,and a multi-scale discriminator to improve the reality of the output image.Extensive qualitative and quantitative evaluations show that our method outperforms existing methods,providing both superior visual quality and style reference consistency in the task of reference-based colorization.

Non-volatile dynamically switchable color display via chalcogenide stepwise cavity resonators

High-resolution multi-color printing relies upon pixelated optical nanostructures,which is crucial to promote color display by producing nonbleaching colors,yet requires simplicity in fabrication and dynamic switching.Antimony trisulfide(Sb_(2)S_(3))is a newly rising chalcogenide material that possesses prompt and significant transition of its optical characteristics in the visible region between amorphous and crystalline phases,which holds the key to color-varying devices.Herein,we proposed a dynamically switchable color printing method using Sb_(2)S_(3)-based stepwise pixelated Fabry-Pérot(FP)cavities with various cavity lengths.The device was fabricated by employing a direct laser patterning that is a less timeconsuming,more approachable,and low-cost technique.As switching the state of Sb_(2)S_(3) between amorphous and crystalline,the multi-color of stepwise pixelated FP cavities can be actively changed.The color variation is due to the profound change in the refractive index of Sb_(2)S_(3) over the visible spectrum during its phase transition.Moreover,we directly fabricated sub-50 nm nano-grating on ultrathin Sb_(2)S_(3) laminate via microsphere 800-nm femtosecond laser irradiation in far field.The minimum feature size can be further decreased down to~45 nm(λ/17)by varying the thickness of Sb_(2)S_(3) film.Ultrafast switchable Sb_(2)S_(3) photonic devices can take one step toward the next generation of inkless erasable papers or displays and enable information encryption,camouflaging surfaces,anticounterfeiting,etc.Importantly,our work explores the prospects of rapid and rewritable fabrication of periodic structures with nano-scale resolution and can serve as a guideline for further development of chalcogenide-based photonics components.

Coupling of reduced inorganic fertilizer with plant-based organic fertilizer as a promising fertilizer management strategy for colored rice in tropical regions

Colored rice is a type of high-quality,high-added-value rice that has attracted increasing attention in recent years.The use of large amounts of inorganic nitrogen fertilizer in rice fields results in low fertilizer use efficiency and high environmental pollution.Organic fertilizer is a promising way to improve soil quality and sustain high yields.However,most studies focus on the effect of animal-based organic fertilizers.The effects of different ratios of plantbased organic fertilizer and inorganic fertilizer on the grain yield and quality of colored rice have rarely been reported.Therefore,a two-year field experiment was conducted in 2020 and 2021 to study the effects of replacing inorganic N fertilizers with plant-based organic fertilizers on the yield,nitrogen use efficiency(NUE),and anthocyanin content of two colored rice varieties in a tropical region in China.The experimental treatments included no nitrogen fertilization(T1),100% inorganic nitrogen fertilizer(T2),30%inorganic nitrogen fertilizer substitution with plant-based organic fertilizer(T3),60%inorganic nitrogen fertilizer substitution with plant-based organic fertilizer(T4),and 100% plantbased organic fertilizer(T5).The total nitrogen provided to all the treatments except T1 was the same at 120 kg ha-1.Our results showed that the T3 treatment enhanced the grain yield and anthocyanin content of colored rice by increasing nitrogen use efficiency compared with T2.On average,grain yields were increased by 9 and 8%,while the anthocyanin content increased by 16 and 10% in the two colored rice varieties under T3 across the two years,respectively,as compared with T2.Further study of the residual effect of partial substitution of inorganic fertilizers showed that the substitution of inorganic fertilizer with plant-based organic fertilizer improved the soil physiochemical properties,and thus increased the rice grain yield,in the subsequent seasons.The highest grain yield of the subsequent rice crop was observed under the T5 treatment.Our results suggested that the application of plantbased organic fertilizers can sustain the production of colored rice with high anthocyanin content in tropical regions,which is beneficial in reconciling the relationship between rice production and environmental protection.

Intelligent diagnosis of retinal vein occlusion based on color fundus photographs

AIM:To develop an artificial intelligence(AI)diagnosis model based on deep learning(DL)algorithm to diagnose different types of retinal vein occlusion(RVO)by recognizing color fundus photographs(CFPs).METHODS:Totally 914 CFPs of healthy people and patients with RVO were collected as experimental data sets,and used to train,verify and test the diagnostic model of RVO.All the images were divided into four categories[normal,central retinal vein occlusion(CRVO),branch retinal vein occlusion(BRVO),and macular retinal vein occlusion(MRVO)]by three fundus disease experts.Swin Transformer was used to build the RVO diagnosis model,and different types of RVO diagnosis experiments were conducted.The model’s performance was compared to that of the experts.RESULTS:The accuracy of the model in the diagnosis of normal,CRVO,BRVO,and MRVO reached 1.000,0.978,0.957,and 0.978;the specificity reached 1.000,0.986,0.982,and 0.976;the sensitivity reached 1.000,0.955,0.917,and 1.000;the F1-Sore reached 1.000,0.9550.943,and 0.887 respectively.In addition,the area under curve of normal,CRVO,BRVO,and MRVO diagnosed by the diagnostic model were 1.000,0.900,0.959 and 0.970,respectively.The diagnostic results were highly consistent with those of fundus disease experts,and the diagnostic performance was superior.CONCLUSION:The diagnostic model developed in this study can well diagnose different types of RVO,effectively relieve the work pressure of clinicians,and provide help for the follow-up clinical diagnosis and treatment of RVO patients.

Structure-Aware Nonlocal Optimization Framework for Image Colorization

This paper proposes a structure-aware nonlocal energy optimization framework for interactive image colo- rization with sparse scribbles. Our colorization technique propagates colors to both local intensity-continuous regions and remote texture-similar regions without explicit image segmentation. We implement the nonlocal principle by computing k nearest neighbors in the high-dimensional feature space. The feature space contains not only image coordinates and intensities but also statistical texture features obtained with the direction-aligned Gabor wavelet filter. Structure maps are utilized to scale texture features to avoid artifacts along high-contrast boundaries. We show various experimental results and comparisons on image colorization, selective recoloring and decoloring, and progressive color editing to demonstrate the effectiveness of the proposed approach.

Affective Image Colorization

Colorization of gray-scale images has attracted many attentions for a long time. An important role of image color is the conveyer of emotions （through color themes）. The colorization with an undesired color theme is less useful, even it is semantically correct. However this has been rarely considered. Automatic colorization respecting both the semantics and the emotions is undoubtedly a challenge. In this paper~ we propose a complete system for affective image colorization. We only need the user to assist object segmentation along with text labels and an affective word. First, the text labels along with other object characters are jointly used to filter the internet images to give each object a set of semantically correct reference images. Second, we select a set of color themes according to the affective word based on art theories. With these themes, a generic algorithm is used to select the best reference for each object, balancing various requirements. Finally, we propose a hybrid texture synthesis approach for colorization. To the best of our knowledge, it is the first system which is able to efficiently colorize a gray-scale image semantically by an emotionally controllable fashion. Our experiments show the effectiveness of our system, especially the benefit compared with the previous Markov random field （MRF） based method.

Automatic Colorization with Improved Spatial Coherence and Boundary Localization

Grayscale image colorization is an important computer graphics problem with a variety of applications. Recent fully automatic colorization methods have made impressive progress by formulating image colorization as a pixel-wise prediction task and utilizing deep convolutional neural networks. Though tremendous improvements have been made, the result of automatic colorization is still far from perfect. Specifically, there still exist common pitfalls in maintaining color consistency in homogeneous regions as well as precisely distinguishing colors near region boundaries. To tackle these problems, we propose a novel fully automatic colorization pipeline which involves a boundary-guided CRF （conditional random field） and a CNN-based color transform as post-processing steps. In addition, as there usually exist multiple plausible colorization proposals for a single image, automatic evaluation for different colorization methods remains a challenging task. We further introduce two novel automatic evaluation schemes to efficiently assess colorization quality in terms of spatial coherence and localization. Comprehensive experiments demonstrate great quality improvement in results of our proposed colorization method under multiple evaluation metrics.

Physiological and psychological responses to tended plant communities with varying color characteristics

Exposure to plants has been reported to promote health and reduce stress,and plant color has direct impacts on physical and mental health.We used images of common types of tended plant communities in Shenyang,China,with combinations of yellow,green,and red foliage,as experimental stimuli.A total of 27 images were used as visual stimuli.We used electroencephalography to measureαwave activity(8-13 Hz)in 40 subjects while they viewed visual stimuli.These data were combined with subjective questionnaire data to analyze the relaxing effect of images of tended plant communities with different color types and proportions on people.The results revealed that,although there were slight differences between the electroencephalography and psychological findings,women were significantly more relaxed than men after viewing the images.Physiological and psychological responses varied with the types and proportions of colors in the tended plant communities:those of foliage with combinations of two or three colors induced stronger responses than images with a single color.Specifically,(1)for one-color plant communities,green or yellow plant communities induced a stronger relaxation effect than red plant communities;(2)for two-color plant communities,the optimal color proportion was 55%+45%,and the green+yellow and green+red color combinations induced a stronger relaxation effect;(3)for three-color plant communities,the relaxation effect was strongest when the color proportion was 55%green+25%yellow+20%red.These data would provide a plant color matching in future plant landscape design,which may be helpful for creating healthy and relaxing environments.

Unsupervised Color Segmentation with Reconstructed Spatial Weighted Gaussian Mixture Model and Random Color Histogram

Image classification and unsupervised image segmentation can be achieved using the Gaussian mixture model.Although the Gaussian mixture model enhances the flexibility of image segmentation,it does not reflect spatial information and is sensitive to the segmentation parameter.In this study,we first present an efficient algorithm that incorporates spatial information into the Gaussian mixture model(GMM)without parameter estimation.The proposed model highlights the residual region with considerable information and constructs color saliency.Second,we incorporate the content-based color saliency as spatial information in the Gaussian mixture model.The segmentation is performed by clustering each pixel into an appropriate component according to the expectation maximization and maximum criteria.Finally,the random color histogram assigns a unique color to each cluster and creates an attractive color by default for segmentation.A random color histogram serves as an effective tool for data visualization and is instrumental in the creation of generative art,facilitating both analytical and aesthetic objectives.For experiments,we have used the Berkeley segmentation dataset BSDS-500 and Microsoft Research in Cambridge dataset.In the study,the proposed model showcases notable advancements in unsupervised image segmentation,with probabilistic rand index(PRI)values reaching 0.80,BDE scores as low as 12.25 and 12.02,compactness variations at 0.59 and 0.7,and variation of information(VI)reduced to 2.0 and 1.49 for the BSDS-500 and MSRC datasets,respectively,outperforming current leading-edge methods and yielding more precise segmentations.

Identification and effective regulation of scarb1 gene involved in pigmentation change in autotetraploid Carassius auratus

The autotetraploid Carassius auratus(4nRR,4n=200,RRRR)is derived from whole-genome duplication of Carassius auratus red var.(RCC,2n=100,RR).In the current study,we demonstrated that chromatophores and pigment changes directly caused the coloration and variation of 4nRR skin(red in RCC,brownish-yellow in4nRR).To further explore the molecular mechanisms underlying coloration formation and variation in 4nRR,we performed transcriptome profiling and molecular functional verification in RCC and 4nRR.Results revealed that scarb1,associated with carotenoid metabolism,underwent significant down-regulation in 4nRR.Efficient editing of this candidate pigment gene provided clear evidence of its significant role in RCC coloration.Subsequently,we identified four divergent scarb1 homeologs in 4nRR:two original scarb1 homeologs from RCC and two duplicated ones.Notably,three of these homeologs possessed two highly conserved alleles,exhibiting biased and allelespecific expression in the skin.Remarkably,after precise editing of both the original and duplicated scarb1homeologs and/or alleles,4nRR individuals,whether singly or multiply mutated,displayed a transition from brownishyellow skin to a cyan-gray phenotype.Concurrently,the proportional areas of the cyan-gray regions displayed a gene-dose correlation.These findings illustrate the subfunctionalization of duplicated scarb1,with all scarb1genes synergistically and equally contributing to the pigmentation of 4nRR.This is the first report concerning the functional differentiation of duplicated homeologs in an autopolyploidfish,substantiallyenrichingour understanding of coloration formation and change within this group of organisms.

Self‑Healing Dynamic Hydrogel Microparticles with Structural Color for Wound Management

Chronic diabetic wounds confront a significant medical challenge because of increasing prevalence and difficult-healing circumstances.It is vital to develop multifunctional hydrogel dressings,with well-designed morphology and structure to enhance flexibility and effectiveness in wound management.To achieve these,we propose a self-healing hydrogel dressing based on structural color microspheres for wound management.The microsphere comprised a photothermal-responsive inverse opal framework,which was constructed by hyaluronic acid methacryloyl,silk fibroin methacryloyl and black phosphorus quantum dots(BPQDs),and was further re-filled with a dynamic hydrogel.The dynamic hydrogel filler was formed by Knoevenagel condensation reaction between cyanoacetate and benzaldehyde-functionalized dextran(DEX-CA and DEX-BA).Notably,the composite microspheres can be applied arbitrarily,and they can adhere together upon near-infrared irradiation by leveraging the BPQDs-mediated photothermal effect and the thermoreversible stiffness change of dynamic hydrogel.Additionally,eumenitin and vascular endothelial growth factor were co-loaded in the microspheres and their release behavior can be regulated by the same mechanism.Moreover,effective monitoring of the drug release process can be achieved through visual color variations.The microsphere system has demonstrated desired capabilities of controllable drug release and efficient wound management.These characteristics suggest broad prospects for the proposed composite microspheres in clinical applications.

A novel histone methyltransferase gene Cg SDG40 positively regulates carotenoid biosynthesis during citrus fruit ripening

The flesh color of pummelo(Citrus maxima)fruits is highly diverse and largely depends on the level of carotenoids,which are beneficial to human health.It is vital to investigate the regulatory network of carotenoid biosynthesis to improve the carotenoid content in pummelo.However,the molecular mechanism underlying carotenoid accumulation in pummelo is not fully understood.In this study,we identified a novel histone methyltransferase gene,CgSDG40,involved in carotenoid regulation by analyzing the flesh transcriptome of typical white-fleshed pummelo,red-fleshed pummelo and extreme-colored F1 hybrids from a segregated pummelo population.Expression of CgSDG40 corresponded to flesh color change and was highly coexpressed with CgPSY1.Interestingly,CgSDG40 and CgPSY1 are located physically adjacent to each other on the chromosome in opposite directions,sharing a partially overlapping promoter region.Subcellular localization analysis indicated that CgSDG40 localizes to the nucleus.Overexpression of CgSDG40 significantly increased the total carotenoid content in citrus calli relative to that in wild type.In addition,expression of CgPSY1 was significantly activated in overexpression lines relative to wild type.Taken together,our findings reveal a novel histone methyltransferase regulator,CgSDG40,involved in the regulation of carotenoid biosynthesis in citrus and provide new strategies for molecular design breeding and genetic improvement of fruit color and nutritional quality.

Fine-mapping and primary analysis of candidate genes associated with seed coat color in mung bean(Vigna radiata L.)

Seed coat color affects the appearance and commodity quality of mung beans(Vigna radiata L.).The substances that affect mung bean seed coat color are mainly flavonoids,which have important medicinal value.Mapping the seed coat color gene in mung beans would facilitate the development of new varieties and improve their value.In this study,an F2 mapping population consisting of 546 plants was constructed using Jilv9(black seed coat)and BIS9805(green seed coat).Using bulk segregated analysis(BSA)sequencing and kompetitive allele-specific PCR(KASP)markers,the candidate region related to seed coat color was finally narrowed to 0.66 Mb on chromosome(Chr.)4 and included eight candidate genes.Combined transcriptome and metabolome analyses showed that three of the eight candidate genes(LOC106758748,LOC106758747,and LOC106759075)were differentially expressed,which may have caused the differences in flavonoid metabolite content between Jilv9 and BIS9805.These findings can provide a research basis for cloning the genes related to seed coat color and accelerate molecular markerassisted selection breeding in mung beans.