An Improved Encoder-Decoder CNN with Region-Based Filtering for Vibrant Colorization

Colorization is the practice of adding appropriate chromatic values to monochrome photographs or videos.A real-valued luminance image can be mapped to a three-dimensional color image.However,it is a severely ill-defined problem and not has a single solution.In this paper,an encoder-decoder Convolutional Neural Network(CNN)model is used for colorizing gray images where the encoder is a Densely Connected Convolutional Network(DenseNet)and the decoder is a conventional CNN.The DenseNet extracts image features from gray images and the conventional CNN outputs a^(*)b^(*)color channels.Due to a large number of desaturated color components compared to saturated color components in the training images,the saturated color components have a strong tendency towards desaturated color components in the predicted a^(*)b^(*)channel.To solve the problems,we rebalance the predicted a^(*)b^(*)color channel by smoothing every subregion individually using the average filter.2 stage k-means clustering technique is applied to divide the subregions.Then we apply Gamma transformation in the entire a^(*)b^(*)channel to saturate the image.We compare our proposed method with several existing methods.From the experimental results,we see that our proposed method has made some notable improvements over the existing methods and color representation of gray-scale images by our proposed method is more plausible to visualize.Additionally,our suggested approach beats other approaches in terms of Peak Signal-to-Noise Ratio(PSNR),Structural Similarity Index Measure(SSIM)and Histogram.

Colorization method based on the linear relationship assumption

A new local cost function is proposed in this paper based on the linear relationship assumption between the values of the color components and the intensity component in each local image window,then a new quadratic objective function is derived from it and the globally optimal chrominance values can be computed by solving a sparse linear system of equations.Through the colorization experiments on various test images,it is confirmed that the colorized images obtained by our proposed method have more vivid colors and sharper boundaries than those obtained by the traditional method.The peak signal to noise ratio(PSNR) of the colorized images and the average estimation error of the chrominance values relative to the original images also show that our proposed method gives more precise estimation than the traditional method.

Effect of Neodymium on the Crystallization, Microstructure and Colorization of Li₂O-Al₂O₃-SiO₂Glass Ceramics

The effect of single coloring agent Nd(NO3)3 on the crystallization, microstructure and colorization of Li2O-Al2O3-SiO2 (LAS) glass ceramics were investigated by differential thermal analysis (DTA), X-ray diffractometry (XRD) and scanning electron microscopy (SEM). The introduction of little neodymium has no effect on the crystallization manner and the formation of main crystallization phase, but more neodymium will weaken the crystallization of LAS glass. Little neodymium can increase the glossiness of LAS glass ceramic, while more neodymium can color LAS glass with light purple or red purple. The colorability of neodymium for LAS glass ceramic decreases with the increase of crystallization temperature.

Decolorization of Direct Black 22 by Aspergillus ficuum

The decolorization of Direct Black 22 by Aspergillus ficuum has been studied. It was found that Aspergillus ficuum could effectively decolorize Direct Black 22 especially when grown as pelleted mycelia. Results showed that the media containing Direct Black 22 at 50 mg/L could be decolorized by 98.05% of the initial color in 24 h. The optimum pH and temperature of decolorization are 4.0 and 33 °C respectively. Aeration was quite beneficial to decolorization. Medium composition and the concentration of Direct Black 22 could affect the rate of decolorization. The dye degraded products assayed by UV-visible spectrophotometer and macroscopic observation showed that the decolorization of Direct Black 22 by mycelial pellets includes two important processes: bioadsorption and biodegradation. The degradation experiment agree with the Michaelis-Menten kinetics equation.

Temporally consistent video colorization with deep feature propagation and self-regularization learning

Video colorization is a challenging and highly ill-posed problem.Although recent years have witnessed remarkable progress in single image colorization,there is relatively less research effort on video colorization,and existing methods always suffer from severe flickering artifacts(temporal inconsistency)or unsatisfactory colorization.We address this problem from a new perspective,by jointly considering colorization and temporal consistency in a unified framework.Specifically,we propose a novel temporally consistent video colorization(TCVC)framework.TCVC effectively propagates frame-level deep features in a bidirectional way to enhance the temporal consistency of colorization.Furthermore,TCVC introduces a self-regularization learning(SRL)scheme to minimize the differences in predictions obtained using different time steps.SRL does not require any ground-truth color videos for training and can further improve temporal consistency.Experiments demonstrate that our method can not only provide visually pleasing colorized video,but also with clearly better temporal consistency than state-of-the-art methods.A video demo is provided at https://www.youtube.com/watch?v=c7dczMs-olE,while code is available at https://github.com/lyh-18/TCVC-Tem porally-Consistent-Video-Colorization.

Continual few-shot patch-based learning for anime-style colorization

The automatic colorization of anime line drawings is a challenging problem in production pipelines.Recent advances in deep neural networks have addressed this problem;however,collectingmany images of colorization targets in novel anime work before the colorization process starts leads to chicken-and-egg problems and has become an obstacle to using them in production pipelines.To overcome this obstacle,we propose a new patch-based learning method for few-shot anime-style colorization.The learning method adopts an efficient patch sampling technique with position embedding according to the characteristics of anime line drawings.We also present a continuous learning strategy that continuously updates our colorization model using new samples colorized by human artists.The advantage of our method is that it can learn our colorization model from scratch or pre-trained weights using only a few pre-and post-colorized line drawings that are created by artists in their usual colorization work.Therefore,our method can be easily incorporated within existing production pipelines.We quantitatively demonstrate that our colorizationmethod outperforms state-of-the-art methods.

Video Colorization:A Survey

Video colorization aims to add color to grayscale or monochrome videos.Although existing methods have achieved substantial and noteworthy results in the field of image colorization,video colorization presents more formidable obstacles due to the additional necessity for temporal consistency.Moreover,there is rarely a systematic review of video colorization methods.In this paper,we aim to review existing state-of-the-art video colorization methods.In addition,maintaining spatial-temporal consistency is pivotal to the process of video colorization.To gain deeper insight into the evolution of existing methods in terms of spatial-temporal consistency,we further review video colorization methods from a novel perspective.Video colorization methods can be categorized into four main categories:optical-flow based methods,scribble-based methods,exemplar-based methods,and fully automatic methods.However,optical-flow based methods rely heavily on accurate optical-flow estimation,scribble-based methods require extensive user interaction and modifications,exemplar-based methods face challenges in obtaining suitable reference images,and fully automatic methods often struggle to meet specific colorization requirements.We also discuss the existing challenges and highlight several future research opportunities worth exploring.

A review of image and video colorization: From analogies to deep learning

Image colorization is a classic and important topic in computer graphics,where the aim is to add color to a monochromatic input image to produce a colorful result.In this survey,we present the history of colorization research in chronological order and summarize popular algorithms in this field.Early work on colorization mostly focused on developing techniques to improve the colorization quality.In the last few years,researchers have considered more possibilities such as combining colorization with NLP(natural language processing)and focused more on industrial applications.To better control the color,various types of color control are designed,such as providing reference images or color-scribbles.We have created a taxonomy of the colorization methods according to the input type,divided into grayscale,sketch-based and hybrid.The pros and cons are discussed for each algorithm,and they are compared according to their main characteristics.Finally,we discuss how deep learning,and in particular Generative Adversarial Networks(GANs),has changed this field.

Reference-guided structure-aware deep sketch colorization for cartoons

Digital cartoon production requires extensive manual labor to colorize sketches with visually pleasant color composition and color shading.During colorization,the artist usually takes an existing cartoon image as color guidance,particularly when colorizing related characters or an animation sequence.Reference-guided colorization is more intuitive than colorization with other hints,such as color points or scribbles,or text-based hints.Unfortunately,reference-guided colorization is challenging since the style of the colorized image should match the style of the reference image in terms of both global color composition and local color shading.In this paper,we propose a novel learning-based framework which colorizes a sketch based on a color style feature extracted from a reference color image.Our framework contains a color style extractor to extract the color feature from a color image,a colorization network to generate multi-scale output images by combining a sketch and a color feature,and a multi-scale discriminator to improve the reality of the output image.Extensive qualitative and quantitative evaluations show that our method outperforms existing methods,providing both superior visual quality and style reference consistency in the task of reference-based colorization.

Structure-Aware Nonlocal Optimization Framework for Image Colorization

This paper proposes a structure-aware nonlocal energy optimization framework for interactive image colo- rization with sparse scribbles. Our colorization technique propagates colors to both local intensity-continuous regions and remote texture-similar regions without explicit image segmentation. We implement the nonlocal principle by computing k nearest neighbors in the high-dimensional feature space. The feature space contains not only image coordinates and intensities but also statistical texture features obtained with the direction-aligned Gabor wavelet filter. Structure maps are utilized to scale texture features to avoid artifacts along high-contrast boundaries. We show various experimental results and comparisons on image colorization, selective recoloring and decoloring, and progressive color editing to demonstrate the effectiveness of the proposed approach.

Affective Image Colorization

Colorization of gray-scale images has attracted many attentions for a long time. An important role of image color is the conveyer of emotions （through color themes）. The colorization with an undesired color theme is less useful, even it is semantically correct. However this has been rarely considered. Automatic colorization respecting both the semantics and the emotions is undoubtedly a challenge. In this paper~ we propose a complete system for affective image colorization. We only need the user to assist object segmentation along with text labels and an affective word. First, the text labels along with other object characters are jointly used to filter the internet images to give each object a set of semantically correct reference images. Second, we select a set of color themes according to the affective word based on art theories. With these themes, a generic algorithm is used to select the best reference for each object, balancing various requirements. Finally, we propose a hybrid texture synthesis approach for colorization. To the best of our knowledge, it is the first system which is able to efficiently colorize a gray-scale image semantically by an emotionally controllable fashion. Our experiments show the effectiveness of our system, especially the benefit compared with the previous Markov random field （MRF） based method.

Automatic Colorization with Improved Spatial Coherence and Boundary Localization

Grayscale image colorization is an important computer graphics problem with a variety of applications. Recent fully automatic colorization methods have made impressive progress by formulating image colorization as a pixel-wise prediction task and utilizing deep convolutional neural networks. Though tremendous improvements have been made, the result of automatic colorization is still far from perfect. Specifically, there still exist common pitfalls in maintaining color consistency in homogeneous regions as well as precisely distinguishing colors near region boundaries. To tackle these problems, we propose a novel fully automatic colorization pipeline which involves a boundary-guided CRF （conditional random field） and a CNN-based color transform as post-processing steps. In addition, as there usually exist multiple plausible colorization proposals for a single image, automatic evaluation for different colorization methods remains a challenging task. We further introduce two novel automatic evaluation schemes to efficiently assess colorization quality in terms of spatial coherence and localization. Comprehensive experiments demonstrate great quality improvement in results of our proposed colorization method under multiple evaluation metrics.

Robust Backstepping Control of a Quadrotor Unmanned Aerial Vehicle under Colored Noises

Advances in software and hardware technologies have facilitated the production of quadrotor unmanned aerial vehicles(UAVs).Nowadays,people actively use quadrotor UAVs in essential missions such as search and rescue,counter-terrorism,firefighting,surveillance,and cargo transportation.While performing these tasks,quadrotors must operate in noisy environments.Therefore,a robust controller design that can control the altitude and attitude of the quadrotor in noisy environments is of great importance.Many researchers have focused only on white Gaussian noise in their studies,whereas researchers need to consider the effects of all colored noises during the operation of the quadrotor.This study aims to design a robust controller that is resistant to all colored noises.Firstly,a nonlinear quadrotormodel was created with MATLAB.Then,a backstepping controller resistant to colored noises was designed.Thedesigned backstepping controller was tested under Gaussian white,pink,brown,blue,and purple noises.PID and Lyapunov-based controller designswere also carried out,and their time responses(rise time,overshoot,settling time)were compared with those of the backstepping controller.In the simulations,time was in seconds,altitude was in meters,and roll,pitch,and yaw references were in radians.Rise and settling time values were in seconds,and overshoot value was in percent.When the obtained values are examined,simulations prove that the proposed backstepping controller has the least overshoot and the shortest settling time under all noise types.

OptoGPT: A foundation model for inverse design in optical multilayer thin film structures

Optical multilayer thin film structures have been widely used in numerous photonic applications.However,existing inverse design methods have many drawbacks because they either fail to quickly adapt to different design targets,or are difficult to suit for different types of structures,e.g.,designing for different materials at each layer.These methods also cannot accommodate versatile design situations under different angles and polarizations.In addition,how to benefit practical fabrications and manufacturing has not been extensively considered yet.In this work,we introduce OptoGPT(Opto Generative Pretrained Transformer),a decoder-only transformer,to solve all these drawbacks and issues simultaneously.

Non-volatile dynamically switchable color display via chalcogenide stepwise cavity resonators

High-resolution multi-color printing relies upon pixelated optical nanostructures,which is crucial to promote color display by producing nonbleaching colors,yet requires simplicity in fabrication and dynamic switching.Antimony trisulfide(Sb_(2)S_(3))is a newly rising chalcogenide material that possesses prompt and significant transition of its optical characteristics in the visible region between amorphous and crystalline phases,which holds the key to color-varying devices.Herein,we proposed a dynamically switchable color printing method using Sb_(2)S_(3)-based stepwise pixelated Fabry-Pérot(FP)cavities with various cavity lengths.The device was fabricated by employing a direct laser patterning that is a less timeconsuming,more approachable,and low-cost technique.As switching the state of Sb_(2)S_(3) between amorphous and crystalline,the multi-color of stepwise pixelated FP cavities can be actively changed.The color variation is due to the profound change in the refractive index of Sb_(2)S_(3) over the visible spectrum during its phase transition.Moreover,we directly fabricated sub-50 nm nano-grating on ultrathin Sb_(2)S_(3) laminate via microsphere 800-nm femtosecond laser irradiation in far field.The minimum feature size can be further decreased down to~45 nm(λ/17)by varying the thickness of Sb_(2)S_(3) film.Ultrafast switchable Sb_(2)S_(3) photonic devices can take one step toward the next generation of inkless erasable papers or displays and enable information encryption,camouflaging surfaces,anticounterfeiting,etc.Importantly,our work explores the prospects of rapid and rewritable fabrication of periodic structures with nano-scale resolution and can serve as a guideline for further development of chalcogenide-based photonics components.

Coupling of reduced inorganic fertilizer with plant-based organic fertilizer as a promising fertilizer management strategy for colored rice in tropical regions

Colored rice is a type of high-quality,high-added-value rice that has attracted increasing attention in recent years.The use of large amounts of inorganic nitrogen fertilizer in rice fields results in low fertilizer use efficiency and high environmental pollution.Organic fertilizer is a promising way to improve soil quality and sustain high yields.However,most studies focus on the effect of animal-based organic fertilizers.The effects of different ratios of plantbased organic fertilizer and inorganic fertilizer on the grain yield and quality of colored rice have rarely been reported.Therefore,a two-year field experiment was conducted in 2020 and 2021 to study the effects of replacing inorganic N fertilizers with plant-based organic fertilizers on the yield,nitrogen use efficiency(NUE),and anthocyanin content of two colored rice varieties in a tropical region in China.The experimental treatments included no nitrogen fertilization(T1),100% inorganic nitrogen fertilizer(T2),30%inorganic nitrogen fertilizer substitution with plant-based organic fertilizer(T3),60%inorganic nitrogen fertilizer substitution with plant-based organic fertilizer(T4),and 100% plantbased organic fertilizer(T5).The total nitrogen provided to all the treatments except T1 was the same at 120 kg ha-1.Our results showed that the T3 treatment enhanced the grain yield and anthocyanin content of colored rice by increasing nitrogen use efficiency compared with T2.On average,grain yields were increased by 9 and 8%,while the anthocyanin content increased by 16 and 10% in the two colored rice varieties under T3 across the two years,respectively,as compared with T2.Further study of the residual effect of partial substitution of inorganic fertilizers showed that the substitution of inorganic fertilizer with plant-based organic fertilizer improved the soil physiochemical properties,and thus increased the rice grain yield,in the subsequent seasons.The highest grain yield of the subsequent rice crop was observed under the T5 treatment.Our results suggested that the application of plantbased organic fertilizers can sustain the production of colored rice with high anthocyanin content in tropical regions,which is beneficial in reconciling the relationship between rice production and environmental protection.

Self‑Healing Dynamic Hydrogel Microparticles with Structural Color for Wound Management

Chronic diabetic wounds confront a significant medical challenge because of increasing prevalence and difficult-healing circumstances.It is vital to develop multifunctional hydrogel dressings,with well-designed morphology and structure to enhance flexibility and effectiveness in wound management.To achieve these,we propose a self-healing hydrogel dressing based on structural color microspheres for wound management.The microsphere comprised a photothermal-responsive inverse opal framework,which was constructed by hyaluronic acid methacryloyl,silk fibroin methacryloyl and black phosphorus quantum dots(BPQDs),and was further re-filled with a dynamic hydrogel.The dynamic hydrogel filler was formed by Knoevenagel condensation reaction between cyanoacetate and benzaldehyde-functionalized dextran(DEX-CA and DEX-BA).Notably,the composite microspheres can be applied arbitrarily,and they can adhere together upon near-infrared irradiation by leveraging the BPQDs-mediated photothermal effect and the thermoreversible stiffness change of dynamic hydrogel.Additionally,eumenitin and vascular endothelial growth factor were co-loaded in the microspheres and their release behavior can be regulated by the same mechanism.Moreover,effective monitoring of the drug release process can be achieved through visual color variations.The microsphere system has demonstrated desired capabilities of controllable drug release and efficient wound management.These characteristics suggest broad prospects for the proposed composite microspheres in clinical applications.

A chromosome-level genome assembly for Chinese plum‘Wushancuili'reveals the molecular basis of its fruit color and susceptibility to rain-cracking

Chinese plum(Prunus salicina Lindl.)originates from China and makes a large contribution to the global production of plums.The P.salicina‘Wushancuili'has a green coloration and high fruit quality and is economically important in eliminating poverty and protecting ecology in the Yangtze River Three Gorges Reservoir.However,rain-induced cracking(rain-cracking,literally skin cracking caused by rain)is a limitation to‘Wushancuili'fruit production and causes severe losses.This study reported a high-quality‘Wushancuili'genome assembly consisting of a 302.17-Mb sequence with eight pseudo-chromosomes and a contig N50 of 23.59 Mb through the combination of Illumina sequencing,Pacific Biosciences HiFiⅢsequencing,and high-throughput chromosome conformation capture technology.A total of 25109 protein-coding genes are predicted and 54.17%of the genome is composed of repetitive sequences.‘Wushancuili'underwent a remarkable orthoselection during evolution.Gene identification revealed that loss-of-function in four core MYB10 genes results in the anthocyanin deficiency and absence of red color,revealing the green coloration due to the residual high chlorophyll in fruit skin.Besides,the occurrence of cracking is assumed to be closely associated with cell wall modification and frequently rain-induced pathogen enrichment through transcriptomic analysis.The loss of MYB10 genes might render fruit more susceptible to pathogen-mediated cracking by weakening the epidermal strength and reactive oxygen species(ROS)scavenging.Our findings provided fundamental knowledge regarding fruit coloration and rain-cracking and will facilitate genetic improvement and cultivation management in Chinese plums.

The protective effect of cyclodextrin on the color quality and stability of Cabernet Sauvignon red wine

The impact of cyclodextrins(CDs)on wine quality and stability remains largely unknown.This study systematically assessed the protective effect of the post-fermentation addition of CDs on color stability of red wine from the viewpoints of color characteristics,copigmentation and phenolic profiles.The grey relational analysis(GRA)and principal component analysis(PCA)methods were employed to dissect the key effective determinants related to color quality.The addition of CDs induced a significant hyperchromic effect of 8.19-25.40%,a significant bathochromic effect and an enhancement of the color intensity.Furthermore,the evolution of anthocyanin forms and the content of monomeric anthocyanins revealed that β-CD is a superior favorable cofactor during wine aging,but for long-term aging,2-HP-β-CD and 2-HP-γ-CD are more beneficial in promoting the formation of polymerized anthocyanins and color stability.This work provides an important reference for the use of CDs to enhance the color quality and stability of red wines.

Intelligent diagnosis of retinal vein occlusion based on color fundus photographs

AIM:To develop an artificial intelligence(AI)diagnosis model based on deep learning(DL)algorithm to diagnose different types of retinal vein occlusion(RVO)by recognizing color fundus photographs(CFPs).METHODS:Totally 914 CFPs of healthy people and patients with RVO were collected as experimental data sets,and used to train,verify and test the diagnostic model of RVO.All the images were divided into four categories[normal,central retinal vein occlusion(CRVO),branch retinal vein occlusion(BRVO),and macular retinal vein occlusion(MRVO)]by three fundus disease experts.Swin Transformer was used to build the RVO diagnosis model,and different types of RVO diagnosis experiments were conducted.The model’s performance was compared to that of the experts.RESULTS:The accuracy of the model in the diagnosis of normal,CRVO,BRVO,and MRVO reached 1.000,0.978,0.957,and 0.978;the specificity reached 1.000,0.986,0.982,and 0.976;the sensitivity reached 1.000,0.955,0.917,and 1.000;the F1-Sore reached 1.000,0.9550.943,and 0.887 respectively.In addition,the area under curve of normal,CRVO,BRVO,and MRVO diagnosed by the diagnostic model were 1.000,0.900,0.959 and 0.970,respectively.The diagnostic results were highly consistent with those of fundus disease experts,and the diagnostic performance was superior.CONCLUSION:The diagnostic model developed in this study can well diagnose different types of RVO,effectively relieve the work pressure of clinicians,and provide help for the follow-up clinical diagnosis and treatment of RVO patients.