Classification of Human Protein in Multiple Cells Microscopy Images Using CNN

The subcellular localization of human proteins is vital for understanding the structure of human cells.Proteins play a significant role within human cells,as many different groups of proteins are located in a specific location to perform a particular function.Understanding these functions will help in discoveringmany diseases and developing their treatments.The importance of imaging analysis techniques,specifically in proteomics research,is becoming more prevalent.Despite recent advances in deep learning techniques for analyzing microscopy images,classification models have faced critical challenges in achieving high performance.Most protein subcellular images have a significant class imbalance.We use oversampling and under sampling techniques in this research to overcome this issue.We have used a Convolutional Neural Network(CNN)model called GapNet-PL for the multi-label classification task on the Human Protein Atlas Classification(HPA)Dataset.Authors have found that the ParametricRectified LinearUnit(PreLU)activation function is better than the Scaled Exponential LinearUnit(SeLU)activation function in the GapNet-PL model in most classification metrics.The results showed that the GapNet-PL model with the PReLU activation function achieved an area under the ROC curve(AUC)equal to 0.896,an F1 score of 0.541,and a recall of 0.473.

A mathematical morphological approach for region of interest coding of microscopy image compression

A novel mathematical morphological approach for region of interest(ROI) automatic determination and JPEG2000-based coding of microscopy image compression is presented.The algorithm is very fast and requires lower computing power,which is particularly suitable for some irregular region-based cell microscopy images with poor qualities.Firstly,an active threshold-based method is discussed to create a rough mask of regions of interest(cells).And then some morphological operations are designed and applied to achieve the segmentation of cells.In addition,an extra morphological operation,dilation,is applied to create the final mask with some redundancies to avoid the"edge effect"after removing false cells.Finally,ROI and region of background(ROB) are obtained and encoded individually in different compression ratio flexibly based on the JPEG2000,which can adjust the quality between ROI and ROB without coding for ROI shape.The experimental results certify the effectiveness of the proposed algorithm,and compared with JPEG2000,the proposed algorithm has better performance in both subjective quality and objective quality at the same compression ratios.

DeepNoise: Signal and Noise Disentanglement Based on Classifying Fluorescent Microscopy Images via Deep Learning

The high-content image-based assay is commonly leveraged for identifying the phenotypic impact of genetic perturbations in biology field.However,a persistent issue remains unsolved during experiments:the interferential technical noises caused by systematic errors(e.g.,temperature,reagent concentration,and well location)are always mixed up with the real biological signals,leading to misinterpretation of any conclusion drawn.Here,we reported a mean teacher-based deep learning model(Deep Noise)that can disentangle biological signals from the experimental noises.Specifically,we aimed to classify the phenotypic impact of 1108 different genetic perturbations screened from 125,510 fluorescent microscopy images,which were totally unrecognizable by the human eye.We validated our model by participating in the Recursion Cellular Image Classification Challenge,and Deep Noise achieved an extremely high classification score(accuracy:99.596%),ranking the 2nd place among 866 participating groups.This promising result indicates the successful separation of biological and technical factors,which might help decrease the cost of treatment development and expedite the drug discovery process.The source code of Deep Noise is available at https://github.com/Scu-sen/Recursion-Cellular-Image-Classification-Challenge.

Monitoring microenvironment of Hep G2 cell apoptosis using two-photon fluorescence lifetime imaging microscopy

Apoptosis is very important for the maintenance of cellular homeostasis and is closely related to the occurrence and treatment of many diseases.Mitochondria in cells play a crucial role in programmed cell death and redox processes.Nicotinamide adenine dinucleotide(NAD(P)H)is the primary producer of energy in mitochondria,changing NAD(P)H can directly reflect the physiological state of mitochondria.Therefore,NAD(P)H can be used to evaluate metabolic response.In this paper,we propose a noninvasive detection method that uses two-photon fluorescence lifetime imaging microscopy(TP-FLIM)to characterize apoptosis by observing the binding kinetics of cellular endogenous NAD(P)H.The result shows that the average fluorescence lifetime of NAD(P)H and the fluorescence lifetime of protein-bound NAD(P)H will be affected by the changing pH,serum content,and oxygen concentration in the cell culture environment,and by the treatment with reagents such as H2O2 and paclitaxel.Taxol(PTX).This noninvasive detection method realized the dynamic detection of cellular endogenous substances and the assessment of apoptosis.

Examination of Validity of Paraxial Approximation in Second Harmonic Generation Microscopy under Low Numerical Aperture

By using a vectorial approach, the validity of paraxial approximation in second harmonic generation （SHG） microscopy under low numerical aperture （NA） is examined when the sample is a collagen fibril. Due to the larger value of dzzz and tensorial nature of SHG, the component Ez of the focused fieM may have strong effect on the radiation pattern of SHG. Numerical results indicate that when the value of NA exceeds 0.3, the effect of Ez can not be neglected, which results in the invalidation of paraxial approximation in SHG microscopy despite the fact that SHG microscopy is still under low NA focusing.

Fluorescence life-time imaging microscopy(FLIM)monitors tumor cell death triggered by photothermal therapy with MoS_(2) nanosheets

Recently,photothermal therapy(PTT)has been proved to have great potential in tumor therapy.In the last several years,MoS_(2),as one novel member of nanomaterials,has been applied into PTT due to its excellent photothermal conversion efficacy.In this work,we applied fuorescence lifetime imaging microscopy(FLIM)techniques into monitoring the PPT-triggered cell death under MoS_(2) nanosheet treatment.Two types of MoS_(2) nanosheets(single layer nanosheets and few layer nanosheets)were obtained,both of which exhibited presentable photothermal conversion fficacy,leading to high cell death rates of 4T1 cells(mouse breast cancer cells)under PTT.Next,live cell images of 4T1 cells were obtained via directly labeling the mitochondria with Rodamine123,which were then continuously observed with FLIM technique.FLIM data showed that the fuorescence lifetimes of mitochondria targeting dye in cells treated with each type of MoS_(2) nanosheets significantly increased during PTT treatment.By contrast,the fuorescence lifetime of the same dye in control cells(without nanomaterials)remained constant after laser irradiation.These findings suggest that FLIM can be of great value in monitoring cell death process during PTT of cancer cells,which could provide dynamic data of the cellular microenvironment at single cell level in multiple biomedical applications.

Rapid screening and visual tracing of melamine in soybean meal by NIR microscopy imaging

Near infrared microscopy imaging fers the opportunity to explore not only what lkinds ofchemical species are present at micro-scale level but also where the chemical species would bepr esent.By revealing the spectral and spatial information,the technique can identify and localizeany interested component.This study investigates the feasibility of using Near infrared mi.croscopy imaging to detect melamine in soybean meal.The results showed that 6805 cm^(-1) is verysensitive for melamine but not for soybean meal,so can be used for univariate analysis,Singlewavelength image and peak integr ation image at 6805 cm^(-1) are simple and efective met hods todetect the melamine in soybean meal.Furthermore,Principal Component Analysis is applied todetect the melamine in soybean meal.

NON-LINEAR SPECTRAL IMAGING MICROSCOPY STUDIES OF HUMAN HYPERTROPHIC SCAR

Skin scar is unique to humans,the major significant negative outcome sustained after thermal injuries,traumatic injuries,and surgical procedures.Hypertrophic scar in human skin is investigated using non-linear spectral imaging microscopy.The high contrast images and spectroscopic intensities of collagen and elastic fibers extracted from the spectral imaging of normal skin tissue,and the normal skin near and far away from the hypertrophic scar tissues in a 10-year-old patient case are obtained.The results show that there are apparent differences in the morphological structure and spectral characteristics of collagen and elastic fibers when comparing the normal skin with the hypertrophic scar tissue.These differences can be good indicators to differentiate the normal skin and hypertrophic scar tissue and demonstrate that non-linear spectral imaging microscopy has potential to noninvasively investigate the pathophysiology of human hypertrophic scar.

Compact and effective photon-resolved image scanning microscope

Fluorescence confocal laser-scanning microscopy(LSM)is one of the most popular tools for life science research.This popularity is expected to grow thanks to single-photon array detectors tailored for LSM.These detectors offer unique single-photon spatiotemporal information,opening new perspectives for gentle and quantitative superresolution imaging.However,a flawless recording of this information poses significant challenges for the microscope data acquisition(DAQ)system.We present a DAQ module based on the digital frequency domain principle,able to record essential spatial and temporal features of photons.We use this module to extend the capabilities of established imaging techniques based on single-photon avalanche diode(SPAD)array detectors,such as fluorescence lifetime image scanning microscopy.Furthermore,we use the module to introduce a robust multispecies approach encoding the fluorophore excitation spectra in the time domain.Finally,we combine time-resolved stimulated emission depletion microscopy with image scanning microscopy,boosting spatial resolution.Our results demonstrate how a conventional fluorescence laser scanning microscope can transform into a simple,information-rich,superresolved imaging system with the simple addition of a SPAD array detector with a tailored data acquisition system.We expected a blooming of advanced single-photon imaging techniques,which effectively harness all the sample information encoded in each photon.

Texture of friction stir welded Ti-6Al-4V alloy

Theα＋βtitanium alloy, Ti-6Al-4V, was welded by friction stir welding using a W-Re pin tool, and the defect-free weld was produced with proper welding parameters. Texture of the Ti-6Al-4V friction stir weld was studied by orientation imaging microscopy. The as-received Ti-6Al-4V sheet mill annealed was composed of elongated primary α and transformed β. A typical rolling texture was observed in the base material. The microstructure of the stir zone was significantly different from that of the base material. The stir zone was characterized by the presence of considerable amount of equiaxed dynamically recrystallized grains and a texture around{Ф1=30°,φ=62°,Ф2=30°}was developed during the friction stir welding.

Aggregation-induced emission luminogen for in vivo three-photon fuorescence lifetime microscopic imaging

Compared with visible light,near infrared(NIR)light has deeper penetration in biological tisues.Three-photon fuorescence microscopy(3PFM)can effectively utilize the NIR excitation to obtain high-contrast images in the deep tisue.However,the weak three photon fluorescence signals may be not well presented in the traditional fuorescence intensity imaging mode.Fluorescence lifetime of certain probes is insensitive to the intensity of the excitation laser.Moreover,fluorescence lifetimne imaging microscopy(FLIM)can detect weak signals by utilizing time correlated single photon counting(TCSPC)technique.Thus,it would be an improved strategy to combine the 3PFM imaging with the FLIM together.Herein,DCDPP-2TPA,a novel agegation-induced emission luminogen(AIEgen),was adopted as the fluorescent probes.The three-photon absorption cros-section of the AlEgen,which has a deep-red fluorescence emission,was proved to be large.DCDPP-2TPA nanoparticles were synthesized,and the three photon fluorescence lifetime of which was measured in water.Moreover,in vrivo thre-photon fuorescence lifetime microscopic imaging of a craniotomy mouse was conducted via a home made optical system.High contrast cerebrovascular images of different vertical depths were obtained and the maximun depth was about 600 pumn.Even reaching the depth of 600 pum,tiny capillary vessels as small as 1.9 pum could still be distinguished.The three photon fuorescence lifetimes of the capillaries in some representative images were in accord with that of DCDPP-2TPA nanoparticles in water.A vivid 3D reconstruction was further organized to present a wealth of lifetime information.In the future,the combination strategy of 3PFM and FLIM could be further applied in the brain functional imaging.

Applications,of fluorescence lifetime imaging in clinical medicine

Fluorescence lifetime is not only associated with the molecular structure f fuorophores,but alsostrongly depends on the environment around them,which llows fuorescence lifetime imagingmicroscopy(FLIM)to be used as a tool for precise measurement of the cell or tisue microenvironment,This review introduces the basic principle of fuorescence lifetime imagingtechnology and its application in clinical medicine,including research and diagnosis of diseases inskin,brain,eyes,mouth,bone,blood vessels and cavity organs,and drug evaluation.As anoninvasive,nontoxic and nonionizing radiation technique,FLIM demonstrates excellent per-formance with high sensitivity and specificity,which allows to determine precise position of thelesion and,thus,has good potential for application in biomedical research and clinical diagnosis.

FLIM as a Promising Tool for Cancer Diagnosis and Treatment Monitoring

Fluorescence lifetime imaging microscopy(FLIM)has been rapidly developed over the past 30 years and widely applied in biomedical engineering.Recent progress in fluorophore-dyed probe design has widened the application prospects of fluorescence.Because fluorescence lifetime is sensitive to microenvironments and molecule alterations,FLIM is promising for the detection of pathological conditions.Current cancer-related FLIM applications can be divided into three main categories:(i)FLIM with autofluorescence molecules in or out of a cell,especially with reduced form of nicotinamide adenine dinucleotide,and flavin adenine dinucleotide for cellular metabolism research;(ii)FLIM with Förster resonance energy transfer for monitoring protein interactions;and(iii)FLIM with fluorophore-dyed probes for specific aberration detection.Advancements in nanomaterial production and efficient calculation systems,as well as novel cancer biomarker discoveries,have promoted FLIM optimization,offering more opportunities for medical research and applications to cancer diagnosis and treatment monitoring.This review summarizes cutting-edge researches from 2015 to 2020 on cancer-related FLIM applications and the potential of FLIM for future cancer diagnosis methods and anti-cancer therapy development.We also highlight current challenges and provide perspectives for further investigation.

Implementation and application of FRET-FLIM technology

With the development of the new detection methods and the function of fluorescent molecule,researchers hope to further explore the internal mechanisms of organisms,monitor changes in the intracellular microenvironment,and dynamic processes of molecular interactions in cells.Fluo-rescence resonance energy transfer(FRET)describes the energy transfer process between donor fluorescent molecules and acceptor fluorescent molecules.It is an important means to detect protein-protein interactions and protein conformation changes in cells.Fluorescence lifetime imaging microscopy(FLIM)enables noninvasive measurement of the fAuorescence lifetime of fluorescent particles in vivo.The FRET-FLIM technology,which is use FLIM to quantify and analyze FRET,enables real-time monitoring of dynamic changes of proteins in biological cells and analysis of protein interaction mechanisms.The distance between donor and acceptor and their respective fAuorescent lifetime,which are of great importance for studying the mechanism of intracellular activity can be obtained by data analysis and algorithm ftting.

Fast fluorescence lifetime imaging techniques:A review on challenge and development

Fluorescence lifetime imaging microscopy(FLIM)is increasingly used in biomedicine,material science,chemistry,and other related research fields,because of its advantages of high specificity and sensitivity in monitoring cellular microenvironments,studying interaction between proteins,metabolic state,screening drugs and analyzing their efficacy,characterizing novel materials,and diagnosing early cancers.Understandably,there is a large interest in obtaining FLIM data within an acquisition time as short as possible.Consequently,there is currently a technology that advances towards faster and faster FLIM recording.However,the maximum speed of a recording technique is only part of the problerm.The acquisition time of a FLIM image is a complex function of many factors.These include the photon rate that can be obtained from the sample,the amount of information a technique extracts from the decay functions,the fficiency at which it determines fluorescence decay parameters from the recorded photons,the demands for the accuracy of these parameters,the number of pixels,and the lateral and axial resolutions that are obtained in biological materials.Starting from a discussion of the parameters which determine the acquisition time,this review will describe existing and emerging FLIM techniques and data analysis algo-rithms,and analyze their performance and recording speed in biological and biomedical applications.

Metabolic state oscillations in cerebral nuclei detected using two-photon fluorescence lifetime imaging microscopy

The fluorescence lifetime of nicotinamide adenine dinucleotide(NADH),a key endogenous coenzyme and metabolic biomarker,can reflect the metabolic state of cells.To implement metabolic imaging of brain tissue at high resolution,we assembled a two-photon fluorescence lifetime imaging microscopy(FLIM)platform and verified the feasibility and stability of NADH-based two-photon FLIM in paraformaldehydefixed mouse cerebral slices.Furthermore,NADH based metabolic state oscillation was observed in cerebral nuclei suprachiasmatic nucleus(SCN).The free NADH fraction displayed a relatively lower level in the daytime than at the onset of night,and an ultradian oscillation at night was observed.Through the combination of high-resolution imaging and immunostaining data,the metabolic tendency of different cell types was detected after the first two hours of the day and at night.Thus,two-photon FLIM analysis of NADH in paraformaldehyde-fixed cerebral slices provides a high-resolution and label-free method to explore the metabolic state of deep brain regions.

Iterative multi-photon adaptive compensation technique for deep tissue two-photon fluorescence lifetime imaging

Fluorescence lifetime imaging can reveal the high-resolution structure of various biophysical and chemical parameters in a microenvironment quantitatively.However,the depth of imaging is generally limited to hundreds of micrometers due to aberration and light scattering in biological tissues.This paper introduces an iterative multi-photon adaptive compensation technique(IMPACT)into a two-photon fluorescence lifetime microscopy system to successfully overcome aberrations and multiple scattering problems in deep tissues.It shows that 400 correction modes can be achieved within 5 min,which was mainly limited by the frame rate of a spatial light modulator.This system was used for high-resolution imaging of mice brain tissue and live zebrafish,further verifying its superior performance in imaging quality and photon accumulation speed.

Long-term live-cell microscopy with labeled nanobodies delivered by laser-induced photoporation

Fluorescence microscopy is the method of choice for studying intracellular dynamics.However,its success depends on the.availability of specific and stable markers.A prominent example of markers that are rapidly gaining interest are nanobodies(Nbs.-15 kDa),which can be functionalized with bright and photostable organic fluorophores.Due to their relatively small size and high specificity,Nbs offer great potential for high-quality long-term subcellular imaging,but suffer from the fact that they cannot spontaneously cross the plasma membrane of live cells.We have recently discovered that laser-induced photoporation is well suited to deliver extrinsic labels to living cells without compromising their viability.Being a laser-based technology,it is readily compatible with light microscopy and the typical cell recipients used for that.Spurred by these promising initial results,we demonstrate here for the first time successful long-term imaging of specific subcellular structures with labeled nanobodies in living cells.We illustrate this using Nbs that target GFP/YFP-protein constructs accessible in the cytoplasm,actin-bundling protein Fascin,and the histone H2A/H2B heterodimers.With an efficiency of more than 80%labeled cells and minimal toxicity(-2%),photoporation proved to be an excellent intracellular delivery method for Nbs.Time-lapse microscopy revealed that cell division rate and migration remained unaffected,confirming excellent cell viability and functionality.We conclude that laser-induced photoporation labeled Nbs can be easily delivered into living cells,laying the foundation for further development of a broad range of Nbs with intracellular targets as a toolbox for long-term live-cell microscopy.

In cell measurement of fluorescence lifetime imaging microscopy revealed C-terminal conformation changes of Ferroportin upon addition of Mn^2＋

Fluorescence microscopy, as a sensitive method to detect microenvironment of molecules, is widely used in protein conformation and dynamic studies in live cells. Fluorescence lifetime imaging microscopy（FLIM）, which is independent of fluorophore concentrations, scattering and bleaching, is a suitable tool to analyze membrane proteins in a single cell. Ferroportin（FPN）, a multi-ion exporter in vertebrates, was modulated by metal ions with unknown mechanism. Herein, we fused green fluorescence protein on Cterminal of FPN（FPN-eGFP） and applied fluorescence lifetime to monitor conformation changes of FPN in a live cell. The fluorescence lifetime distribution showed a shift to shorter lifetime upon Mn^（2＋） treatment,suggesting a preference conformation of FPN in Mn^（2＋） exposure. It is also observed that the lifetime（rather than intensity） measurement was not strongly influenced by laser power. The observed fluorescence lifetime changes of FPN-eGFP upon Mn^（2＋） treatments indicated that extracellular metal ions can modulate FPN through conformation exchanges between several different states.

3D depth-coded photoacoustic microscopy with a large field of view for human skin imaging

Photoacoustic （PA） microscopy comes with high potential for human skin imaging, since it allows noninvasively high-resolution imaging of the natural hemoglobin at depths of several millimeters. Here, we developed a PA microscopy to achieve high-resolution, high-contrast, and large field of view imaging of skin. A three-dimensional （3D） depth-coding technology was used to encode the depth information in PA images, which is very intuitive for identifying the depth of blood vessels in a two-dimensional image, and the vascular structure can be analyzed at different depths. Imaging results demonstrate that the 3D depth-coded PA microscopy should be translated from the bench to the bedside.