Estimation-free spatial-domain image reconstruction of structured illumination microscopy

Structured illumination microscopy(SIM)achieves super-resolution(SR)by modulating the high-frequency information of the sample into the passband of the optical system and subsequent image reconstruction.The traditional Wiener-filtering-based reconstruction algorithm operates in the Fourier domain,it requires prior knowledge of the sinusoidal illumination patterns which makes the time-consuming procedure of parameter estimation to raw datasets necessary,besides,the parameter estimation is sensitive to noise or aberration-induced pattern distortion which leads to reconstruction artifacts.Here,we propose a spatial-domain image reconstruction method that does not require parameter estimation but calculates patterns from raw datasets,and a reconstructed image can be obtained just by calculating the spatial covariance of differential calculated patterns and differential filtered datasets(the notch filtering operation is performed to the raw datasets for attenuating and compensating the optical transfer function(OTF)).Experiments on reconstructing raw datasets including nonbiological,biological,and simulated samples demonstrate that our method has SR capability,high reconstruction speed,and high robustness to aberration and noise.

Speckle structured illumination endoscopy with enhanced resolution at wide field of view and depth of field

Structured illumination microscopy(SIM)is one of the most widely applied wide field super resolution imaging techniques with high temporal resolution and low phototoxicity.The spatial resolution of SIM is typically limited to two times of the diffraction limit and the depth of field is small.In this work,we propose and experimentally demonstrate a low cost,easy to implement,novel technique called speckle structured illumination endoscopy(SSIE)to enhance the resolution of a wide field endoscope with large depth of field.Here,speckle patterns are used to excite objects on the sample which is then followed by a blind-SIM algorithm for super resolution image reconstruction.Our approach is insensitive to the 3D morphology of the specimen,or the deformation of illuminations used.It greatly simplifies the experimental setup as there are no calibration protocols and no stringent control of illumination patterns nor focusing optics.We demonstrate that the SSIE can enhance the resolution 2–4.5 times that of a standard white light endoscopic(WLE)system.The SSIE presents a unique route to super resolution in endoscopic imaging at wide field of view and depth of field,which might be beneficial to the practice of clinical endoscopy.

New LED light source system with uniform illumination for line-scan CCD imaging systems

A system with uniform light reflection in the inner surface within a horizontal 2/3 cylindrical structure for line-scan CCD of the print testing was designed. The design was based on diffuse reflection uniformity of the integrating sphere and requirement of the strip uniform illumination region. This system was called dome light. White light LED array light sources were used for uniform illumi- nation. The LEDs were filtrated to composite array light source based on coefficient of variation of a single LED. The standard white board and SG color checkers were used in the line-scan CCD imaging experiments under the dome light and ordinary illumination light source. The average color difference （AE） of SG color checkers in CIELAB space was 2. 091 under the dome light and 2. 286 under ordinary illumination light source respectively. Experimental results indicate that the dome light can satisfy illumination uniformity and color rendering consistency for line-scan CCD and provide a standard light source for uniform calibration of different cameras.

Miniaturization of a coherent monocular structured illumination system for future combination with digital holography

Holographic and 3D-measurement processes are an often-used tool in industry,medicine,and scientific applications.While small deviations of objects can be visualized by holographic means with high accuracy,optical systems with active structured illumination are a reliable source of absolute 3D-information in these fields.The combination of digital holography with structured illumination allows to simultaneously measure deformations and absolute 3D coordinates but also requires coherent light and has already been demonstrated in principle with a stereo camera setup.Multi-camera systems are limited to certain setup sizes given by the volume and distance of the detectors.Reducing the system to a one-camera(monocular)setup reduces space and acquisition costs.By using a multi-aperture illumination source an extremely high projection rate could be realized and reduced to a monocular approach with a novel voxel-calibration technique,while the projection system itself still requires a large amount of space.In this paper we present a miniaturized,monocular 3D-measurement system that works with repeatable,coherent speckles,generated by a fiber-coupled laser whose light was distributed by a fiber-switch to a diffuser plate connected with a measurement-head,also including a camera.By addressing different fibers through the switch,varying but repeatable patterns are generated.The size of the device(diameter<3 cm)is now mainly limited by the volume of the camera.A first 3D-reconstruction of an object and an outlook for a combination of this system with digital holography is given,allowing absolute 3D-coordinates and relative deviations of object points to be measured simultaneously.

Compact and robust dual-color linearly polarized illumination source for three-photon fluorescence imaging

The miniaturized femtosecond laser in near infrared-Ⅱregion is the core equipment of threephoton microscopy.In this paper,we design a compact and robust illumination source that emits dual-color linearly polarized light for three-photon microscopy.Based on an all-polarizationmaintaining passive mode-locked fiber laser,we shift the center wavelength of the pulses to the 1.7m band utilizing cascade Raman effect,thereby generate dual-wavelength pulses.To enhance clarity,the two wavelengths are separated through the graded-index multimode fiber.Then we obtain the dual-pulse sequences with 1639.4 nm and 1683.7 nm wavelengths,920 fs pulse duration,and 23.75 MHz pulse repetition rate.The average power of the signal is 53.64mW,corresponding to a single pulse energy of 2.25 nJ.This illumination source can be further amplified and compressed for three-photon fluorescence imaging,especially dual-color three-photon fluorescence imaging,making it an ideal option for biomedical applications.

Complex-amplitude Fourier single-pixel imaging via coherent structured illumination

We propose a method of complex-amplitude Fourier single-pixel imaging(CFSI)with coherent structured illumination to acquire both the amplitude and phase of an object.In the proposed method,an object is illustrated by a series of coherent structured light fields,which are generated by a phase-only spatial light modulator,the complex Fourier spectrum of the object can be acquired sequentially by a single-pixel photodetector.Then the desired complex-amplitude image can be retrieved directly by applying an inverse Fourier transform.We experimentally implemented this CFSI with several different types of objects.The experimental results show that the proposed method provides a promising complex-amplitude imaging approach with high quality and a stable configuration.Thus,it might find broad applications in optical metrology and biomedical science.

Comparison of point detection and area detection for point-scanning structured illumination microscopy

Structured illumination microscopy(SIM)is suitable for biological samples because of its relatively low-peak illumination intensity requirement and high imaging speed.The system resolution is affected by two typical detection modes:Point detection and area detection.However,a systematic analysis of the imaging performance of the different detection modes of the system has rarely been conducted.In this study,we compared laser point scanning point detection(PS-PD)and point scanning area detection(PS-AD)imaging in nonconfocal microscopy through theoretical analysis and simulated imaging.The results revealed that the imaging resolutions of PSPD and PS-AD depend on excitation and emission point spread functions(PSFs),respectively.Especially,we combined the second harmonic generation(SHG)of point detection(P-SHG)and area detection(A-SHG)with SIM to realize a nonlinear SIM-imaging technique that improves the imaging resolution.Moreover,we analytically and experimentally compared the nonlinear SIM performance of P-SHG with that of A-SHG.

Deep Facial Emotion Recognition Using Local Features Based on Facial Landmarks for Security System

Emotion recognition based on facial expressions is one of the most critical elements of human-machine interfaces.Most conventional methods for emotion recognition using facial expressions use the entire facial image to extract features and then recognize specific emotions through a pre-trained model.In contrast,this paper proposes a novel feature vector extraction method using the Euclidean distance between the landmarks changing their positions according to facial expressions,especially around the eyes,eyebrows,nose,andmouth.Then,we apply a newclassifier using an ensemble network to increase emotion recognition accuracy.The emotion recognition performance was compared with the conventional algorithms using public databases.The results indicated that the proposed method achieved higher accuracy than the traditional based on facial expressions for emotion recognition.In particular,our experiments with the FER2013 database show that our proposed method is robust to lighting conditions and backgrounds,with an average of 25% higher performance than previous studies.Consequently,the proposed method is expected to recognize facial expressions,especially fear and anger,to help prevent severe accidents by detecting security-related or dangerous actions in advance.

Radial Basis Approximations Based BEMD for Enhancement of Non-Uniform Illumination Images

An image can be degraded due to many environmental factors like foggy or hazy weather,low light conditions,extra light conditions etc.Image captured under the poor light conditions is generally known as non-uniform illumination image.Non-uniform illumination hides some important information present in an image during the image capture Also,it degrades the visual quality of image which generates the need for enhancement of such images.Various techniques have been present in literature for the enhancement of such type of images.In this paper,a novel architecture has been proposed for enhancement of poor illumination images which uses radial basis approximations based BEMD(Bi-dimensional Empirical Mode Decomposition).The enhancement algorithm is applied on intensity and saturation components of image.Firstly,intensity component has been decomposed into various bi-dimensional intrinsic mode function and residue by using sifting algorithm.Secondly,some linear transformations techniques have been applied on various bidimensional intrinsic modes obtained and residue and further on joining the transformed modes with residue,enhanced intensity component is obtained.Saturation part of an image is then enhanced in accordance to the enhanced intensity component.Final enhanced image can be obtained by joining the hue,enhanced intensity and enhanced saturation parts of the given image.The proposed algorithm will not only give the visual pleasant image but maintains the naturalness of image also.

Adaptive Design of Fluorescence Imaging Systems for Custom Resolution, Fields of View, and Geometries

Objective and Impact Statement:We developed a generalized computational approach to design uniform,high-intensity excitation light for low-cost,quantitative fluorescence imaging of in vitro,ex vivo,and in vivo samples with a single device.Introduction:Fluorescence imaging is a ubiquitous tool for biomedical applications.Researchers extensively modify existing systems for tissue imaging,increasing the time and effort needed for translational research and thick tissue imaging.These modifications are applicationspecific,requiring new designs to scale across sample types.Methods:We implemented a computational model to simulate light propagation from multiple sources.Using a global optimization algorithm and a custom cost function,we determined the spatial positioning of optical fibers to generate 2 illumination profiles.These results were implemented to image core needle biopsies,preclinical mammary tumors,or tumor-derived organoids.Samples were stained with molecular probes and imaged with uniform and nonuniform illumination.Results:Simulation results were faithfully translated to benchtop systems.We demonstrated that uniform illumination increased the reliability of intraimage analysis compared to nonuniform illumination and was concordant with traditional histological findings.The computational approach was used to optimize the illumination geometry for the purposes of imaging 3 different fluorophores through a mammary window chamber model.Illumination specifically designed for intravital tumor imaging generated higher image contrast compared to the case in which illumination originally optimized for biopsy images was used.Conclusion:We demonstrate the significance of using a computationally designed illumination for in vitro,ex vivo,and in vivo fluorescence imaging.Applicationspecific illumination increased the reliability of intraimage analysis and enhanced the local contrast of biological features.This approach is generalizable across light sources,biological applications,and detectors.

ED-Ged:Nighttime Image Semantic Segmentation Based on Enhanced Detail and Bidirectional Guidance

Semantic segmentation of driving scene images is crucial for autonomous driving.While deep learning technology has significantly improved daytime image semantic segmentation,nighttime images pose challenges due to factors like poor lighting and overexposure,making it difficult to recognize small objects.To address this,we propose an Image Adaptive Enhancement(IAEN)module comprising a parameter predictor(Edip),multiple image processing filters(Mdif),and a Detail Processing Module(DPM).Edip combines image processing filters to predict parameters like exposure and hue,optimizing image quality.We adopt a novel image encoder to enhance parameter prediction accuracy by enabling Edip to handle features at different scales.DPM strengthens overlooked image details,extending the IAEN module’s functionality.After the segmentation network,we integrate a Depth Guided Filter(DGF)to refine segmentation outputs.The entire network is trained end-to-end,with segmentation results guiding parameter prediction optimization,promoting self-learning and network improvement.This lightweight and efficient network architecture is particularly suitable for addressing challenges in nighttime image segmentation.Extensive experiments validate significant performance improvements of our approach on the ACDC-night and Nightcity datasets.

Efficacy and safety of Usights UC100 illuminated microcatheter in microcatheter-assisted trabeculotomy

AIM:To evaluate the efficacy and safety of Usights UC100 illuminated microcatheter in microcatheter-assisted trabeculotomy(MAT).METHODS:Totally 10 eyes of 10 patients with primary open angle glaucoma(POAG)who underwent MAT facilitated by Usights UC100(5 eyes)or iTrack(5 eyes)were reviewed.The success of this surgery was defined as intraocular pressure(IOP)<22 mm Hg with>30%reduction,without oral glaucoma medications,or additional glaucoma surgery.RESULTS:The mean pre-operative IOP was 25.38±10.22 mm Hg in the Usights UC100 group and 19.98±3.87 mm Hg in the iTrack group.MAT was achieved in all eyes in both groups.The success rates for the Usights UC100 group and iTrack groups were in all and 4 eyes,respectively.Both microcatheters produced a statistically significant reduction in IOP,and eyes using Usights UC100 achieved a lower IOP than the iTrack group at 3mo followup(12.58±1.52 and 14.84±1.89 mm Hg,respectively),but no statistical significance was there.No severe side effects were observed in either group.CONCLUSION:MAT using Usights UC100 or iTrack both achieve significant pressure reduction in cases of POAG,and Usights UC100 is as safe as iTrack.

Nonlinear scanning structured illumination microscopy based on nonsinusoidal modulation

Structured illumination microscopy(SIM)is an essential super-resolution microscopy technique that enhances resolution.Several images are required to reconstruct a super-resolution image.However,linear SIM resolution enhancement can only increase the spatial resolution of micros-copy by a factor of two at most because the frequency of the structured illumination pattern is limited by the cutoff frequency of the excitation point spread function.The frequency of the pattern generated by the nonlinear response in samples is not limited;therefore,nonlinear SIM(NL-SIM),in theory,has no inherent limit to the resolution.In the present study,we describe a two-photon nonlinear SIM(2P-SIM)technique using a multiple harmonics scanning pattern that employs a composite structured illumination pattern,which can produce a higher order harmonic pattern based on the fluorescence nonlinear response in a 2P process.The theoretical models of super-resolution imaging were established through our simulation,which describes the working mechanism of the multi-frequency structure of the nonsinusoidal function to improve the reso-lution.The simulation results predict that a 5-fold improvement in resolution in the 2P-SIM is possible.

Structured illumination microscopy based on asymmetric three-beam interference

Structured illumination microscopy(SIM)is a rapidly developing super-resolution technology.It has been widely used in various application fields of biomedicine due to its excellent two-and three-dimensional imaging capabilities.Furthermore,faster three-dimensional imaging methods are required to help enable more research-oriented living cell imaging.In this paper,a fast and sensitive three-dimensional structured illumination microscopy based on asymmetric three-beam interference is proposed.An innovative time-series acquisition method is employed to halve the time required to obtain each raw image.A segmented half-wave plate as a substantial linear polarization modulation method is applied to the three-dimensional SIM system for the first time.Although it needs to acquire 21 raw images instead of 15 to reconstruct one super-resolution image,the SIM setup proposed in this paper is 30%faster than the traditional spatial light modulator-SIM(SLM-SIM)in imaging each super-resolution image.The related theoretical derivation,hardware system,and verification experiment are elaborated in this paper.The stable and fast 3D super-resolution imaging method proposed in this paper is of great significance to the research of organelle interaction,intercellular communication,and other biomedical fields.

DECREASING PITTING SUSCEPTIBILITY OF PASSIVE FILMS ON X70 PIPELINE STEEL IN NaCl SOLUTIONS BY ILLUMINATION

The influence of UV illumination on passivity and pitting susceptibility on X70 pipoeline steel in a borate buffer （pH=8.4 ） solution containing NaCl is described. It is observed that illumination of the sample leads to a decrease in its pitting susceptibility as indicated by pitting potential and incubation time measurements in chloride containing electrolytes. This effect is strongly dependent on the applied potential during passivation. The electronic properties of the passive films on X70 steel were studied by Mott-Schottky analysis and photocurrent transient measurements. The results indicated that illumination during passivation led to modifications in the electronic properties of the passive films, mainly to a decrease of the bulk doping and an increase in the surface state density. The cause for the decrease in the pitting susceptibility is preliminary explained in terms of such modifications of the passive flm.

Adaptive illumination light source for online machine vision inspection of tin steel strips

The different reflection characteristics of the surface of tin steel strips and the different speeds of the tinning line demand an adaptive illumination light source for online machine vision inspection. This light source can be integrated with a time delay integration charge-coupled device （TDI CCD ） to capture the images of moving objects and facilitate inspection of the surface quality of tin steel strips. On-site application show the effectiveness of the TDI camera with the adaptive illumination light source in detecting the surface defects on tin steel strips of three different materials and with different tin coating weights.

Structured illumination microscopy and its new developments

Optical microscopy allows us to observe the biological structures and processes within living cells.However,the spatial resolution of the optical microscopy is limited to about half of the wavelength by the light di®raction.Structured illumination microscopy(SIM),a type of new emerging super-resolution microscopy,doubles the spatial resolution by illuminating the specimen with a patterned light,and the sample and light source requirements of SIM are not as strict as the other super-resolution microscopy.In addition,SIM is easier to combine with the other imaging techniques to improve their imaging resolution,leading to the developments of diverse types of SIM.SIM has great potential to meet the various requirements of living cells imaging.Here,we review the recent developments of SIM and its combination with other imaging techniques.

Modulated illumination localization microscopy-enabled sub-10 nm resolution

Optical microscopy is an essential tool for exploring the structures and activities of cells and tissues.To break the limit of resolution caused by diffraction,researchers have made continuous advances and innovations to improve the resolution of optical microscopy since the 1990s.These contributions,however,still make sub-10nm imaging an obstacle.Here,we name a series of technologies as modulated illumination localization microscopy(MILM),which makes ultra-high-resolution imaging practical.Besides,we review the recent progress since 2017 when MINFLUX was proposed and became the inspiration and foundation for the follow-up devel-opment of MILM.This review divides MILM into two types:point-scanning and wide-field.The schematics,principles and future research directions of MILM are discussed elaborately.

Optimization of laser illumination configuration for directly driven inertial confinement fusion

Optimum laser configurations are presented to achieve high illumination uniformity with directly driven inertial confinement fusion targets.Assuming axisymmetric absorption pattern of individual laser beams,theoretical models are reviewed in terms of the number of laser beams,system imperfection,and laser beam patterns.Utilizing a self-organizing system of charged particles on a sphere,a simple numerical model is provided to give an optimal configuration for an arbitrary number of laser beams.As a result,such new configurations as“M48”and“M60”are found to show substantially higher illumination uniformity than any other existing direct drive systems.A new polar direct-drive scheme is proposed with the laser axes keeping off the target center,which can be applied to laser configurations designed for indirectly driven inertial fusion.

Structured Illumination Chip Based on Integrated Optics

A compact structured illumination chip based on integrated optics is proposed and fabricated on a silicon-on- insulator platform. Based on the simulation of Caussian beam interference, we adopt a chirped diffraction grating to achieve a specific interference pattern. The experimental results match well with the simulations. The portability and flexibility of the structured illumination chip can be increased greatly through horizontal encapsulation. High levels of integration, compared with the conventional structured illumination approach, make this chip very compact, with a footprint of only around 1 mm2. The chip has no optical lenses and can be easily combined with a microfluidic system. These properties would make the chip very suitable for portable 3D scanner and compact super-resolution microscopy applications.