SSL Depth: self-supervised learning enables 16× speedup in confocal microscopy-based 3D surface imaging [Invited]

In scientific and industrial research, three-dimensional (3D) imaging, or depth measurement, is a critical tool that provides detailed insight into surface properties. Confocal microscopy, known for its precision in surface measurements, plays a key role in this field. However, 3D imaging based on confocal microscopy is often challenged by significant data requirements and slow measurement speeds. In this paper, we present a novel self-supervised learning algorithm called SSL Depth that overcomes these challenges. Specifically, our method exploits the feature learning capabilities of neural networks while avoiding the need for labeled data sets typically associated with supervised learning approaches. Through practical demonstrations on a commercially available confocal microscope, we find that our method not only maintains higher quality, but also significantly reduces the frequency of the z-axis sampling required for 3D imaging. This reduction results in a remarkable 16×measurement speed, with the potential for further acceleration in the future. Our methodological advance enables highly efficient and accurate 3D surface reconstructions, thereby expanding the potential applications of confocal microscopy in various scientific and industrial fields.

LABEL-FREE DETECTION OF PROTEIN MICROARRAY WITH HIGH THROUGHPUT SURFACE PLASMON RESONANCE IMAGING(SPRI)

A surface plasmon resonance imaging(SPRI)system was developed for the discrimination of proteins on a gold surface.As a label-free and high-throughput technique,SPRI enables simultaneously monitoring of the biomolecular interactions at low concentrations.We used SPRI as a label-free and parallel method to detect different proteins based on protein microarray.Bovine Serum Albumin(BSA),Casein and Immunoglobulin G(IgG)were immobilized onto the Au surface of a gold-coated glass chip as spots forming a 6×6 matrix.These proteins can be discriminated directly by changing the incident angle of light.Excellent reproducibility for label-free detection of protein molecules was achieved.This SPRI platform represents a simple and robust method for performing high-sensitivity detection of protein microarray.

PDGI-BASED REGULAR SWEPT SURFACE EXTRACTION FROM POINT CLOUD

A principal direction Gaussian image （PDGI）-based algorithm is proposed to extract the regular swept surface from point cloud. Firstly, the PDGI of the regular swept surface is constructed from point cloud, then the bounding box of the Gaussian sphere is uniformly partitioned into a number of small cubes （3D grids） and the PDGI points on the Gaussian sphere are associated with the corresponding 3D grids. Secondly, cluster analysis technique is used to sort out a group of 3D grids containing more PDGI points among the 3D grids. By the connected-region growing algorithm, the congregation point or the great circle is detected from the 3D grids. Thus the translational direction is determined by the congregation point and the direction of the rotational axis is determined by the great circle. In addition, the positional point of the rotational axis is obtained by the intersection of all the projected normal lines of the rotational surface on the plane being perpendicular to the estimated direction of the rotational axis. Finally, a pattem search method is applied to optimize the translational direction and the rotational axis. Some experiments are used to illustrate the feasibility of the above algorithm.

Inversion of 3D velocity and anisotropy of near surfaces based on an azimuth-dependent dispersion curve inversion method

In petroleum seismic exploration,dense seismic ray coverage is often guaranteed through dense seismic sources and geophones.Dense ray coverage facilitates the high-resolution 3D velocity structure imaging of near surfaces using surface waves.In this study,the 3D velocity and anisotropy structure of a petroleum exploration area are obtained using the azimuth-dependent dispersion curve inversion(ADDCI)method.Imaging results show that low-velocity zones correspond to a river channel.The fast propagation direction(FPD)of S-waves along this channel is basically consistent with the direction of the channel.The eastern part of the study area has a surface sediment layer with a thickness of less than 20 m,which corresponds to the sand and gravel deposits formed by the river alluvial deposition near the surface.In addition,a relatively thick sedimentary layer is formed on the southern side of the study area.The anisotropy shows that the FPD is positively correlated with the direction of alluvial fl ow and that the magnitude of anisotropy in the deep part is greater than that in the shallow part.Inversion results are basically consistent with the geological data and suggest that the obtained model can truly refl ect the 3D velocity structure and anisotropy of the near-surface area.This study shows that the ADDCI method can maximize the high-energy surface waves in exploration data to obtain near-surface velocity structures,which provide a highly accurate model for near-surface static correction.

Breast Asymmetry Evaluation Using Objective Measures after Breast Cancer Surgery

Although a positive cosmetic outcome is an important goal of breast cancer reconstruction, the objective analysis of breast aesthetics has yet to reach a gold standard or unified method to quantify breast symmetry. Several scoring systems, both subjective and objective, have been developed over the years to ensure the desirable outcome in breast symmetry, but these methods have yet to reach the unanimous acceptance in terms of accuracy, value and ease of use to be implemented in the clinical setting. By assessing several existing symmetry scores, most of which are based on 2D imaging, along with our own set of symmetry parameters applied to 3D patient images, the goal of this study is to determine if there is an advantage of 3D imaging in formulating an accurate objective breast aesthetic score over the existing objective scores. A reliable breast aesthetic score would improve the decision-making in surgery as well as improve patient satisfaction. Additionally, knowing the quantity and degree of breast asymmetry objectively will improve outcome and reduce revision rates, minimizing patient suffering and improving the overall quality of patient life and body image.

Elements of automated survey of pavements and a 3D methodology

Sound transportation infrastructure is critical for economic development and sustainability. Pavement condition is a primary concern among agencies of the roadway infrastructure. Automation has become possible in recent years on collecting data and producing results for certain aspects of pavement performance, while challenges remain in several other categories, such as automated cracking survey. This paper reviews the technological advances on automated survey of pavements, and discusses the most recent breakthroughs by the team led by the author in using 3D laser imaging for capturing 1 mm surface images of pavements.

Short-working-distance optical imaging system and method for surface detection of underwater structures

In the surface imaging of underwater structures, long working distance will reduce image quality due to the turbidity of water. To acquire high definition and large field of view(FOV) images for surface detection, a short-working-distance underwater imaging system is proposed based on camera array. A multi-view calibration and rectification method is developed. A look-up table(LUT) method and a multi-resolution spline(MRS) method are applied to stitch array images real-time and seamlessly.Experiments both in the air and in the water are conducted. Strength and weakness of the LUT and MRS methods are discussed.Based on the results, the effectiveness in surface detection of underwater structures is verified.

Experimental investigation of ghost imaging of reflective objects with different surface roughness

We present an experimental demonstration of ghost imaging of reflective objects with different surface roughness.The influence of the surface roughness, the transverse size of the test detector, and the reflective angle on the signal-to-noise ratio(SNR) is analyzed by measuring the second-order correlation of the light field based on classical statistical optics. It is shown that the SNR decreases with an increment of the surface roughness and the detector's transverse size or a decrease of the reflective angle. Additionally, the comparative studies between the rough object and the smooth one under the same conditions are also discussed.

The Partition of Unity Method for High-Order Finite Volume Schemes Using Radial Basis Functions Reconstruction

This paper introduces the use of partition of unity method for the development of a high order finite volume discretization scheme on unstructured grids for solving diffusion models based on partial differential equations.The unknown function and its gradient can be accurately reconstructed using high order optimal recovery based on radial basis functions.The methodology proposed is applied to the noise removal problem in functional surfaces and images.Numerical results demonstrate the effectiveness of the new numerical approach and provide experimental order of convergence.

Optical coherence tomography for ocular surface and corneal diseases: a review

The advent of optical coherence tomography(OCT)imaging has changed the way ophthalmologists image the ocular surface and anterior segment of the eye.Its ability to obtain dynamic,high and ultra-high resolution,cross-sectional images of the ocular surface and anterior segment in a noninvasive and rapid manner allows for ease of use.In this review,we focus on the use of anterior segment OCT,which provides an“optical biopsy”or in vivo imaging of various ocular surface and corneal pathologies,allowing the clinician to diagnose diseases otherwise not visualized by traditional methods.The utility of anterior segment OCT for various anterior segment pathologies is reviewed.

Surface area-volume ratios in insects

Body mass, volume and surface area are important for many aspects of the physiology and performance of species. Whereas body mass scaling received a lot of attention in the literature, surface areas of animals have not been measured explicitly in this context. We quantified surface area-volume （SA/V） ratios for the first time using 3D surface models based on a structured light scanning method for 126 species of pollinating insects from 4 orders （Diptera, Hymenoptera, Lepidoptera, and Coleoptera）. Water loss of 67 species was measured gravimetrically at very dry conditions for 2 h at 15 and 30 ℃ to demonstrate the applicability of the new 3D surface measurements and relevance for predicting the performance of insects. Quantified SA/V ratios significantly explained the variation in water loss across species, both directly or after accounting for isometric scaling （residuals of the SA/V - mass2/3 relationship）. Small insects with a proportionally larger surface area had the highest water loss rates. Surface scans of insects to quantify allometric SA/Vratios thus provide a promising method to predict physiological responses, improving the potential of body mass isometry alone that assume geometric similarity.

Utility of high-resolution anterior segment optical coherence tomography in the diagnosis and management of sub-clinical ocular surface squamous neoplasia

Background:To evaluate the frequency and characteristics of sub-clinical ocular surface squamous neoplasia(OSSN)detected by high-resolution anterior segment tomography(HR-OCT)in patients with clinically unapparent disease following topical treatment.Methods:A retrospective chart review of patients with OSSN identified through a pharmacy database at the Bascom Palmer Eye Institute from January 2013 to December 2018 was conducted.Patients undergoing primary therapy with topical 5-fluorouracil 1%(5-FU)(4 times a day for 7 days with a 21-day break)or interferon-alpha-2b(IFN)(4 times a day)were reviewed.Patients were separated into two groups.Group 1 included individuals whose clinical resolution of OSSN aligned with complete resolution on HR-OCT.Group 2(sub-clinical OSSN group)included individuals with clinical OSSN resolution but with features of persistent disease on HR-OCT.Patients excluded included those treated at an outside institution and those who used topical therapy as a surgical adjunct.Results:A total of 95 patients(95 eyes)were reviewed.Sub-clinical OSSN was detected at a frequency of 17%in our study patients(n=16 patients,9 treated with 5-FU and 7 treated with IFN).In the 16 individuals,the mean time to clinical resolution was 3.6±1.0 cycles for 5-FU and 4.0±0.0 months for IFN.An additional 2.1±0.8 cycles for 5-FU and 1.2±0.4 months for IFN were needed to achieve HR-OCT resolution of OSSN.Recurrence in Group 1 was noted in 10 patients(12%)while no recurrences occurred in Group 2,the cohort with subclinical disease that received the extended medical therapy.The mean follow-up was 24.0±17.9 months.Conclusion:We found that at least 17%of individuals with apparent clinical resolution of OSSN have sub-clinical disease detected on HR-OCT.This information can be used to optimize treatment and extend therapy past the point of clinical resolution.

Statistical Estimation of High-Resolution Surface Air Temperature from MODIS over the Yangtze River Delta,China

High-resolution surface air temperature data are critical to regional climate modeling in terms of energy balance,urban climate change,and so on.This study demonstrates the feasibility of using Moderate Resolution Imaging Spectroradiometer（MODIS）land surface temperature（LST）to estimate air temperature at a high resolution over the Yangtze River Delta region,China.It is found that daytime LST is highly correlated with maximum air temperature,and the linear regression coefficients vary with the type of land surface.The air temperature at a resolution of 1 km is estimated from the MODIS LST with linear regression models.The estimated air temperature shows a clear spatial structure of urban heat islands.Spatial patterns of LST and air temperature differences are detected,indicating maximum differences over urban and forest regions during summer.Validations are performed with independent data samples,demonstrating that the mean absolute error of the estimated air temperature is approximately 2.5°C,and the uncertainty is about 3.1°C,if using all valid LST data.The error is reduced by 0.4°C（15%）if using best-quality LST with errors of less than 1 K.The estimated high-resolution air temperature data have great potential to be used in validating high-resolution climate models and other regional applications.

Background-free three-dimensional selective imaging of anisotropic plasmonic nanoparticles

There is an increasing demand for advanced optical imaging techniques that can detect and resolve nanosize objects at a spatial resolution below the optical diffraction limit, especially in three-dimensional （3D） cellular environments. In this study, using a polarization-activated localization scheme based on the orientation-dependent properties of anisotropic plasmonic metal nanoparticles （MNPs）, ＂photoswitchable＂ imaging of single gold nanorods （AuNRs） was accomplished not only in two dimensions but also in three dimensions. Moreover, the Rayleigh scattering background arising from the congested subcellular structures was efficiently suppressed. Thus, we obtained the 3D distributions of both the position and the orientation of the AuNRs inside the cells and investigated their intemalization kinetics. To our knowledge, this is the first demonstration of the confocal-like 3D imaging of non-fluorescence nanoparticles with a high resolution and almost zero background. This technique is easy to implement and should greatly facilitate MNP studies and applications in biomedicine and biology.

In-situ soil texture classification and physical clay content measurement based on multi-source information fusion

Soil texture is one of the most important soil characteristics that affect soil properties.Rapid acquisition of soil texture information is of great significance for accurate farmland management.Traditional soil texture analysis methods are relatively complicated and cannot meet the requirements of temporal and spatial resolution.This research introduced a self-developed vehicle-mounted in-situ soil texture detection system,which can predict the type of soil texture and the particle composition of the texture,and obtain real-time data during the measurement process without preprocessing the soil samples.The detection system is mainly composed of a conductivity measuring device,a camera,an auxiliary mechanical structure,and a control system.The soil electrical conductivity(ECa)and the texture features extracted from the surface image were input into the embedded model to realize real-time texture analysis.In order to find the best model suitable for the detection system,measurements were carried out in three test fields in Northeast and North China to compare the performance of different models applied to the detection system.The results showed that for soil texture classification,ExtraTrees performed best,with Precision,Recall,and F1 all being 0.82.For particle content of soil texture prediction,the R2 of ExtraTrees was 0.77,and RMSE and MAPE were 74.72 and 39.58.It was observed that ECa,Moment of inertia,and Entropy had larger weights in the drawn model influence weight map,and they are the main contributors to predicting soil texture.These results showed the potential of the vehicle-mounted in-situ soil texture detection system,which can provide a basis for fast,cost-effective,and efficient soil texture analysis.

The use of high resolution anterior segment optical coherence tomography for the characterization of conjunctival lymphoma, conjunctival amyloidosis and benign reactive lymphoid hyperplasia

Background:Conjunctival lymphoma,conjunctival amyloidosis and benign reactive lymphoid hyperplasia(BRLH)are conditions that often have a similar appearance on the ocular surface.The use of high resolution anterior segment optical coherence tomography(HR-OCT)enables clinicians to evaluate distinctive differences in tissue morphology and cellular patterns in various ocular surface conditions.In this study,we characterize the morphological differences seen in conjunctival lymphoma,conjunctival amyloidosis and BRLH on HR-OCT imaging.Methods:A retrospective chart review was performed of patients with biopsy proven conjunctival lymphoma,conjunctival amyloidosis and BRLH between 2012 and 2019 at the Bascom Palmer Eye Institute.Patients were excluded if HR-OCT imaging was not performed on initial presentation.Results:Thirty-four total eyes of 27 patients were identified.Twenty eyes had conjunctival lymphoma(16 patients),8 eyes had conjunctival amyloidosis(6 patients)and 6 eyes had BRLH(5 patients).All conditions appeared clinically as pink,red or yellow subepithelial lesions but had different features on HR-OCT.In lymphoma,HR-OCT images typically showed homogenous,dark subepithelial lesions with smooth borders,containing monomorphic dot-like infiltrates.HR-OCT images of amyloidosis typically showed heterogeneous,dark lesions with irregular borders,often containing hyperreflective linear infiltrates.HR-OCT images of BRLH showed variable infiltration of the subepithelial tissue,at times with homogenous lesions containing dot-like infiltrates like lymphoma and other times with more hyperreflective,subepithelial tissue.Flow cytometry and gene rearrangement was needed for final differentiation between BRLH and lymphoma lesions.Conclusions:Distinctive features on HR-OCT of conjunctival lymphoma,conjunctival amyloidosis and BRLH can help characterize these lesions beyond what is apparent with the clinical examination.Future studies can further validate this technology’s use with more subtle and challenging lesions.

An evaluation of digital elevation modeling in GIS and Cartography

This study considers seven commonly used surface fitting methods within Golden Software and ArcGIS^(TM) environments.Using grid sizes of 68 rows by 100 columns(6800 grids)and 680 rows by 1000 columns(680,000 grids)and 294,208 elevation points covering the entire landmass of Nigeria,the study evaluates the performance of these methods in terms of execution time and faithfulness in the representation of the spatial elements.Results show marked differences in time taken to execute the fitting and that Inverse Distance,the Natural Neighbor,the Nearest Neighbor,and Triangulation with Linear Interpolation seem to give the highest level of correspondence or faithfulness.

Sulfur speciation defined subcellular localization of coumarin derivatives:Correlation of structural relationship to biological behaviors

Targeted-delivery is of great importance to molecular probes and drugs for cell biology study. Herein we reported 11 sulfur-containing coumarins as cell imaging probes. Different sulfur speciation of the 4 representative coumarins SC1-SC4 renders them significantly different subcellular localizations and cellular uptake pathways： SC1 containing thioether group located in lysosomes, while sulfoxide and sulfone compounds SC2 and SC3 distributed in the whole cell. Furthermore, the cationic sulfonium containing compound SC4 was internalized by clathrin-mediated endocytosis and localized at mitochondria. By analyzing the molecular parameters of all 11 coumarins, we found that different sulfur speciation affected their lipophilicity and electrostatic surface potential. These two key factors play roles in altering biological behaviors of the coumarins. The results revealed the importance of sulfur speciation on the physicochemical properties and thus subcellular localization of bioprobes. This is useful for designing new functional bioprobes.