Joint specular highlight detection and removal in single images via Unet-Transformer

Specular highlight detection and removal is a fundamental problem in computer vision and image processing.In this paper,we present an efficient endto-end deep learning model for automatically detecting and removing specular highlights in a single image.In particular,an encoder–decoder network is utilized to detect specular highlights,and then a novel Unet-Transformer network performs highlight removal;we append transformer modules instead of feature maps in the Unet architecture.We also introduce a highlight detection module as a mask to guide the removal task.Thus,these two networks can be jointly trained in an effective manner.Thanks to the hierarchical and global properties of the transformer mechanism,our framework is able to establish relationships between continuous self-attention layers,making it possible to directly model the mapping between the diffuse area and the specular highlight area,and reduce indeterminacy within areas containing strong specular highlight reflection.Experiments on public benchmark and real-world images demonstrate that our approach outperforms state-of-the-art methods for both highlight detection and removal tasks.

Design of a new kind of chemical experiment container with virtual reality fusion

Background At present,the teaching of experiments in primary and secondary schools is affected by cost and security factors.Existing research on virtual experiment platforms has alleviated these problems.However,the lack of real experimental equipment and use of a single channel to understand user intentions weaken these platforms operationally and degrade the naturalness of interactions.Methods To solve these problems,we propose an intelligent experimental container structure and a situational awareness algorithm,both of which are verified and applied to a chemical experiment involving virtual-real fusion.First,the acquired images are denoised in the visual channel using the maximum diffuse reflection chroma to remove overexposure.Second,container situational awareness is realized by segmenting the image liquid level and establishing a relation-fitting model.Then,strategies for constructing complete behaviors and making priority comparisons among behaviors are adopted for information complementarity and independence,respectively.A multichannel intentional understanding model and an inter-active paradigm that integrates vision,hearing,and touch are proposed.Results The experimental results show that the accuracy of the intelligent container situation awareness proposed in this paper reaches 99%,and the accuracy of the proposed intention understanding algorithm reaches 94.7%.The test shows that the intelligent experimental system based on the new interaction paradigm also has better performance and a more realistic sense of operation experience in terms of experimental efficiency.Conclusion The results indicate that the proposed experimental container and algorithm can achieve a natural level of human-computer interaction in a virtual chemical experiment platform,enhance the user′s sense of operation,and achieve high levels of user satisfaction.