Multimodal Spatiotemporal Feature Map for Dynamic Gesture Recognition

Gesture recognition technology enables machines to read human gestures and has significant application prospects in the fields of human-computer interaction and sign language translation.Existing researches usually use convolutional neural networks to extract features directly from raw gesture data for gesture recognition,but the networks are affected by much interference information in the input data and thus fit to some unimportant features.In this paper,we proposed a novel method for encoding spatio-temporal information,which can enhance the key features required for gesture recognition,such as shape,structure,contour,position and hand motion of gestures,thereby improving the accuracy of gesture recognition.This encoding method can encode arbitrarily multiple frames of gesture data into a single frame of the spatio-temporal feature map and use the spatio-temporal feature map as the input to the neural network.This can guide the model to fit important features while avoiding the use of complex recurrent network structures to extract temporal features.In addition,we designed two sub-networks and trained the model using a sub-network pre-training strategy that trains the sub-networks first and then the entire network,so as to avoid the subnetworks focusing too much on the information of a single category feature and being overly influenced by each other’s features.Experimental results on two public gesture datasets show that the proposed spatio-temporal information encoding method achieves advanced accuracy.

Encoding Energy-Density as Geometry

Physicists possess an intuitive awareness of Euclidian space and time and Galilean transformation, and are then challenged with Minkowski space-time and Einstein’s curved space-time. Relativistic experiments support the “time-dilation” interpretation and others support “curved space-time” interpretation. In this, and related work, we investigate the key issues in terms of the intuitive space-time frame. In particular, we provide alternative approaches to explain “time dilation” and to explain the energy density for gravity systems. We approach the latter problem from an information perspective.

A Re-Interpretation of Quasi-Local Mass

Because the equivalence principle forbids local mass density, we cannot formulate general relativistic mass as an integral over mass density as in Newtonian gravity. This century-old problem was addressed forty years ago by Penrose, and many papers have since extended the concept. Currently there is no satisfactory physical understanding of the nature of quasi-local mass. In this paper I review the key issues, the current status, and propose an alternative interpretation of the problem of local mass and energy density for gravity systems from an information perspective.

Transforming information from spin encoding to route encoding

Spin （polarization） is widely used in free-space optics, while in photouic integrated circuits （PICs）, infor- mation is usually encoded in optical route. So a practical way to connect these two encoding methods is necessary for information communication. In this letter, an encoding convertor is designed to connect spin encoding and route encoding. Finite element method is used to calculate the conversion efficiency and extinction ratio of the encoding convertor and the theoretical analyses are also given. Our protocol shows a friendly way to convert optical spin information to route information, which will promote the compatibility of free-space optics and PICs.

Wave filtering and firing modes in a light-sensitive neural circuit

Inspired by the photoelectric effect,a phototube is incorporated into a simple neural circuit,and then the output voltage and dynamics become sensitive to external illumination within a specific frequency band.The firing modes are also dependent on the amplitude and frequency band in the illumination.In this paper,the signal outputs from a chaotic circuit are used as external optical signals,which are filtered and encoded by a phototube.Then,the functional neural circuit is excited to present a variety of firing modes and patterns.An exponential function of the filtering wave is proposed to discover the biophysical mechanism for frequency selection in the retina as most of wave bands of the external illumination are absorbed in the cathode material of the phototube while a specific band is effective in inducing a photocurrent for stimulating the visual neurons.Based on our light-sensitive neural circuit and model,external illumination is filtered and firing modes in the neuron are reproduced;furthermore,the mode transition induced by parameter shift is also investigated in detail.This result discovers the signal processing mechanism in the visual neurons and provides helpful guidance for designing artificial sensors for encoding optical signals and for repairing abnormalities in the retina of the visual system.