Long Short-Term Memory Recurrent Neural Network-Based Acoustic Model Using Connectionist Temporal Classification on a Large-Scale Training Corpus

A Long Short-Term Memory(LSTM) Recurrent Neural Network(RNN) has driven tremendous improvements on an acoustic model based on Gaussian Mixture Model(GMM). However, these models based on a hybrid method require a forced aligned Hidden Markov Model(HMM) state sequence obtained from the GMM-based acoustic model. Therefore, it requires a long computation time for training both the GMM-based acoustic model and a deep learning-based acoustic model. In order to solve this problem, an acoustic model using CTC algorithm is proposed. CTC algorithm does not require the GMM-based acoustic model because it does not use the forced aligned HMM state sequence. However, previous works on a LSTM RNN-based acoustic model using CTC used a small-scale training corpus. In this paper, the LSTM RNN-based acoustic model using CTC is trained on a large-scale training corpus and its performance is evaluated. The implemented acoustic model has a performance of 6.18% and 15.01% in terms of Word Error Rate(WER) for clean speech and noisy speech, respectively. This is similar to a performance of the acoustic model based on the hybrid method.

TP-MobNet: A Two-pass Mobile Network for Low-complexity Classification of Acoustic Scene

Acoustic scene classification(ASC)is a method of recognizing and classifying environments that employ acoustic signals.Various ASC approaches based on deep learning have been developed,with convolutional neural networks(CNNs)proving to be the most reliable and commonly utilized in ASC systems due to their suitability for constructing lightweight models.When using ASC systems in the real world,model complexity and device robustness are essential considerations.In this paper,we propose a two-pass mobile network for low-complexity classification of the acoustic scene,named TP-MobNet.With inverse residuals and linear bottlenecks,TPMobNet is based on MobileNetV2,and following mobile blocks,coordinate attention and two-pass fusion approaches are utilized.The log-range dependencies and precise position information in feature maps can be trained via coordinate attention.By capturing more diverse feature resolutions at the network’s end sides,two-pass fusions can also train generalization.Also,the model size is reduced by applying weight quantization to the trained model.By adding weight quantization to the trained model,the model size is also lowered.The TAU Urban Acoustic Scenes 2020 Mobile development set was used for all of the experiments.It has been confirmed that the proposed model,with a model size of 219.6 kB,achieves an accuracy of 73.94%.

Joint On-Demand Pruning and Online Distillation in Automatic Speech Recognition Language Model Optimization

Automatic speech recognition(ASR)systems have emerged as indispensable tools across a wide spectrum of applications,ranging from transcription services to voice-activated assistants.To enhance the performance of these systems,it is important to deploy efficient models capable of adapting to diverse deployment conditions.In recent years,on-demand pruning methods have obtained significant attention within the ASR domain due to their adaptability in various deployment scenarios.However,these methods often confront substantial trade-offs,particularly in terms of unstable accuracy when reducing the model size.To address challenges,this study introduces two crucial empirical findings.Firstly,it proposes the incorporation of an online distillation mechanism during on-demand pruning training,which holds the promise of maintaining more consistent accuracy levels.Secondly,it proposes the utilization of the Mogrifier long short-term memory(LSTM)language model(LM),an advanced iteration of the conventional LSTM LM,as an effective alternative for pruning targets within the ASR framework.Through rigorous experimentation on the ASR system,employing the Mogrifier LSTM LM and training it using the suggested joint on-demand pruning and online distillation method,this study provides compelling evidence.The results exhibit that the proposed methods significantly outperform a benchmark model trained solely with on-demand pruning methods.Impressively,the proposed strategic configuration successfully reduces the parameter count by approximately 39%,all the while minimizing trade-offs.

Subcarrier BD with Cooperative Communication for MIMO-NOMA System

With the rapid evolution of Internet of things(IoT),many edge devices require simultaneous connection in 5G communication era.To afford massive data of IoT devices,multiple input multiple output non-orthogonal multiple access(MIMO-NOMA)method has been considered as a promising technology.However,there are numerous drawbacks due to error propagation and inter-user interferences.Therefore,proposed scheme aims to improve the reliability of the MIMO-NOMA system with digital beamforming and intracluster cooperative multi point(CoMP)to efficiently support IoT system.In the conventional MIMO-NOMA system,user entities are grouped into clusters.Block diagonalization(BD)is applied to efficiently eliminate the inter-cluster interference of the MIMO-NOMA system.However,since the channel path of the data stream from a single antenna to a single cluster doesn’t hold other cluster’s data,the system can’t fully utilize the selective subcarrier channel states.It indicates that there can be better channel paths for a data stream at a certain subcarrier index.Therefore,proposed scheme allocates data streams to antennas adaptively considering selective channel states.Additionally,intra-cluster CoMP method is adjusted to enhance the reliability of the system in the clusters.The simulation results show that the proposed scheme improves BER and throughput performance compared to the conventional MIMO-NOMA system.

Language Model Using Differentiable Neural Computer Based on Forget Gate-Based Memory Deallocation

A differentiable neural computer(DNC)is analogous to the Von Neumann machine with a neural network controller that interacts with an external memory through an attention mechanism.Such DNC’s offer a generalized method for task-specific deep learning models and have demonstrated reliability with reasoning problems.In this study,we apply a DNC to a language model(LM)task.The LM task is one of the reasoning problems,because it can predict the next word using the previous word sequence.However,memory deallocation is a problem in DNCs as some information unrelated to the input sequence is not allocated and remains in the external memory,which degrades performance.Therefore,we propose a forget gatebased memory deallocation(FMD)method,which searches for the minimum value of elements in a forget gate-based retention vector.The forget gatebased retention vector indicates the retention degree of information stored in each external memory address.In experiments,we applied our proposed NTM architecture to LM tasks as a task-specific example and to rescoring for speech recognition as a general-purpose example.For LM tasks,we evaluated DNC using the Penn Treebank and enwik8 LM tasks.Although it does not yield SOTA results in LM tasks,the FMD method exhibits relatively improved performance compared with DNC in terms of bits-per-character.For the speech recognition rescoring tasks,FMD again showed a relative improvement using the LibriSpeech data in terms of word error rate.

Stretchable and colorless freestanding microwire arrays for transparent solar cells with flexibility

Transparent solar cells(TSCs)are emerging devices that combine the advantages of visible transparency and light-toelectricity conversion.Currently,existing TSCs are based predominantly on organics,dyes,and perovskites;however,the rigidity and color-tinted transparent nature of those devices strongly limit the utility of the resulting TSCs for realworld applications.Here,we demonstrate a flexible,color-neutral,and high-efficiency TSC based on a freestanding form of n-silicon microwires(SiMWs).Flat-tip SiMWs with controllable spacing are fabricated via deep-reactive ion etching and embedded in a freestanding transparent polymer matrix.The light transmittance can be tuned from ~10 to 55% by adjusting the spacing between the microwires.For TSCs,a heterojunction is formed with a p-type polymer in the top portion of the n-type flat-tip SiMWs.Ohmic contact with an indium-doped ZnO film occurs at the bottom,and the side surface has an Al2O3 passivation layer.Furthermore,slanted-tip SiMWs are developed by a novel solventassisted wet etching method to manipulate light absorption.Finite-difference time-domain simulation revealed that the reflected light from slanted-tip SiMWs helps light-matter interactions in adjacent microwires.The TSC based on the slanted-tip SiMWs demonstrates 8%efficiency at a visible transparency of 10% with flexibility.This efficiency is the highest among Si-based TSCs and comparable with that of state-of-the-art neutral-color TSCs based on organic–inorganic hybrid perovskite and organics.Moreover,unlike others,the stretchable and transparent platform in this study is promising for future TSCs.

Hybridization of different types of exceptional points

A large number of different types of second-order non-Hermitian degeneracies called exceptional points(EPs)were found in various physical systems depending on the mechanism of coupling between eigenstates.We show that these EPs can be hybridized to form higher-order EPs,which preserve the original properties of the initial EPs before hybridization.For a demonstration,we hybridize chiral and supermode second-order EPs,where the former and the latter are the results of intra-disk and inter-disk mode coupling in an optical system comprised of two Mie-scale microdisks and one Rayleigh-scale scatterer.The high sensitivity of the resulting third-order EP against external perturbations in our feasible system is emphasized.

Impact of non-Hermitian mode interaction on inter-cavity light transfer

Understanding inter-site mutual mode interaction in coupled physical systems is essential to comprehend large compound systems,as this local interaction determines the successive multiple inter-site energy transfer efficiencies.In the present study,we demonstrate that only the non-Hermitian coupling can correctly account for the light transfer between two coupled optical cavities.We also reveal that the non-Hermitian coupling effect becomes crucial as the system dimension decreases.Our results provide important insight for handling general-coupled devices in the subwavelength regime.