Low rank optimization for efficient deep learning:making a balance between compact architecture and fast training

Deep neural networks(DNNs)have achieved great success in many data processing applications.However,high computational complexity and storage cost make deep learning difficult to be used on resource-constrained devices,and it is not environmental-friendly with much power cost.In this paper,we focus on low-rank optimization for efficient deep learning techniques.In the space domain,DNNs are compressed by low rank approximation of the network parameters,which directly reduces the storage requirement with a smaller number of network parameters.In the time domain,the network parameters can be trained in a few subspaces,which enables efficient training for fast convergence.The model compression in the spatial domain is summarized into three categories as pre-train,pre-set,and compression-aware methods,respectively.With a series of integrable techniques discussed,such as sparse pruning,quantization,and entropy coding,we can ensemble them in an integration framework with lower computational complexity and storage.In addition to summary of recent technical advances,we have two findings for motivating future works.One is that the effective rank,derived from the Shannon entropy of the normalized singular values,outperforms other conventional sparse measures such as the?_1 norm for network compression.The other is a spatial and temporal balance for tensorized neural networks.For accelerating the training of tensorized neural networks,it is crucial to leverage redundancy for both model compression and subspace training.

Evaluation of Chinese Learning Efficiency of International Students in China:Comparing by Dimension

This paper introduces the Value Engineering method to calculate the value coefficient of Chinese learning efficiency for 377 international students by dimension.Results suggest that attention should be paid to male,European and American,rural,introverted,and“work or education needs”international students;Give full play to the driving role of the reference-type,explore the space for improving the learning efficiency of the improvement-type and attention-type,and pay attention to the problems of problem-type international students.

Evaluation of Chinese Learning Efficiency of International Students in China

This paper investigates the related factors of Chinese learning engagement and performance of 377 students from three universities in Guangzhou of China,employs the value engineering method,and calculates the value coefficient of learning efficiency.From the value coefficient of Chinese language learning,it proves that over 60%of international students studying in China have a higher efficiency in learning Chinese.

On Improving Students' Learning Efficiency in English Teaching

In our country,traditional English teaching attaches excessive importance to the explanation of vocabulary and grammar,causing students' disability in language practice.This paper provides some strategies to improve students' learning efficiency.

Near is more:learning efficiency in research and development innovation among interlocking firms

Research question/issue:This study examines whether geographic proximity produces a proximity preference as interlocking firms observe each other and learn innovative behaviors through information transmission among interlocking directors.Research findings/insights:We study the performance of A-share-listed companies in China from 2007 to 2017 on the basis of resource dependence theory,agglomeration effect theory,and Porter’s competitive theory.When target firms learn about research and development–related innovation behaviors from interlocking firms closer to them,they experience more efficient learning effects and have improved convergent traits.Moreover,this proximity advantage increases the willingness of the target firm to communicate with and learn from interlocking firms closer to them.Highly developed areas and research and development–intensive industries positively affect the learning efficiency of interlocking firms.Theoretical/academic implications:Our conclusion is consistent with resource dependence theory;target firms in highly developed areas are more willing to imitate and study nearby interlocking firms to maintain their peer relations,innovation potential,and competitiveness.Our conclusion is also consistent with competition theory,which states that the exchange of information between target firms in highly research and development–intensive industries and distant interlocking firms increases innovation differentiation,innovation potential,and competitiveness,even when such exchange has a high cost.Practitioner/policy implications:The results support resource dependence theory and peers’effects.The information obtained by interlocking directorates through external social relations guides firm decision-making,and closer distances reveal more obvious effects.

Joint User Selection and Resource Allocation for Fast Federated Edge Learning

By periodically aggregating local learning updates from edge users, federated edge learning (FEEL) is envisioned as a promising means to reap the benefit of local rich da?ta and protect users'privacy. However, the scarce wireless communication resource greatly limits the number of participated users and is regarded as the main bottleneck which hin?ders the development of FEEL. To tackle this issue, we propose a user selection policy based on data importance for FEEL system. In order to quantify the data importance of each user, we first analyze the relationship between the loss decay and the squared norm of gradi?ent. Then, we formulate a combinatorial optimization problem to maximize the learning effi?ciency by jointly considering user selection and communication resource allocation. By problem transformation and relaxation, the optimal user selection policy and resource alloca?tion are derived, and a polynomial-time optimal algorithm is developed. Finally, we deploy two commonly used deep neural network (DNN) models for simulation. The results validate that our proposed algorithm has strong generalization ability and can attain higher learning efficiency compared with other traditional algorithms.

An Overview of Autonomous Learning and Reflections on EFL Teaching

This paper mainly focuses on the development of autonomous learning worldwide and its reflections on English teachingin China.Autonomous learning,which is beneficial to one’s lifelong journey,is the ability to take responsibility for one’s learning.The final goal of education is to cultivate students with autonomous learning skills.As for EFL teachers,three steps are as followed.Firstly,EFL teachers need to be trained to learn autonomously.Secondly,EFL teachers should construct some online autonomouslearning platforms.Thirdly,EFL teachers need to motivate learners to listen and watch more about English to arouse students’ in-terests in the target language.

Social network learning efficiency in the principal-agent relationship

Under the bounded rationality assumption,a principal rarely provides an optimal contract to an agent.Learning from others is one way to improve such a contract.This paper studies the efficiency of social network learning(SNL)in the principal–agent framework.We first introduce the Cobb-Douglas production function into the classic Holmstrom and Milgrom(1987)model with a constant relative risk-averse agent and work out the theoretically optimal contract.Algorithms are then designed to model the SNL process based on profit gaps between contracts in a network of principals.Considering the uncertainty of the agent's labor output,we find that the principals can reach a consensus that tends to result in overcompensation compared to the optimal contract.Then,this study examines how network attributes and model parameters impact learning efficiency and posits several summative hypotheses.The simulation results validate these hypotheses,and we discuss the relevant economic implications of the observed changes in SNL efficiency.

Optimal control of end-port glass tank furnace regenerator temperature based on artificial neural network

In the paper, an artificial neural network (ANN) method is put forward to optimize melting temperature control, which reveals the nonlinear relationships of tank melting temperature disturbances with secondary wind flow and fuel pressure, implements dynamic feed-forward complementation and dynamic correctional ratio between air and fuel in the main control system. The application to Anhui Fuyang Glass Factory improved the control character of the melting temperature greatly.

AUTONOMOUS LEARNING STRATEGIES AND EFFICIENCY & APTITUDE OF ENGLISH LEARNERS

Based on theories and empirical study of autonomous learning, this paper analyzed the correlations between English learners' autonomous learning strategies and learning efficiency. The research showed that meta-cognitive strategies, cognitive strategies, affective strategies, communicative strategies were closely related to the English learning efficiency and aptitude. It also revealed that the traditional English teaching mode strongly influenced the English learners' learning strategies and that meta-cognitive strategies were more frequently used than cognitive strategies while affective and communicative strategies were employed inadequately. Therefore, the development of the English learners' awareness of learning strategies, especially the awareness of meta-cognitive strategies, could help them learn English more effectively.

Efficient Pose: Efficient human pose estimation with neural architecture search

Human pose estimation from image and video is a key task in many multimedia applications.Previous methods achieve great performance but rarely take efficiency into consideration,which makes it difficult to implement the networks on lightweight devices.Nowadays,real-time multimedia applications call for more efficient models for better interaction.Moreover,most deep neural networks for pose estimation directly reuse networks designed for image classification as the backbone,which are not optimized for the pose estimation task.In this paper,we propose an efficient framework for human pose estimation with two parts,an efficient backbone and an efficient head.By implementing a differentiable neural architecture search method,we customize the backbone network design for pose estimation,and reduce computational cost with negligible accuracy degradation.For the efficient head,we slim the transposed convolutions and propose a spatial information correction module to promote the performance of the final prediction.In experiments,we evaluate our networks on the MPII and COCO datasets.Our smallest model requires only0.65 GFLOPs with 88.1%PCKh@0.5 on MPII and our large model needs only 2 GFLOPs while its accuracy is competitive with the state-of-the-art large model,HRNet,which takes 9.5 GFLOPs.