Accelerating local SGD for non-IID data using variance reduction

Distributed stochastic gradient descent and its variants have been widely adopted in the training of machine learning models,which apply multiple workers in parallel.Among them,local-based algorithms,including Local SGD and FedAvg,have gained much attention due to their superior properties,such as low communication cost and privacypreserving.Nevertheless,when the data distribution on workers is non-identical,local-based algorithms would encounter a significant degradation in the convergence rate.In this paper,we propose Variance Reduced Local SGD(VRL-SGD)to deal with the heterogeneous data.Without extra communication cost,VRL-SGD can reduce the gradient variance among workers caused by the heterogeneous data,and thus it prevents local-based algorithms from slow convergence rate.Moreover,we present VRL-SGD-W with an effectivewarm-up mechanism for the scenarios,where the data among workers are quite diverse.Benefiting from eliminating the impact of such heterogeneous data,we theoretically prove that VRL-SGD achieves a linear iteration speedup with lower communication complexity even if workers access non-identical datasets.We conduct experiments on three machine learning tasks.The experimental results demonstrate that VRL-SGD performs impressively better than Local SGD for the heterogeneous data and VRL-SGD-W is much robust under high data variance among workers.

Highly sensitive integrated flexible tactile sensors with piezoresistive Ge2Sb2Te5 thin films

Flexible tactile sensor has been extensively investigated as a key component for emerging electronics applications such as robotics,wearable devices,computer hardware,and security systems.Tactile sensors based on various one-dimensional materials have been widely explored.However,precise control of the direction and distribution of these nanomaterials remains a great challenge,and it has been difficult to scale down the device.Here,we introduce highly sensitive integrated flexible tactile sensors based on uniform phase-change Ge_(2)Sb_(2)Te_(5)(GST)thin films that can scale device size down,at least,to micrometer range.Significant piezoresistive effect has been observed in GST-based sensors,showing a giant gauge factor of 338.A proof of concept 5×5 sensor array functioning as a touch panel has been demonstrated.Also,the flexible GST tactile sensor has been utilized for monitoring of radial artery pulse.In addition to the well-known tunable electrical and optical properties,the piezoresistive GST films provide a versatile platform for the integration of sensing,recording,and displaying functions.