Edge-Federated Self-Supervised Communication Optimization Framework Based on Sparsification and Quantization Compression

The federated self-supervised framework is a distributed machine learning method that combines federated learning and self-supervised learning, which can effectively solve the problem of traditional federated learning being difficult to process large-scale unlabeled data. The existing federated self-supervision framework has problems with low communication efficiency and high communication delay between clients and central servers. Therefore, we added edge servers to the federated self-supervision framework to reduce the pressure on the central server caused by frequent communication between both ends. A communication compression scheme using gradient quantization and sparsification was proposed to optimize the communication of the entire framework, and the algorithm of the sparse communication compression module was improved. Experiments have proved that the learning rate changes of the improved sparse communication compression module are smoother and more stable. Our communication compression scheme effectively reduced the overall communication overhead.

Sparsification: In Theory and Practice

The Kadison-Singer problem has variants in different branches of the sciences and one of these variants was proved in 2013. Based on the idea of “sparsification” and with its origins in quantum physics, at the sixtieth anniversary of the problem, we revisit the problem in its original formulation and also explore its transition to a result with wide ranging applications. We also describe how the notion of “sparsification” transcended various fields and how this notion led to resolution of the problem.

GRAPH SPARSIFICATION BY UNIVERSAL GREEDY ALGORITHMS

Graph sparsification is to approximate an arbitrary graph by a sparse graph and is useful in many applications,such as simplification of social networks,least squares problems,and numerical solution of symmetric positive definite linear systems.In this paper,inspired by the well-known sparse signal recovery algorithm called orthogonal matching pursuit(OMP),we introduce a deterministic,greedy edge selection algorithm,which is called the universal greedy approach(UGA)for the graph sparsification problem.For a general spectral sparsification problem,e.g.,the positive subset selection problem from a set of m vectors in R n,we propose a nonnegative UGA algorithm which needs O(mn^(2)+n^(3)/ϵ^(2))time to find a 1+ϵ/β/1-ϵ/β-spectral sparsifier with positive coefficients with sparsity at most[n/ϵ^(2)],where β is the ratio between the smallest length and largest length of the vectors.The convergence of the nonnegative UGA algorithm is established.For the graph sparsification problem,another UGA algorithm is proposed which can output a 1+O(ϵ)/1-O(ϵ)-spectral sparsifier with[n/ϵ^(2)]edges in O(m+n^(2)/ϵ^(2))time from a graph with m edges and n vertices under some mild assumptions.This is a linear time algorithm in terms of the number of edges that the community of graph sparsification is looking for.The best result in the literature to the knowledge of the authors is the existence of a deterministic algorithm which is almost linear,i.e.O(m^(1+o(1)))for some o(1)=O((log log(m))^(2/3)/log^(1/3)(m)).Finally,extensive experimental results,including applications to graph clustering and least squares regression,show the effectiveness of proposed approaches.

Wavelet Bases Made of Piecewise Polynomial Functions: Theory and Applications

We present wavelet bases made of piecewise (low degree) polynomial functions with an (arbitrary) assigned number of vanishing moments. We study some of the properties of these wavelet bases;in particular we consider their use in the approximation of functions and in numerical quadrature. We focus on two applications: integral kernel sparsification and digital image compression and reconstruction. In these application areas the use of these wavelet bases gives very satisfactory results.

l_(1)-based calibration of POD-Galerkin models of two-dimensional unsteady flows

This paper discusses a physics-informed methodology aimed at reconstructing efficiently the fluid state of a system.Herein,the generation of an accurate reduced order model of twodimensional unsteady flows from data leverages on sparsity-promoting statistical learning techniques.The cornerstone of the approach is l_(1) regularised regression,resulting in sparselyconnected models where only the important quadratic interactions between modes are retained.The original dynamical behaviour is reproduced at low computational costs,as few quadratic interactions need to be evaluated.The approach has two key features.First,interactions are selected systematically as a solution of a convex optimisation problem and no a priori assumptions on the physics of the flow are required.Second,the presence of a regularisation term improves the predictive performance of the original model,generally affected by noise and poor data quality.Test cases are for two-dimensional lid-driven cavity flows,at three values of the Reynolds number for which the motion is chaotic and energy interactions are scattered across the spectrum.It is found that:(A)the sparsification generates models maintaining the original accuracy level but with a lower number of active coefficients;this becomes more pronounced for increasing Reynolds numbers suggesting that extension of these techniques to real-life flow configurations is possible;(B)sparse models maintain a good temporal stability for predictions.The methodology is ready for more complex applications without modifications of the underlying theory,and the integration into a cyberphysical model is feasible.