The G-SGD algorithm significantly outperforms the conventional SGD algorithm in ReLU neural networks by adopting the basis path set.However,how the inner mechanism of basis paths works remains mysterious,and the G-SGD...The G-SGD algorithm significantly outperforms the conventional SGD algorithm in ReLU neural networks by adopting the basis path set.However,how the inner mechanism of basis paths works remains mysterious,and the G-SGD algorithm that helps to find a basis path set is heuristic.This paper employs graph theory to investigate structure properties of basis paths in a more general and complicated neural network with unbalanced layers and edge-skipping.The hierarchical Algorithm HBPS is proposed to find a basis path set,by decomposing the complicated network into several independent and parallel substructures.The paper theoretically extends the study of basis paths and provides one methodology to find the basis path set in a more general neural network.展开更多
The newly appeared G-SGD algorithm can only heuristically find the basis path set in a simple neural network,so its generalization to a more practical network is hindered.From the perspective of graph theory,the Basis...The newly appeared G-SGD algorithm can only heuristically find the basis path set in a simple neural network,so its generalization to a more practical network is hindered.From the perspective of graph theory,the BasisP athSetSearching problem is formulated to find the basis path set in a complicated fully connected neural network.This paper proposes algorithm DEAH to hierarchically solve the BasisP athSetSearching problem by eliminating the path dependencies.For this purpose,the authors discover the underlying cause of the path dependency between two independent substructures.The path subdivision chain is proposed to effectively eliminate the path dependency,both inside the chain and between chains.The theoretical proofs and the analysis of time complexity are presented for Algorithm DEAH.This paper therefore provides one methodology to find the basis path set in a general and practical neural network.展开更多
The Sixth Beijing Forum on Human Rights is opened today, which is co-spon- sored by the China Society for Human Rights Studies and the China Foundation for Human Rights Development. First, on behalf of the China Found...The Sixth Beijing Forum on Human Rights is opened today, which is co-spon- sored by the China Society for Human Rights Studies and the China Foundation for Human Rights Development. First, on behalf of the China Foundation for Human Rights Development, I would like to express warm congratulations to the opening of the Forum, and heartfelt thanks tothe friends who have always cared for and supported for the development and progress of human rights cause of China.展开更多
基金supported by the National Nature Science Foundation of China under Grant No.11601471Project for Innovation Team(Cultivation)of Yunnan Province under Grant No.202005AE160006Key Project of Yunnan Provincial Science and Technology Department and Yunnan University under Grant No.2018FY001014。
文摘The G-SGD algorithm significantly outperforms the conventional SGD algorithm in ReLU neural networks by adopting the basis path set.However,how the inner mechanism of basis paths works remains mysterious,and the G-SGD algorithm that helps to find a basis path set is heuristic.This paper employs graph theory to investigate structure properties of basis paths in a more general and complicated neural network with unbalanced layers and edge-skipping.The hierarchical Algorithm HBPS is proposed to find a basis path set,by decomposing the complicated network into several independent and parallel substructures.The paper theoretically extends the study of basis paths and provides one methodology to find the basis path set in a more general neural network.
基金supported by Project for Innovation Team(Cultivation)of Yunnan Province under Grant No.202005AE160006Key Project of Yunnan Provincial Science and Technology Department and Yunnan University under Grant No.2018FY001014。
文摘The newly appeared G-SGD algorithm can only heuristically find the basis path set in a simple neural network,so its generalization to a more practical network is hindered.From the perspective of graph theory,the BasisP athSetSearching problem is formulated to find the basis path set in a complicated fully connected neural network.This paper proposes algorithm DEAH to hierarchically solve the BasisP athSetSearching problem by eliminating the path dependencies.For this purpose,the authors discover the underlying cause of the path dependency between two independent substructures.The path subdivision chain is proposed to effectively eliminate the path dependency,both inside the chain and between chains.The theoretical proofs and the analysis of time complexity are presented for Algorithm DEAH.This paper therefore provides one methodology to find the basis path set in a general and practical neural network.
文摘The Sixth Beijing Forum on Human Rights is opened today, which is co-spon- sored by the China Society for Human Rights Studies and the China Foundation for Human Rights Development. First, on behalf of the China Foundation for Human Rights Development, I would like to express warm congratulations to the opening of the Forum, and heartfelt thanks tothe friends who have always cared for and supported for the development and progress of human rights cause of China.
