Rademacher Complexity in Neyman-Pearson Classification

Neyman-Pearson（NP） criterion is one of the most important ways in hypothesis testing. It is also a criterion for classification. This paper addresses the problem of bounding the estimation error of NP classification, in terms of Rademacher averages. We investigate the behavior of the global and local Rademacher averages, and present new NP classification error bounds which are based on the localized averages, and indicate how the estimation error can be estimated without a priori knowledge of the class at hand.

Improve Robustness and Accuracy of Deep Neural Network with L_(2,∞) Normalization

In this paper,the L_(2,∞)normalization of the weight matrices is used to enhance the robustness and accuracy of the deep neural network(DNN)with Relu as activation functions.It is shown that the L_(2,∞)normalization leads to large dihedral angles between two adjacent faces of the DNN function graph and hence smoother DNN functions,which reduces over-fitting of the DNN.A global measure is proposed for the robustness of a classification DNN,which is the average radius of the maximal robust spheres with the training samples as centers.A lower bound for the robustness measure in terms of the L_(2,∞)norm is given.Finally,an upper bound for the Rademacher complexity of DNNs with L_(2,∞)normalization is given.An algorithm is given to train DNNs with the L_(2,∞)normalization and numerical experimental results are used to show that the L_(2,∞)normalization is effective in terms of improving the robustness and accuracy.

A Rate of Convergence of Physics Informed Neural Networks for the Linear Second Order Elliptic PDEs

In recent years,physical informed neural networks(PINNs)have been shown to be a powerful tool for solving PDEs empirically.However,numerical analysis of PINNs is still missing.In this paper,we prove the convergence rate to PINNs for the second order elliptic equations with Dirichlet boundary condition,by establishing the upper bounds on the number of training samples,depth and width of the deep neural networks to achieve desired accuracy.The error of PINNs is decomposed into approximation error and statistical error,where the approximation error is given in C2 norm with ReLU^(3)networks(deep network with activation function max{0,x^(3)})and the statistical error is estimated by Rademacher complexity.We derive the bound on the Rademacher complexity of the non-Lipschitz composition of gradient norm with ReLU^(3)network,which is of immense independent interest.