Dropout and other feature noising schemes have shown promise in controlling over-fitting by artificially corrupting the training data. Though extensive studies have been performed for generalized linear models, little...Dropout and other feature noising schemes have shown promise in controlling over-fitting by artificially corrupting the training data. Though extensive studies have been performed for generalized linear models, little has been done for support vector machines （SVMs）, one of the most successful approaches for supervised learning. This paper presents dropout training for both linear SVMs and the nonlinear extension with latent representation learning. For linear SVMs, to deal with the intractable expectation of the non-smooth hinge loss under corrupting distributions, we develop an iteratively re-weighted least square （IRLS） algorithm by exploring data augmentation techniques. Our algorithm iteratively minimizes the expectation of a re- weighted least square problem, where the re-weights are analytically updated. For nonlinear latent SVMs, we con- sider learning one layer of latent representations in SVMs and extend the data augmentation technique in conjunction with first-order Taylor-expansion to deal with the intractable expected hinge loss and the nonlinearity of latent representa- tions. Finally, we apply the similar data augmentation ideas to develop a new IRLS algorithm for the expected logistic loss under corrupting distributions, and we further develop a non-linear extension of logistic regression by incorporating one layer of latent representations. Our algorithms offer insights on the connection and difference between the hinge loss and logistic loss in dropout training. Empirical results on several real datasets demonstrate the effectiveness of dropout training on significantly boosting the classification accuracy of both linear and nonlinear SVMs.展开更多
Although advances in research into autonomous underwater vehicles(AUVs)have been made to extend their working depth and endurance,underwater experiments and missions remain to be restricted by the positioning performa...Although advances in research into autonomous underwater vehicles(AUVs)have been made to extend their working depth and endurance,underwater experiments and missions remain to be restricted by the positioning performance of AUVs.With the Global Navigation Satellite System(GNSS)precluded due to the rapid attenuation of radio signals in underwater environments,acoustic positioning methods serve as an effective substitution.A long-range continuous and precise positioning solution for AUVs in deep ocean is proposed in this study,relying on acoustic signals from beacons at the same depth and aided by onboard inertial sensors.A signal system is investigated to provide time of arrival(TOA)estimation in a resolution of milliseconds.Without pre-knowledge or local measurement of the accurate sound speed,an AUV is enabled to continuously locate its horizontal position based on rough ranges estimated by an iterative least square(ILS)based algorithm.For better accuracy and robustness,range deviations are compensated with a reference point of known position and outliers in the trajectory are eliminated by an implementation of the extended Kalman filter(EKF)coupled with the state-acceptance filter.The solution is evaluated in simulation experiments with environmental information measured on the spot,providing an average position error from ground truth below 10 m with a standard deviation below 5 m.展开更多
文摘Dropout and other feature noising schemes have shown promise in controlling over-fitting by artificially corrupting the training data. Though extensive studies have been performed for generalized linear models, little has been done for support vector machines （SVMs）, one of the most successful approaches for supervised learning. This paper presents dropout training for both linear SVMs and the nonlinear extension with latent representation learning. For linear SVMs, to deal with the intractable expectation of the non-smooth hinge loss under corrupting distributions, we develop an iteratively re-weighted least square （IRLS） algorithm by exploring data augmentation techniques. Our algorithm iteratively minimizes the expectation of a re- weighted least square problem, where the re-weights are analytically updated. For nonlinear latent SVMs, we con- sider learning one layer of latent representations in SVMs and extend the data augmentation technique in conjunction with first-order Taylor-expansion to deal with the intractable expected hinge loss and the nonlinearity of latent representa- tions. Finally, we apply the similar data augmentation ideas to develop a new IRLS algorithm for the expected logistic loss under corrupting distributions, and we further develop a non-linear extension of logistic regression by incorporating one layer of latent representations. Our algorithms offer insights on the connection and difference between the hinge loss and logistic loss in dropout training. Empirical results on several real datasets demonstrate the effectiveness of dropout training on significantly boosting the classification accuracy of both linear and nonlinear SVMs.
基金the Science and Technology Innovation Base Project of Shanghai Science and Technology Commission(No.19DZ2255200)the Shanghai Commercial Aircraft System Engineering Joint Research Fund。
文摘Although advances in research into autonomous underwater vehicles(AUVs)have been made to extend their working depth and endurance,underwater experiments and missions remain to be restricted by the positioning performance of AUVs.With the Global Navigation Satellite System(GNSS)precluded due to the rapid attenuation of radio signals in underwater environments,acoustic positioning methods serve as an effective substitution.A long-range continuous and precise positioning solution for AUVs in deep ocean is proposed in this study,relying on acoustic signals from beacons at the same depth and aided by onboard inertial sensors.A signal system is investigated to provide time of arrival(TOA)estimation in a resolution of milliseconds.Without pre-knowledge or local measurement of the accurate sound speed,an AUV is enabled to continuously locate its horizontal position based on rough ranges estimated by an iterative least square(ILS)based algorithm.For better accuracy and robustness,range deviations are compensated with a reference point of known position and outliers in the trajectory are eliminated by an implementation of the extended Kalman filter(EKF)coupled with the state-acceptance filter.The solution is evaluated in simulation experiments with environmental information measured on the spot,providing an average position error from ground truth below 10 m with a standard deviation below 5 m.
