The scientific community recognizes the seriousness of rockbursts and the need for effective mitigation measures.The literature reports various successful applications of machine learning(ML)models for rockburst asses...The scientific community recognizes the seriousness of rockbursts and the need for effective mitigation measures.The literature reports various successful applications of machine learning(ML)models for rockburst assessment;however,a significant question remains unanswered:How reliable are these models,and at what confidence level are classifications made?Typically,ML models output single rockburst grade even in the face of intricate and out-of-distribution samples,without any associated confidence value.Given the susceptibility of ML models to errors,it becomes imperative to quantify their uncertainty to prevent consequential failures.To address this issue,we propose a conformal prediction(CP)framework built on traditional ML models(extreme gradient boosting and random forest)to generate valid classifications of rockburst while producing a measure of confidence for its output.The proposed framework guarantees marginal coverage and,in most cases,conditional coverage on the test dataset.The CP was evaluated on a rockburst case in the Sanshandao Gold Mine in China,where it achieved high coverage and efficiency at applicable confidence levels.Significantly,the CP identified several“confident”classifications from the traditional ML model as unreliable,necessitating expert verification for informed decision-making.The proposed framework improves the reliability and accuracy of rockburst assessments,with the potential to bolster user confidence.展开更多
This paper advances new directions for cyber security using adversarial learning and conformal prediction in order to enhance network and computing services defenses against adaptive, malicious, persistent, and tactic...This paper advances new directions for cyber security using adversarial learning and conformal prediction in order to enhance network and computing services defenses against adaptive, malicious, persistent, and tactical offensive threats. Conformal prediction is the principled and unified adaptive and learning framework used to design, develop, and deploy a multi-faceted?self-managing defensive shield to detect, disrupt, and deny intrusive attacks, hostile and malicious behavior, and subterfuge. Conformal prediction leverages apparent relationships between immunity and intrusion detection using non-conformity measures characteristic of affinity, a typicality, and surprise, to recognize patterns and messages as friend or foe and to respond to them accordingly. The solutions proffered throughout are built around active learning, meta-reasoning, randomness, distributed semantics and stratification, and most important and above all around adaptive Oracles. The motivation for using conformal prediction and its immediate off-spring, those of semi-supervised learning and transduction, comes from them first and foremost supporting discriminative and non-parametric methods characteristic of principled demarcation using cohorts and sensitivity analysis to hedge on the prediction outcomes including negative selection, on one side, and providing credibility and confidence indices that assist meta-reasoning and information fusion.展开更多
The tropical Pacific has begun to experience a new type of El Nio, which has occurred particularly frequently during the last decade, referred to as the central Pacific(CP) El Nio. Various coupled models with differen...The tropical Pacific has begun to experience a new type of El Nio, which has occurred particularly frequently during the last decade, referred to as the central Pacific(CP) El Nio. Various coupled models with different degrees of complexity have been used to make real-time El Nio predictions, but high uncertainty still exists in their forecasts. It remains unknown as to how much of this uncertainty is specifically related to the new CP-type El Nio and how much is common to both this type and the conventional Eastern Pacific(EP)-type El Nio. In this study, the deterministic performance of an El Nio–Southern Oscillation(ENSO) ensemble prediction system is examined for the two types of El Nio. Ensemble hindcasts are run for the nine EP El Nio events and twelve CP El Nio events that have occurred since 1950. The results show that(1) the skill scores for the EP events are significantly better than those for the CP events, at all lead times;(2) the systematic forecast biases come mostly from the prediction of the CP events; and(3) the systematic error is characterized by an overly warm eastern Pacific during the spring season, indicating a stronger spring prediction barrier for the CP El Nio. Further improvements to coupled atmosphere–ocean models in terms of CP El Nio prediction should be recognized as a key and high-priority task for the climate prediction community.展开更多
Positive data are very common in many scientific fields and applications;for these data,it is known that estimation and inference based on relative error criterion are superior to that of absolute error criterion.In p...Positive data are very common in many scientific fields and applications;for these data,it is known that estimation and inference based on relative error criterion are superior to that of absolute error criterion.In prediction problems,conformal prediction provides a useful framework to construct flexible prediction intervals based on hypothesis testing,which has been actively studied in the past decade.In view of the advantages of the relative error criterion for regression problems with positive responses,in this paper,we combine the relative error criterion(REC)with conformal prediction to develop a novel REC-based predictive inference method to construct prediction intervals for the positive response.The proposed method satisfies the finite sample global coverage guarantee and to some extent achieves the local validity.We conduct extensive simulation studies and two real data analysis to demonstrate the competitiveness of the new proposed method.展开更多
文摘The scientific community recognizes the seriousness of rockbursts and the need for effective mitigation measures.The literature reports various successful applications of machine learning(ML)models for rockburst assessment;however,a significant question remains unanswered:How reliable are these models,and at what confidence level are classifications made?Typically,ML models output single rockburst grade even in the face of intricate and out-of-distribution samples,without any associated confidence value.Given the susceptibility of ML models to errors,it becomes imperative to quantify their uncertainty to prevent consequential failures.To address this issue,we propose a conformal prediction(CP)framework built on traditional ML models(extreme gradient boosting and random forest)to generate valid classifications of rockburst while producing a measure of confidence for its output.The proposed framework guarantees marginal coverage and,in most cases,conditional coverage on the test dataset.The CP was evaluated on a rockburst case in the Sanshandao Gold Mine in China,where it achieved high coverage and efficiency at applicable confidence levels.Significantly,the CP identified several“confident”classifications from the traditional ML model as unreliable,necessitating expert verification for informed decision-making.The proposed framework improves the reliability and accuracy of rockburst assessments,with the potential to bolster user confidence.
文摘This paper advances new directions for cyber security using adversarial learning and conformal prediction in order to enhance network and computing services defenses against adaptive, malicious, persistent, and tactical offensive threats. Conformal prediction is the principled and unified adaptive and learning framework used to design, develop, and deploy a multi-faceted?self-managing defensive shield to detect, disrupt, and deny intrusive attacks, hostile and malicious behavior, and subterfuge. Conformal prediction leverages apparent relationships between immunity and intrusion detection using non-conformity measures characteristic of affinity, a typicality, and surprise, to recognize patterns and messages as friend or foe and to respond to them accordingly. The solutions proffered throughout are built around active learning, meta-reasoning, randomness, distributed semantics and stratification, and most important and above all around adaptive Oracles. The motivation for using conformal prediction and its immediate off-spring, those of semi-supervised learning and transduction, comes from them first and foremost supporting discriminative and non-parametric methods characteristic of principled demarcation using cohorts and sensitivity analysis to hedge on the prediction outcomes including negative selection, on one side, and providing credibility and confidence indices that assist meta-reasoning and information fusion.
基金supported by the National Program for Support of Top-notch Young Professionalsthe National Natural Science Foundation of China (Grant No. 41576019)J.-Y. YU was supported by the US National Science Foundation (Grant No. AGS-150514)
文摘The tropical Pacific has begun to experience a new type of El Nio, which has occurred particularly frequently during the last decade, referred to as the central Pacific(CP) El Nio. Various coupled models with different degrees of complexity have been used to make real-time El Nio predictions, but high uncertainty still exists in their forecasts. It remains unknown as to how much of this uncertainty is specifically related to the new CP-type El Nio and how much is common to both this type and the conventional Eastern Pacific(EP)-type El Nio. In this study, the deterministic performance of an El Nio–Southern Oscillation(ENSO) ensemble prediction system is examined for the two types of El Nio. Ensemble hindcasts are run for the nine EP El Nio events and twelve CP El Nio events that have occurred since 1950. The results show that(1) the skill scores for the EP events are significantly better than those for the CP events, at all lead times;(2) the systematic forecast biases come mostly from the prediction of the CP events; and(3) the systematic error is characterized by an overly warm eastern Pacific during the spring season, indicating a stronger spring prediction barrier for the CP El Nio. Further improvements to coupled atmosphere–ocean models in terms of CP El Nio prediction should be recognized as a key and high-priority task for the climate prediction community.
文摘Positive data are very common in many scientific fields and applications;for these data,it is known that estimation and inference based on relative error criterion are superior to that of absolute error criterion.In prediction problems,conformal prediction provides a useful framework to construct flexible prediction intervals based on hypothesis testing,which has been actively studied in the past decade.In view of the advantages of the relative error criterion for regression problems with positive responses,in this paper,we combine the relative error criterion(REC)with conformal prediction to develop a novel REC-based predictive inference method to construct prediction intervals for the positive response.The proposed method satisfies the finite sample global coverage guarantee and to some extent achieves the local validity.We conduct extensive simulation studies and two real data analysis to demonstrate the competitiveness of the new proposed method.
