Predictive power of statistical significance

A statistically significant research finding should not be defined as a P-value of 0.05 or less, because this definition does not take into account study power. Statistical significance was originally defined by Fisher RA as a P-value of 0.05 or less. According to Fisher, any finding that is likely to occur by random variation no more than 1 in 20 times is considered significant. Neyman J and Pearson ES subsequently argued that Fisher's definition was incomplete. They proposed that statistical significance could only be determined by analyzing the chance of incorrectly considering a study finding was significant(a Type Ⅰ?error) or incorrectly considering a study finding was insignificant(a Type Ⅱ error). Their definition of statistical significance is also incomplete because the error rates are considered separately, not together. A better definition of statistical significance is the positive predictive value of a P-value, which is equal to the power divided by the sum of power and the P-value. This definition is more complete and relevant than Fisher's or Neyman-Peason's definitions, because it takes into account both concepts of statistical significance. Using this definition, a statistically significant finding requires a P-value of 0.05 or less when the power is at least 95%, and a P-value of 0.032 or less when the power is 60%. To achieve statistical significance, P-values must be adjusted downward as the study power decreases.

Why a P value<0.05 does not necessarily mean statistical significance:controversy over overall survival results of the ORIENT-11 trial

On February 10,2022,the U.S.Food and Drug Administration(FDA)’s Oncologic Drugs Advisory Committee voted 14-1 against using data from the ORIENT-11 trial to support a biologics license application for sintilimab injection plus pemetrexed and platinum-based chemotherapy as first-line treatment for patients with nonsquamous non-small cell lung cancer(NSCLC)1.One major reason was that the FDA claimed overall survival(OS)was not statistically tested in ORIENT-11,while previous regular approvals were granted on the basis of statistically significant improvements in OS2.This may be a surprise to some physicians,as the ORIENT-11 trial previously reported improved OS with a hazard ratio(HR)of 0.60(95%confidence interval[CI]:0.45–0.79)and a P value of 0.00033,which is far below the commonly accepted threshold of 0.05 for declaring statistical significance.Some may argue that the OS results of ORIENT-11 were not considered statistically significant by the FDA because OS was only a secondary endpoint.However,in the KEYNOTE-024 trial4,which compared pembrolizumab with chemotherapy in patients with previously untreated advanced NSCLC with PD-L1 expression on at least 50%of tumor cells,OS was also a secondary endpoint.Nevertheless,the OS benefit of the KEYNOTE-024 trial was acknowledged by the FDA and included in the drug label5.The fundamental reason why OS results of ORIENT-11 were not considered statistically significant by the FDA is that the OS endpoint was not included in the multiplicity control strategy.

Seismogenic ULF/ELF Wave Phenomena: Recent Advances and Future Perspectives

There has been enormous progress in the field of electromagnetic phenomena associated with earthquakes (EQs) and EQ prediction during the last three decades, and it is recently agreed that electromagnetic effects do appear prior to an EQ. A few phenomena are well recognized as being statistically correlated with EQs as promising candidates for short-term EQ predictors: the first is ionospheric perturbation not only in the lower ionosphere as seen by subionospheric VLF (very low frequency, 3 kHz f 30 kHz)/LF (low frequency, 30 kHz f 300 kHz) propagation but also in the upper F region as detected by ionosondes, TEC (total electron content) observations, satellite observations, etc, and the second is DC earth current known as SES (Seismic electric signal). In addition to the above two physical phenomena, this review highlights the following four physical wave phenomena in ULF (ultra low frequency, frequency Hz)/ELF (extremely low frequency, 3 Hz frequency 3 kHz) ranges, including 1) ULF lithospheric radiation (i.e., direct radiation from the lithosphere), 2) ULF magnetic field depression effect (as an indicator of lower ionospheric perturbation), 3) ULF/ELF electromagnetic radiation (radiation in the atmosphere), and 4) Schumann resonance (SR) anomalies (as an indicator of the perturbations in the lower ionosphere and stratosphere). For each physical item, we will repeat the essential points and also discuss recent advances and future perspectives. For the purpose of future real EQ prediction practice, we pay attention to the statistical correlation of each phenomenon with EQs, and its predictability in terms of probability gain. Of course, all of those effects are recommended as plausible candidates for short-term EQ prediction, and they can be physically explained in terms of the unified concept of the lithosphere-atmosphere-ionosphere coupling (LAIC) process, so a brief description of this coupling has been carried out by using these four physical parameters though the mechanism of each phenomenon is still poorly understood. In conclusion, we have to emphasize the importance of more statistical studies for more abundant datasets sometimes with the use of AI (artificial intelligence) techniques, more case studies for huge (M greater than 7) EQ events, recommendation of critical analyses, and finally multi-parameters observation (even though it is tough work).

Traditional Chinese Medicine’s Challenge to Clinical Science and Health Policy

Traditional Chinese medicine(TCM)has been improving human health for millennia.And for that,it has gradually gained the attention of the global scientific community.TCM clinical research progresses,but slowly.I see it as being held back by perverse incentive structures in science and regulatory politics.

The quest for conditional independence in prospectivity modeling： weights-of-evidence, boost weights-of-evidence, and logistic regression

The objective of prospectivity modeling is prediction of the conditional probability of the presence T = 1 or absence T = 0 of a target T given favorable or prohibitive predictors B, or construction of a two classes {0,1} classification of T. A special case of logistic regression called weights-of-evidence （WofE） is geolo- gists＇ favorite method of prospectivity modeling due to its apparent simplicity. However, the numerical simplicity is deceiving as it is implied by the severe mathematical modeling assumption of joint conditional independence of all predictors given the target. General weights of evidence are explicitly introduced which are as simple to estimate as conventional weights, i.e., by counting, but do not require conditional independence. Complementary to the regres- sion view is the classification view on prospectivity modeling. Boosting is the construction of a strong classifier from a set of weak classifiers. From the regression point of view it is closely related to logistic regression. Boost weights-of-evidence （BoostWofE） was introduced into prospectivity modeling to counterbalance violations of the assumption of conditional independence even though relaxation of modeling assumptions with respect to weak classifiers was not the （initial） purpose of boosting. In the original publication of BoostWofE a fabricated dataset was used to ＂validate＂ this approach. Using the same fabricated dataset it is shown that BoostWofE cannot generally compensate lacking condi- tional independence whatever the consecutively proces- sing order of predictors. Thus the alleged features of BoostWofE are disproved by way of counterexamples, while theoretical findings are confirmed that logistic regression including interaction terms can exactly com- pensate violations of joint conditional independence if the predictors are indicators.

MRHCA： a nonparametric statistics based method for hub and co-expression module identification in large gene co-expression network

Background： Gene co-expression and differential co-expression analysis has been increasingly used to study co- functional and co-regulatory biological mechanisms from large scale transcriptomics data sets. Methods： In this study, we develop a nonparametric approach to identify hub genes and modules in a large co- expression network with low computational and memory cost, namely MRHCA. Results： We have applied the method to simulated transcriptomics data sets and demonstrated MRHCA can accurately identify hub genes and estimate size of co-expression modules. With applying MRHCA and differential co- expression analysis to E. coil and TCGA cancer data, we have identified significant condition specific activated genes in E. coil and distinct gene expression regulatory mechanisms between the cancer types with high copy number variation and small somatic mutations. Conclusion： Our analysis has demonstrated MRItCA can （i） deal with large association networks, （ii） rigorously assess statistical significance for hubs and module sizes, （iii） identify co-expression modules with low associations, （iv） detect small and significant modules, and （v） allow genes to be present in more than one modules, compared with existing methods.