Abstract:
We present an exact probability distribution (meta-distribution) for p-values across ensembles of statistically iden-tical phenomena, as well as the distribution of the minimump-value among
mc independents tests. We derive the distribution for small samples as well as the limiting one as the sample size n becomes large. We also look at the properties of the "power" of a test through the distribution of its inverse for a given p-value and parametrization.P-values are shown to be extremely skewed and volatile,regardless of the sample size n, and vary greatly across repetitions of exactly same protocols under identical stochastic copies of the phenomenon; such volatility makes the minimum p value diverge significantly from the "true" one. Setting the power is shown to offer little remedy unless sample size is increased markedly or the p-value is lowered by at least one order of magnitude.The formulas allow the investigation of the stability of there production of results and "p-hacking" and other aspects of meta-analysis.From a probabilistic standpoint, neither a p-value of .05 nora "power" at .9 appear to make the slightest sense.
- Full paper by Nassim Taleb Here
We present an exact probability distribution (meta-distribution) for p-values across ensembles of statistically iden-tical phenomena, as well as the distribution of the minimump-value among
mc independents tests. We derive the distribution for small samples as well as the limiting one as the sample size n becomes large. We also look at the properties of the "power" of a test through the distribution of its inverse for a given p-value and parametrization.P-values are shown to be extremely skewed and volatile,regardless of the sample size n, and vary greatly across repetitions of exactly same protocols under identical stochastic copies of the phenomenon; such volatility makes the minimum p value diverge significantly from the "true" one. Setting the power is shown to offer little remedy unless sample size is increased markedly or the p-value is lowered by at least one order of magnitude.The formulas allow the investigation of the stability of there production of results and "p-hacking" and other aspects of meta-analysis.From a probabilistic standpoint, neither a p-value of .05 nora "power" at .9 appear to make the slightest sense.
- Full paper by Nassim Taleb Here
No comments:
Post a Comment