Effect Size Matters: Empirical Investigations to Help Researchers Make Informed Decisions on Commonly Used Statistical Techniques

The present journal article formatted dissertation assessed the characteristics of effect sizes of commonly used statistical techniques. In the first study, the author examined the American Educational Research Journal (AERJ) and select American Psychological Association (APA) and American Counseling Association (ACA) journals to provide an historical account and synthesis of which statistical techniques were most prevalent in the fields of education and psychology. These reviews represented a total of 17,698 techniques recorded from 12,012 articles. Findings point to a general decrease in the use of the tvtest and ANOVA/ANCOVA and a general increase in the use of regression and factor/cluster analysis. In the second study, the author compared the efficacy of one Pearson r2 and seven multiple R2 correction formulas for the Pearson r2. The author computed adjustment bias and precision under 108 conditions (6 population p2 values, 3 shape conditions and 6 sample size conditions). The Pratt and the Olkin-Pratt Extended formulas more consistently provided unbiased estimates across sample sizes, p2 values and the shape conditions investigated. In the third study, the author evaluated the robustness of estimates of practical significance (n2, e2 and w2) in one-way between subjects univariate ANOVA. There were 360 simulation conditions (5 population Cohen's d values, 4 group proportion ratios, 3 shape conditions, 3 variance conditions, and 2 total sample size conditions) for each of three group configurations (2, 3 and 4 groups). Three indices of practical significance (n2, e2, w2) and two indices of statistical significance (Type I error and power) were computed for each of the 5,400, 000 (5,000 replications x 360 simulation conditions x 3 group configurations). Simulation findings for n2 under heterogeneous variance conditions indicated that for the k=2 and k=3 condition Cohen's d values up to 0.2 (up to 0.5 for k=4) tend to produce overestimated population n2 values. Under heterogeneous variance conditions for e2 and w2 at Cohen's d = 0.0 and 0.2, the negative variance pairing overestimated and the positive variance pairing underestimated the parameter n2 but at Cohen's d greater than or equal to 0.5, both the positive and negative variance conditions resulted in underestimated parameter n2 values.