Statistical QA Report
Pre-submission statistical review · sampleThis review covers the statistical reporting, model specification, and internal consistency of the Methods and Results sections and Tables 2 through 4. It does not evaluate the underlying research design, theoretical framing, or substantive interpretation of findings — only the statistical accuracy and reporting completeness of what is presented.
Summary of findings
| # | Issue | Location | Status |
|---|---|---|---|
| 1 | Reported degrees of freedom (df = 42) do not match the sample size in Table 2 (N = 39) | p. 11, Table 2 | ⚑ Flagged |
| 2 | No correction method stated for post hoc pairwise comparisons across 3 groups | p. 12 | ⚑ Flagged |
| 3 | ICC not reported for multilevel model despite clustering by classroom | p. 13 | ⚑ Flagged |
| 4 | Effect sizes and 95% CIs reported consistently and match APA 7th edition formatting | Throughout | ✓ Passed |
| 5 | Missing data handling (FIML) clearly described and appropriate for the design | p. 10 | ✓ Passed |
Detailed findings
Finding 1 — Degrees of freedom mismatch
p. 11 · Table 2
The Results section reports F(2, 42) for the primary ANOVA, but Table 2 lists N = 39 for the same analysis. With three groups and N = 39, the error degrees of freedom would be 36, not 42; df = 42 would imply N = 45. This kind of mismatch is one of the most common triggers for a statistical reviewer request during peer review.
Confirm which N was used in the final model (with listwise deletion applied or not) and correct either the text or Table 2 so both match exactly.
Finding 2 — Uncorrected multiple comparisons
p. 12
Post hoc pairwise comparisons are reported across the three intervention groups without a stated correction method (e.g., Tukey, Bonferroni, Holm). Running three pairwise tests at α = .05 without correction inflates the family-wise error rate above the nominal .05 threshold, to roughly .14 if the tests were independent.
Apply and report a correction method, or justify why uncorrected comparisons are appropriate for this design (e.g., pre-registered planned contrasts).
Finding 3 — Missing intraclass correlation (ICC)
p. 13
The multilevel model clusters students within classrooms but does not report the ICC from the unconditional (null) model. Reviewers in education and school psychology routinely expect the ICC to justify the multilevel approach itself.
Report the ICC from the null model alongside the design effect, briefly noting the proportion of variance attributable to classroom-level clustering.
Checks passed
Effect sizes and confidence intervals (throughout). Effect sizes and 95% CIs are reported consistently and match APA 7th edition formatting.
Missing data (p. 10). Handling of missing data with full information maximum likelihood (FIML) is clearly described and appropriate for the design.
Overall recommendation
The core analytic approach is sound and well-suited to the research questions. The three flagged issues above are all addressable without re-running the primary models, and resolving them substantially reduces the likelihood of a reviewer statistical critique. No re-analysis is recommended at this time.
This is a de-identified sample report for illustration. Manuscript title, findings, and figures are fictional and constructed to demonstrate report format and depth.