|
Full Report  (1.92 MB) || Educator's Summary (161 KB)
For its quantitative analyses of overall achievement, CSRQ identified the education service providers serving the largest numbers of schools and districts. They then did an extensive search for all types of studies that evaluated these programs. The evaluations’ validity was rated as inconclusive, suggestive, or conclusive. Conclusive studies met the following criteria:
- Schools using each program had to be compared to equivalent control groups
- Schools had to have pretest scores or other baseline measures
- The outcome measures had to be reliable and valid
- The study duration had to be at least one year
To be considered “suggestive,” a study had to meet the same standards, but could have up to two “non-critical threats to validity.”
CSR programs were rated in terms of their evidence of positive achievement effects as follows:
 Very Strong Evidence of Effectiveness: At least 10 qualifying studies, with at least 5 rated “conclusive” and 75% of outcomes significantly positive, with an overall mean effect size of ES=+0.25. “Effect size” is the experimental-control difference divided by the standard deviation.
 Moderately Strong Evidence of Effectiveness: 5-9 qualifying studies, with at least 3 rated “conclusive,” and 51% to 75% of outcomes significantly positive, with an overall mean effect size of at least +0.15.
 Moderate Evidence of Effectiveness: 2-4 qualifying studies, with at least 1 rated “conclusive,” and 26% to 50% of outcomes significantly positive, with an overall mean effect size of at least +0.15.
 Limited Evidence of Effectiveness: 1 qualifying study, with 1-25% of outcomes significantly positive.
 Zero Evidence of Effectiveness: No qualifying study outcomes were significantly positive.
NR: No Rating: No qualifying studies
CSRQ also looked at evidence in three additional areas:
- Additional student outcomes
- The model's design based on research
- Services and supports provided to schools
|
