Outcomes
How big is big?
In education, like other fields, we often lack well agreed upon benchmarks for what size of change, say in student performance, is “large”. In these effect size studies we employed three different benchmarks to try to provide context for the magnitude of the changes that were measured. This concept can perhaps best be explained through an analogy. For example, if a runner institutes a new training routine and runs a race 10 seconds faster today than he/she did before using the training routine and you want to get a sense of how big or small this improvement was you could use several different criteria. For example, you could ask a lot of other experienced runners what they would expect the change to be and compared the actual change to their expectations for the change (normative expectation). Or you could look historically at the changes other runners of various types, such as recreational runners, high school track team members, professionals, or world record holders, make and compare those results to the given runner's performance (policy relevant). Or you could look at the changes that runners who used training routines different from this runner's new training routine have performed (similar intervention). Each of these is a different benchmark for comparison. Each of these approaches has advantages and disadvantages and answers slightly different questions. Taken together, we hope to obtain a deeper understanding of the phenomena under investigation, in this case student performance.
