Student Outcomes

Types of ISAT Studies

Previous Page Next Page

Cross Study Implications

What one finds in research has a lot to do with the questions asked and the outcome variables of interest. Across the different ISAT studies focused on the CMSI, researchers asked many questions and used a range of outcome and predictor variables. In other words, the body of ISAT studies is diverse. The body of research has been produced over time, which has had important implications for understanding the results. The studies did not cover common time periods. Some studies are cross-sectional and cover only a single year. Some are longitudinal covering several years. Some covered the entire six-year period of the reform, 2002-2008; others covered "implementation" periods. The latter, while longitudinal in nature, focused on shorter time frames often built around two-year implementations periods for each cohort.

The findings are further complicated by the different outcome measures examined, the two most common measures being the percentage of students meeting or exceeding state standards (%ME) and scale scores. In addition to these two outcome variables, CMSI research involved analyses of derivative of these measures. These included annual change (in scale score or %ME), "momentum," or "rate of change," and it involved independent variables associated with CMSI program implementation, e.g., registration for CMSI training sessions, attendance rates at CMSI sessions, and number of sessions attended.

Further, the research methods varied from study to study. Some studies used minimal controls but covered relatively long periods. Others used highly controlled multivariate models, but covered relatively short time periods. While each set of studies separately has limitations, taken together they provide a stronger, more diverse, research base. Taken together, these analyses lead to a number of conclusions, namely: