Student Assessment and Program Evaluation

Aims, Actions, Adaptations

Previous Page Next Page

Elementary mathematics benchmark assessment development

At the district level, reform leaders developed components of an assessment system that served different purposes than the statewide, high-stakes assessments, but were aligned with them. For the elementary grades, the evaluation team led a design process for a mathematics benchmark assessment system—including a formative assessment component– to:

The district developed this program by bringing together individuals with a range of expertise including university faculty with knowledge of assessment, psychometric measurement, and cognitive sciences research (Dr. James Pellegrino’s group at the University of Illinois at Chicago); district staff with knowledge of the Illinois Learning Standards and district-supported mathematics instructional materials; and teachers with knowledge of their students’ learning needs and abilities.

The development process was a long one. In 2004-05, the district collected and categorized, according to the State of Illinois’ Learning Standards, over 1,000 mathematics items in grades 3-8. A pilot test was used to establish the psychometric properties of these items for CPS students. Using a matrix block design similar to that used for the National Assessment of Educational Progress (NAEP), the pilot test involved approximately 500 items in 24 schools, involving approximately 250 classrooms and 6250 students. From the results of this pilot test, three benchmark assessments (one for the fall, winter, and spring) were constructed and field tested in 2005-06. This field testing involved: further testing of the items, testing of the extended-response scoring procedures, and an examination of alternative models for district-wide administration. The field test involved over 10,000 students in grades 3-8. The district expanded the benchmark assessment to all elementary schools in the 2006-07 school year. The district also partnered with a private sector vendor to increase the item pool and handle the logistics of administration. Finally, the district utilized a technology platform for the dissemination of assessment results at the school, grade, classroom, and student levels.

Assessments as part of the high school Instructional Development Systems (IDSs)

At the high school level, the district developed, in partnership with university staff and a private test vendor, four benchmark and summative course assessments for each of the high school mathematics and science . These were the three-year, comprehensive courses that were developed as part of the district’s High School Transformation initiative. For each IDS, the district developed a benchmark assessment (analogous to but distinct from the elementary benchmark assessment discussed above) that was closely aligned to the specific content and pacing of the IDS. A common summative assessment was developed for the math IDSs, and another for the science. The development process for these assessments mirrored that used for the elementary benchmark assessment project. This resulted in IDS assessments that were correlated with the state PSAE assessment and highly coherent within the district.

Through this process the reform leaders sought to improve the teacher workforce’s knowledge of assessment literacy and capacity to use assessments for learning (as opposed to simply using summative assessments of learning), increase the coherence of the system in terms of expectations for what students should know and be able to do in mathematics, as well as improve classroom instruction through the targeted use of appropriate formative assessments.

Utilization-focused program evaluation

CMSI leaders sought to develop an environment where program evaluations were designed and executed to give stakeholders information and data that could be readily utilized. To them this meant that findings and data were delivered in timely ways as appropriate decisions were being discussed and which were presented in user-friendly ways in order to increase the likelihood of use. Additionally, the initiative leaders sought to involve program evaluation as an integral part of the design of program activities, often building in ways to evaluate the program activities during the initial planning stages of the activities. Finally the reform leaders allocated regular time to discuss evaluation findings and encouraged other stakeholders to both participate in these discussions and review program evaluation reports, data, and presentations.

Previous Page Next Page