Student Assessment and Program Evaluation
Aims, Actions, Adaptations
Elementary mathematics benchmark assessment development
At the district level, reform leaders developed components of an assessment system that served different purposes than the statewide, high-stakes assessments, but were aligned with them. For the elementary grades, the evaluation team led a design process for a mathematics benchmark assessment system—including a formative assessment component– to:
- Provide information in a useable way to teachers and students about student learning relative to the Illinois Learning Standards.
- Provide information that would inform instruction.
- Provide administrators and teachers with ongoing measures of student performance to help schools target areas of instructional need.
- Use formats consistent with the state’s large-scale assessment
- Deepen teachers’ content and pedagogical knowledge by promoting dialogue among teachers about student learning.
- Build and develop norms for reflective practice.
- Be reliable and valid for the designed purposes.
The district developed this program by bringing together individuals with a range of expertise including university faculty with knowledge of assessment, psychometric measurement, and cognitive sciences research (Dr. James Pellegrino’s group at the University of Illinois at Chicago); district staff with knowledge of the Illinois Learning Standards and district-supported mathematics instructional materials; and teachers with knowledge of their students’ learning needs and abilities.
The development process was a long one. In 2004-05, the district collected and categorized, according to the State of Illinois’ Learning Standards, over 1,000 mathematics items in grades 3-8. A pilot test was used to establish the psychometric properties of these items for CPS students. Using a matrix block design similar to that used for the National Assessment of Educational Progress (NAEP), the pilot test involved approximately 500 items in 24 schools, involving approximately 250 classrooms and 6250 students. From the results of this pilot test, three benchmark assessments (one for the fall, winter, and spring) were constructed and field tested in 2005-06. This field testing involved: further testing of the items, testing of the extended-response scoring procedures, and an examination of alternative models for district-wide administration. The field test involved over 10,000 students in grades 3-8. The district expanded the benchmark assessment to all elementary schools in the 2006-07 school year. The district also partnered with a private sector vendor to increase the item pool and handle the logistics of administration. Finally, the district utilized a technology platform for the dissemination of assessment results at the school, grade, classroom, and student levels.
Assessments as part of the high school Instructional Development Systems (IDSs)
At the high school level, the district developed, in partnership with university staff and a private test vendor, four benchmark and summative course assessments for each of the high school mathematics and science
Through this process the reform leaders sought to improve the teacher workforce’s knowledge of assessment literacy and capacity to use assessments for learning (as opposed to simply using summative assessments of learning), increase the coherence of the system in terms of expectations for what students should know and be able to do in mathematics, as well as improve classroom instruction through the targeted use of appropriate formative assessments.
Utilization-focused program evaluation
CMSI leaders sought to develop an environment where program evaluations were designed and executed to give stakeholders information and data that could be readily utilized. To them this meant that findings and data were delivered in timely ways as appropriate decisions were being discussed and which were presented in user-friendly ways in order to increase the likelihood of use. Additionally, the initiative leaders sought to involve program evaluation as an integral part of the design of program activities, often building in ways to evaluate the program activities during the initial planning stages of the activities. Finally the reform leaders allocated regular time to discuss evaluation findings and encouraged other stakeholders to both participate in these discussions and review program evaluation reports, data, and presentations.
