Student Assessment and Program Evaluation
Context
Measuring Student Achievement
History of high-stakes assessments in general
Over the last few decades the standards movement has heavily influenced K-12 education (Education Week, 1999). At the national level, several organizations have developed and published standards for mathematical and science content knowledge and skills. These standards articulate what students should know and be able to do, and so prescribe what should be taught in schools. Organizations include the National Council of Teachers of Mathematics (National Council of Teachers of Mathematics, 2000), National Science Education standards (National Research Council (U.S.), 1996), Achieve, and New Standards (New Standards, 1997), to name just a few. In addition, most states and large school districts have implemented achievement standards. To know if standards-based systems of teaching and learning are effective, states and school districts have invested heavily in assessment systems. Throughout the 1990s states and districts attached increasingly higher stakes to assessment results. This trend continued at the national level with the passing of the No Child Left Behind Act of 2001 (“NCLB” or “the Act”) (107th Congress of the Unites States, 2001).
High-stakes assessments in Chicago
These standards-driven trends have been evident in Chicago Public Schools. Beginning in 1995, CPS undertook one of the largest efforts in the nation to end “social promotion” by attaching high stakes consequences to standardized test results. At the elementary level, individual student performance on the nationally norm-referenced Iowa Tests of Basic Skills (ITBS) in reading and mathematics determined if a student would advance to the next grade. At the high school level, performance on a nationally norm-referenced assessment, the Test of Academic Proficiency (TAP), as well as district-constructed course exams called the Chicago Academic Standards Exams (CASE), determined if a student would be promoted to the next grade. In addition, poorly performing schools received various sanctions, ranging from probation to reconstitution, based on their school’s assessment results.
Over the next ten years, the district gradually replaced the emphasis on these particular assessments with a focus on the statewide, NCLB-mandated assessments. Beginning in 1999 for mathematics and reading, and in 2000 for science, the Illinois State Board of Education (ISBE) transitioned the state student assessment from the Illinois Goals Assessment Program (IGAP) to the Illinois Standards Achievement Test (ISAT). Driven by NCLB, Illinois reconfigured ISAT in 2005 to assess elementary students statewide against Illinois Learning Standards. Each March, the ISAT measures mathematics and reading achievement in grades 3-8, and science achievement in grades 4 and 7. In April of each year, the ISBE administers the Prairie State Achievement Examination (PSAE) to high school students in the 11th grade. This large-scale assessment includes English, reading, mathematics, and science and is composed of three parts, including a full ACT assessment. With NCLB, these statewide assessments (ISAT and PSAE) represent increasingly higher stakes for schools, as they are the basis for determining an Illinois school’s annual yearly progress (AYP), with all the potential sanctions for under-performing schools prescribed under the Act.
In addition to imposing sanctions, the district moved by 2008 to rewarding schools for positive performance, based on both “absolute attainment” and “gain” metrics for schools. Rewards can include autonomy from certain district mandates, additional teacher compensation, and more flexible school budgeting. The district has changed its assessments for making student retention decisions and it has incorporated additional measures (including student grades, attendance, and teacher recommendations) into its promotion policy, and added potential rewards for teachers and schools. However, the ISAT and PSAE are still seen as high-stakes assessments for both students and schools in Chicago.
Other kinds of assessments.
While high-stakes accountability assessments are summative assessments of learning, others, such as curriculum-embedded assessments, are designed to be formative learning aids (Black & Wiliam, 1998). The purpose of an assessment defines its design. At its foundation, assessment practice is the process of reasoning from evidence. Therefore, assessments will have different characteristics based on their purpose. Furthermore, an increasing body of research in the cognitive sciences about how people learn (National Research Council (U.S.), et al., 1999) is leading a re-examination of the role of assessment in education (Pellegrino, Chudowsky, Glaser, & National Research Council (U.S.), Committee on the Foundations of Assessment, 2001). According to this research, effective assessment cannot exist in isolation, but must be closely aligned with instructional goals. While every assessment must possess the requisite psychometric properties of reliability, validity, and fairness, the criteria for each property may be different depending on an assessment’s purpose.
Applying the lessons from cognitive science research, school districts can design different assessments for different purposes and balance the need for both large-scale and classroom-based assessments. This has been a challenge in Chicago, with its history of assessments focused exclusively on high-stakes accountability. In this context OMS leadership developed components of an assessment system for teachers and district staff to use to help make improvements in mathematics and science teaching and learning.
