© 2001 International Association for the Evaluation of Educational Achievement (IEA)
How Do Benchmarking Participants Compare with International Benchmarks of Mathematics Achievement?
The TIMSS mathematics achievement scale summarizes student performance on test items designed to measure a wide range of student knowledge and proficiency. In order to provide descriptions of what performance could mean in terms of the mathematics that students know and can do, TIMSS identified four points on the scale for use as international benchmarks(7) or reference points, and conducted an ambitious scale anchoring exercise to describe students performance at these benchmarks. Exhibit 1.3 shows the four international benchmarks of mathematics achievement and briefly describes what students scoring at these benchmarks typically know and can do. More detailed descriptions appear in Chapter 2, together with example test items illustrating performance at each benchmark.
The Top 10% Benchmark is defined at the 90th percentile on the TIMSS mathematics scale, taking into account the performance of all students in all countries participating in 1999. It corresponds to a scale score of 616 and is the point above which the top 10 percent of students in the TIMSS 1999 assessment scored. Students performing at this level demonstrated that they could organize information, make generalizations, and explain solution strategies in non-routine problem-solving situations.
The Upper Quarter Benchmark is the 75th percentile on the mathematics scale. This point, corresponding to a scale score of 555, is the point above which the top 25 percent of students scored. Students scoring at this benchmark demonstrated that they could apply their mathematical understanding and knowledge in a wide variety of relatively complex situations involving fractions, decimals, geometric properties, and algebraic expressions.
The Median Benchmark, with a score of 479, corresponds to the 50th percentile, or median. This is the point above which the top half of students scored on the TIMSS 1999 assessment. Students performing at this level showed that they could apply basic mathematical knowledge in straightforward situations, such as one-step word problems involving addition and subtraction or computational problems based on basic properties of geometric figures and simple algebraic relationships.
The Lower Quarter Benchmark is the 25th percentile and corresponds to a scale score of 396. This score point is reached by the top 75 percent of students and may be used as a benchmark of performance for lower-achieving students. Students scoring at this level typically demonstrated computational facility with whole numbers.
Exhibit 1.4 displays the percentage of students in each participating entity that reached each international benchmark, in decreasing order by the percentage reaching the Top 10% Benchmark. If student achievement in mathematics were distributed alike in every entity, then each entity would be expected to have about 10 percent of its students reaching the Top 10% Benchmark, 25 percent the Upper Quarter Benchmark, 50 percent the Median Benchmark, and 75 percent the Lower Quarter Benchmark. Although countries such as New Zealand, and Benchmarking participants such as Maryland, North Carolina, and the Delaware Science Coalition, came fairly close, no entity followed this pattern exactly. Instead, the high-performing entities generally had greater percentages of students reaching each benchmark, and the low-performing entities had lesser percentages.
Among the high performers, for example, Singapore, Chinese Taipei, Korea, Hong Kong, and Japan had one-third or more of their students reaching the Top 10% Benchmark, about two-thirds reaching the Upper Quarter Benchmark, around 90 percent reaching the Median Benchmark, and almost all (95 to 99 percent) reaching the Lower Quarter Benchmark. In comparison, the Naperville School District and the First in the World Consortium had 24 and 22 percent of their students, respectively, reaching the Top 10% Benchmark and 59 and 56 percent, respectively, reaching the Upper Quarter Benchmark, somewhat less than in the high-performing Asian countries. More like the top-performing Asian countries, these two high-performing districts had close to 90 percent of their students reaching the Median Benchmark (91 and 87 percent, respectively) and nearly all of their students reaching the Lower Quarter Benchmark (99 and 98 percent, respectively).
In contrast, the three lowest-performing Benchmarking participants, all urban districts, had two percent of their students reaching the Top 10% Benchmark, 9 to 12 percent reaching the Upper Quarter Benchmark, and from 29 to 41 percent reaching the Median Benchmark. The lowest-performing countries of South Africa, the Philippines, and Morocco had almost no students reaching the Top 10% Benchmark, no more than one percent reaching the Upper Quarter Benchmark, less than 10 percent reaching the Median Benchmark, and no more than 31 percent reaching the Lower Quarter Benchmark.
Although Exhibit 1.4 is organized to draw particular attention to the percentage of high-achieving students in each entity, it conveys information about the distribution of middle and low performers also. For example, Canada, Australia, and Malaysia had 12 percent of their students reaching the Top 10% Benchmark, as might be expected, but 94 to 96 percent (rather than 75 percent) reaching the Lower Quarter Benchmark. Similarly, the Academy School District, the Michigan Invitational Group, and the Project smart Consortium had 11 to 12 percent of their students reaching the Top 10% Benchmark but 95 to 96 percent reaching the Lower Quarter Benchmark.
TIMSS 1999 is a project of the International
Boston College, Lynch School of Education