TIMSS 1999 Benchmarking Logo
PublicationsTIMSS Benchmarking NewsTIMSS Benchmarking Participants
TIMSS 1999 Benchmarking
Go to the TIMSS Benchmarking home page

TIMSS Benchmarking Questions and Answers


Media Contacts:

Bill McDonald

Patricia Delaney Director of Media Relations
Boston College

TIMSS Project Contact:
Michael O. Martin
Ina V.S. Mullis
Co-Directors International Study Center


What Is the TIMSS 1999 Benchmarking Study?

The TIMSS 1999 Benchmarking Study enabled states and school districts in the United States to participate in TIMSS 1999, also known as TIMSS-Repeat or TIMSS-R. The Benchmarking Study is a voluntary component of TIMSS 1999 in which U.S. states and school districts were given an unprecedented opportunity to "benchmark" the mathematics and science achievement of their students against the world class performance of students in the top-scoring TIMSS 1999 countries. TIMSS 1999, a successor to the 1995 Third International Mathematics and Science Study (TIMSS), focused on the mathematics and science achievement of eighth-grade students. Thirty-eight countries including the United States participated in TIMSS 1999, and 27 jurisdictions from all across the U.S., including 13 states and 14 districts or consortia, participated in the Benchmarking Study.

In addition to measuring achievement, TIMSS 1999 and the Benchmarking Study investigated the contexts for learning mathematics and science in the participating entities through background questionnaires completed by students, teachers, school principals, and project coordinators from the participating entities. Information was collected about educational systems, curriculum, instructional practices, and characteristics of students, teachers, and schools, providing an extremely rich source of valuable insights into the teaching and learning of mathematics and science. The TIMSS results have stirred debate, spurred reform efforts, and provided important information to decision makers, researchers, and practitioners the world over.

Who Participated in the TIMSS 1999 Benchmarking Study?

The Participants Map (PDF), which lists the 13 states and 14 districts or consortia that participated in TIMSS Benchmarking and the 38 countries that participated in TIMSS 1999.

Geographically the Benchmarking jurisdictions were from all across the U.S., although there was a concentration of east coast participants with six of the states and several of the districts and consortia from the eastern seaboard. There was substantial diversity across the Benchmarking jurisdictions in the size and socioeconomic composition of their student populations, as well as in their per pupil expenditure on education. Although taken collectively the Benchmarking participants are not representative of the United States, the effort was substantial in scope involving approximately 1,000 schools, 4,000 teachers, and 50,000 students.

States participating in the Benchmarking Study were required to sample at least 50 schools and approximately 2,000 eighth-grade students. School districts and consortia were required to sample at least 25 schools and at least 1,000 students. Where there were fewer than 25 schools in a district or consortium, all schools were to be included, and the within-school sample increased to yield the total of 1,000 students. All but five Benchmarking jurisdictions (four states and one consortium) included public schools only. For the most part, the United States TIMSS 1999 national sample was separate from the students assessed in each of the Benchmarking jurisdictions. Each Benchmarking participant had its own sample to provide comparisons with each of the TIMSS 1999 countries including the United States.

What Are the Consortia?

The consortia consist of groups of entire school districts or individual schools from several districts that organized together either to participate in the Benchmarking Study or to collaborate across a range of educational issues. Descriptions of the consortia that participated in the project follow.

Delaware Science Coalition. The Delaware Science Coalition (DSC) is comprised of 15 school districts working in partnership with the Delaware Department of Education and the business-based Delaware Foundation for Science and Mathematics Education to improve the teaching and learning of science for all students in grades K-8.

First in the World Consortium. The First in the World Consortium consists of 18 school districts from the North Shore of Chicago that have joined forces to bring a world-class education to the region’s students and to improve mathematics and science achievement in their schools.

Fremont/Lincoln/Westside Public Schools. The Fremont/Lincoln/Westside consortium is comprised of three school districts in Nebraska. These districts joined together specifically to participate in the Benchmarking Study.

Michigan Invitational Group. The Michigan Invitational Group is a socioeconomically diverse consortium composed of urban, suburban, and rural schools across Michigan. Schools invited to participate as part of this consortium were those that were using National Science Foundation (NSF) materials, had well-developed curricula, and provided staff development to teachers.

Project SMART Consortium. SMART (Science & Mathematics Achievement Required For Tomorrow) is a consortium of 30 diverse school districts in northeast Ohio committed to long term systemic change and improved student learning in science and mathematics in grades K-12. It is jointly funded by the Ohio Department of Education and the Martha Holden Jennings Foundation.

Southwest Pennsylvania Math and Science Collaborative. The Southwest Pennsylvania Math and Science Collaborative coordinates efforts and focuses resources on strengthening mathematics and science education in the entire southwest Pennsylvania workforce region that has Pittsburgh as its center. The Collaborative is composed of all 118 "local control" public school districts, as well as the parochial and private schools in the nine-county region.

Why a Benchmarking Study?

To meet the challenge of preparing children around the world for a technologically oriented 21st century, policy makers and educators need information about students’ understanding of mathematics and science to improve learning and instruction. Over the last decade, many states and school districts have created content and performance standards targeted at improving students’ achievement in mathematics and science. There has been an enormous amount of energy expended in states and school districts not only on developing mathematics and science content standards but also on improving teacher quality and school environments as well as on developing assessments and accountability measures.

Participation in the TIMSS 1999 Benchmarking Study was intended to help U.S. states and school districts assess the comparative international standing of their students’ achievement, evaluate the rigor and effectiveness of their mathematics and science programs in a global context, and improve the teaching and learning of mathematics and science. Regardless of its performance, each state, district, and consortium will have a better idea of the challenges ahead and access to a rich array of information about various facets of its educational system. The TIMSS 1999 Benchmarking results provide an excellent basis for examining how best to move from developing curriculum frameworks or content standards in mathematics and science to meeting the extraordinary challenge of actually implementing the standards in schools and classrooms often characterized by considerable cultural, socioeconomic, and experiential diversity.

What Was the Nature of the Test?

The mathematics and science tests were based on the TIMSS curriculum frameworks, which were developed by groups of educators with input from the TIMSS National Research Coordinators (NRCs). The tests were developed through a consensus process by international experts in mathematics, science, and educational measurement, and were endorsed by all participating countries. Working within the frameworks, test specifications were developed that included items representing a wide range of mathematics and science topics and eliciting a range of skills from the students. The mathematics test covered five content areas: fractions and number sense; measurement; data representation, analysis, and probability; geometry; and algebra. The science test covered six content areas: earth science; life science; physics; chemistry; environmental and resource issues; and scientific inquiry and the nature of science.

The tests included multiple-choice questions, comprising about three-fourths of the items, and open-response items requiring students to solve problems and explain their answers. To achieve broad content coverage, a matrix sampling technique was used in which the 308 test items (162 mathematics and 146 science) were systematically distributed across eight test booklets, and the booklets were randomly distributed to students. Each student in the sampled classrooms responded to one test booklet that included about 80 mathematics and science questions, requiring 90 minutes to complete. About half the items used in 1999 have been released for public use (available at the TIMSS 1999 International web site).

What Is the Comparability of the Results?

To conduct the Benchmarking Study, the TIMSS 1999 assessments were administered to representative samples of eighth-grade students in each of the participating jurisdictions in the spring of 1999, at the same time and following the same guidelines as those established for all 38 countries. Procedures used throughout both TIMSS 1999 and the Benchmarking Study ensure that the results are comparable across participating entities. To ensure comparability in testing, rigorous procedures were designed to translate the tests where necessary, and numerous training sessions were held in data collection and scoring activities. Quality control monitors observed testing sessions in all jurisdictions and reported back to the International Study Center at Boston College, which manages the TIMSS studies. The samples of students selected for testing were scrutinized according to rigorous standards designed to prevent bias and ensure comparability. In general, the Benchmarking samples were drawn in accordance with the TIMSS standards, and achievement results can be compared with confidence. Prior to analysis, the data from each participating entity were subjected to exhaustive checks for accuracy and consistency. In short, TIMSS 1999 and the Benchmarking Study used the same achievement tests and background questionnaires, the same sampling definitions and procedures, the same test administration procedures, and the same data analysis and scaling methods, all of which ensure the comparability of the results.

How Are the Results Reported?

The results of the TIMSS 1999 Benchmarking Study are presented in two companion reports, the Mathematics Benchmarking Report: TIMSS 1999 – Eighth Grade and the Science Benchmarking Report: TIMSS 1999 – Eighth Grade.

The reports contain rankings of all participants and jurisdiction-by-jurisdiction comparisons of mathematics and science achievement overall and for each content area; comparisons of performance against international benchmarks (see below); and gender differences in performance. The achievement data are accompanied by extensive questionnaire data about the home, classroom, school, and jurisdictional contexts within which mathematics and science learning take place. In some cases results for the Benchmarking participants are reported in comparison to the results for all TIMSS 1999 countries, and in other cases in comparison to the results for selected reference countries, comprised of the United States as well as a dozen European and Asian countries of interest. These include several high-performing European countries (Belgium (Flemish), the Czech Republic, the Netherlands, and the Russian Federation), countries that are major economic trading partners of the United States (Canada, England, and Italy), and the top-scoring Asian countries of Chinese Taipei, Hong Kong SAR, Japan, the Republic of Korea, and Singapore.

In addition to reporting achievement as scale scores, student performance is also described in terms of international benchmarks of performance. In order to provide meaningful descriptions of what performance on the achievement scale could mean in terms of the mathematics and science that students know and can do, TIMSS identified four points on the scale – Top 10%, Upper Quarter, Median, Lower Quarter (90th, 75th, 50th, and 25th percentiles, respectively) – for use as international benchmarks, and conducted an ambitious scale-anchoring exercise to describe performance at these benchmarks. The percentage of students in each jurisdiction that reached each benchmark is reported. The benchmark descriptions are accompanied by example test items illustrating student performance at each benchmark.

What Publications and Resources Are Available?

In addition to the two Benchmarking Study reports, the results for the 38 countries participating in TIMSS 1999, including those for the United States, were reported in December 2000 in two companion reports, the TIMSS 1999 International Mathematics Report and the TIMSS 1999 International Science Report. Performance in the United States relative to that of other nations was reported by the National Center for Education Statistics of the U.S. Department of Education in Pursuing Excellence. The TIMSS 1999 and Benchmarking Study publications and resources are listed below.

Released April 4, 2001:
• Mathematics Benchmarking Report: TIMSS 1999 – Eighth Grade
• Science Benchmarking Report: TIMSS 1999 – Eighth Grade
• TIMSS 1999 Mathematics Released Item Set
• TIMSS 1999 Science Released Item Set

Released December 5, 2000:

• Pursuing Excellence: Comparisons of International Eighth-Grade Mathematics and Science Achievement from a U.S. Perspective, 1995 and 1999
• TIMSS 1999 International Mathematics Report
• TIMSS 1999 International Science Report
• TIMSS 1999 Technical Report

For future release:
• TIMSS 1999 Benchmarking Technical Report
• TIMSS 1999 Benchmarking Database and User Guide
• TIMSS 1999 International Database and User Guide

All the above are published by the International Study Center at Boston College, with the exception of the U.S. national report Pursuing Excellence, published by the National Center for Education Statistics.

When Is the Next TIMSS?

TIMSS 1999 was the second phase of a long-term study designed to measure trends in mathematics and science achievement, much like the regular cycle of national assessments in the U.S. conducted by the National Assessment of Educational Progress (NAEP). Work has begun on TIMSS 2003, which will assess students in grades 4 and 8. A trend study such as this has the potential for affecting policy and practice by investigating the effects on achievement of efforts in educational improvement.

Who Conducted the TIMSS 1999 Benchmarking Study?

The TIMSS 1999 Benchmarking Study was a shared venture. In conjunction with the Office of Educational Research and Improvement (OERI) of the U.S. Department of Education and the National Science Foundation (NSF), the National Center for Education Statistics (NCES) worked with the International Study Center (ISC) at Boston College to develop the study.

The TIMSS studies are conducted under the auspices of the International Association for the Evaluation of Educational Achievement (IEA), an independent cooperative of national and governmental research agencies with a permanent secretariat based in Amsterdam, the Netherlands. Its primary purpose is to conduct large-scale comparative studies of educational achievement to gain a deeper understanding of the effects of policies and practices within and across systems of education.

The IEA delegated responsibility for the overall direction and management of TIMSS 1999 to the International Study Center in the Lynch School of Education at Boston College. In carrying out the project, the International Study Center worked closely with the IEA Secretariat, Statistics Canada in Ottawa, the IEA Data Processing Center in Hamburg, Germany, and Educational Testing Service in Princeton, New Jersey. Westat in Rockville, Maryland, was responsible for sampling and data collection both for the Benchmarking Study and the U.S. component of TIMSS 1999.

Funding for the overall design, administration, data management, and quality assurance activities of the TIMSS 1999 Benchmarking Study was provided by NCES, NSF, and OERI. Each Benchmarking participant contracted directly with Boston College to fund data collection activities in its own jurisdiction. Funding for the international coordination of TIMSS 1999 was provided by NCES, NSF, the World Bank, and participating countries. Each participating country was responsible for funding local project costs and implementing TIMSS 1999 in accordance with the international procedures.


Click here to return to the ISC homepage