At the TIMSS 2019 2nd National Research Coordinators Meeting in Hamburg, Germany, representatives of participating countries worked together to develop mathematics and science items for the assessment.
Most item writers huddled in clusters of two or three, working with intense focus. One murmured to himself, deep in thought, while carving out geometrical shapes in the air. Some shared their thoughts on the process of item writing, and what makes a good item.
“It would have a real world applicability, a realistic context,” said Terry O'Brien of Ireland. “The child would be familiar with the scenario.”
Most item writers underscored the importance of brief, easily understood wording that still is probing. Some suggested the addition of graphics for visual interest, or starting with a clearly defined framework.
Mark Chetty of South Africa said the TIMSS framework is a very good one for a basis on which to construct an item. He added that a good item writer has content knowledge and assessment knowledge. “Often it’s one or the other, and to get the package is a difficult mix,” he said.
“You need experience in the teaching field, to know where to calculate,” added his colleague, Mariette Visser. Teachers assess students on an ongoing basis, and are familiar with how they think, she said.
While several item writers said their focus was on the cognitive domains of knowing, applying, and reasoning, June Chau of Singapore said she was thinking of items to fit into the eTIMSS framework, and visuals to accompany them. “I was thinking of how to build in the e-features,” she explained. “The drop-down menus, the sorting, and so on.”
Some countries including Singapore, the United Arab Emirates, Norway, and South Africa came prepared with items already written, and had a head start.
Item writers often professed a preference for mathematics or science. Some favored writing Grade 4 items because they previously taught at that level, or found the concepts and language more straightforward. Others preferred to create Grade 8 items because they could relate to the higher level of knowledge at that grade.
“I think the Grade 4 science items are the hardest ones to write because the language needs to be relatively simple for students to read,” said Robyn Caygill of New Zealand. “Those students tend to be lower in science [competency], and our knowledge is way above the level of the students. So bringing ourselves to that level can be quite difficult.”
Other item writers echoed some of those thoughts. “Science items are more complicated because of the length of the situation you have to establish,” said Marc Colmant of France, saying science items require more wording. “It’s more difficult to find a short situation in science.”
TIMSS & PIRLS International Study Center thanks our our item writers, who contributed close to 1,000 draft mathematics and science items for the TIMSS 2019 field test by week’s end.