Re‐examining Test Item Issues in the TIMSS Mathematics and Science Assessments

As the largest international study ever taken in history, the Trend in Mathematics and Science Study (TIMSS) has been held as a benchmark to measure U.S. student performance in the global context. In‐depth analyses of the TIMSS project are conducted in this study to examine key issues of the comparative investigation: (1) item flaws in mathematics and science; (2) inability to reflect item score differences between adjacent grades; (3) ambiguity of the test items behind nonrandom guessing; and (4) unfair comparisons resulted from inconsistent item difficulties across the nations. The TIMSS item checking could help education stakeholders understand more profound assessment issues through the information triangulation.     

How Common Is the Common Core? A Global Analysis of Math Teaching and Learning

The U.S. educational system is undergoing rapid and substantial changes with many states grappling with the adoption of the Common Core State Standards in Mathematics (CCSSM). Important research questions remain unanswered regarding the potential efficacy of the CCSSM to improve student math performance compared with students around the globe. This article utilized TIMSS 2007 8th‐grade math assessment results and curricular frameworks to (1) measure the degree of overlap between the CCSSM and TIMSS standards, and (2) use this finding to create a predictive model to determine the potential efficacy of the CCSSM in improving the U.S. 8th‐grade student math performance compared with six culturally matched, TIMSS‐assessed countries, provinces, and states. Comparisons of CCSSM and TIMSS‐assessed jurisdictions show that the CCSSM holds many items in common with TIMSS‐assessed jurisdictions, but lacks rigor in some key areas. The CCSSM deficiencies include algebraic knowledge and problem solving at the 8th‐grade level, and are a significant detractor from the CCSSM when compared with TIMSS.     

Examination of the 8th grade students' TIMSS mathematics success in terms of different variables1

The aim of this study is to determine how the TIMSS mathematics success of the 8th grade students differentiates according to the school type, gender, mathematics report mark, parents' education level, cognitive domains and cognitive domains by gender. Relational survey method was used in the study. Six-hundred fifty two 8th grade students studying in the same city in Turkey participated in this study. In this study, a 45 question test that was made up by choosing TIMSS 2011 mathematics questionnaire was used as a data collection tool. Quantitative data analysis methods were used in the data analysis, frequency, percentage, average, standard deviation, independent sample test, one-way analysis of variance and post-hoc tests were applied to data by using SPSS packaged software. At the end of the study, it was determined that the school type, mathematics school mark, parents' education level and cognitive domains influenced the students' TIMSS mathematics success but their gender was a neutral element. Moreover, it was seen that schools which are really successful in national exams are more successful in TIMSS exam; students whose mathematics school marks are 5 and whose parents graduated from university are more successful in TIMSS exams than others.     

Secondary teachers’ meanings for function notation in the United States and South Korea

This study investigates what teachers in U.S. reveal about their meanings for function notation in their written responses to the Mathematical Meanings for Teaching secondary mathematics (MMTsm) items, with particular attention to how productive those meanings would be if conveyed to students in a classroom setting. We then report South Korean teachers’ responses to see whether the meanings U.S. teachers demonstrated are shared with South Korean teachers. The results show that many U.S. teachers use function notation to name rules instead of to represent relationships. The data from South Korean teachers indicates that the problematic meanings in U.S. teachers’ responses are shared with a minority of South Korean teachers. The results suggest a need for attention to ideas regarding function notation in teacher education for pre-service teachers and professional development programs for in-service teachers.     

Mathematics Teaching in Italy: A Cross-Cultural Video Analysis

This study investigates the cultural nature of teaching. It compares a sample of 39 videotaped Italian mathematics lessons to German, Japanese, and U.S. lessons videotaped in TIMSS. This study expands on earlier work that was based on a smaller sample; analysis is also extended to the nature of the mathematical content presented. The results confirm the existence of an Italian cultural pattern for mathematics teaching, whose features we outline here. Italian teachers prefer whole-class instruction to individual seatwork; they engage in teacher talk/demonstration to transmit information; and they often call on students to solve problems at the board before the rest of the class. Italian lessons are characterized by the inclusion of a large number of mathematical principles and properties. These are explained 50% of the time, and simply stated the rest of the time. This study adds yet another perspective from which mathematics teaching can be studied, and, by acknowledging the difficulty to change cultural practices, it offers practical implications for teacher learning.     

Results of an analysis of the TIMS study from a gender perspective

The results of gender analyses of the Third International Mathamatics and Science Study with students of the lower secondary school level—exactly on Year 8—are described. The analyses have been carried out and restricted to twelve selected nations. These TIMSS data were analysed following various categories, as there are mathematical content areas of the items, required mathematical qualification, type of answer, real world context and level of difficulty. The analyses do not show definite and consistent gender patterns for each category, and the apparent tendencies often are even contradictory. On the whole the analyses demonstrate that differences between countries are distinctly bigger than gender differences.     

Why do U.S. and Chinese students think differently in mathematical problem solving?: Impact of early algebra learning and teachers’ beliefs

This paper reports two studies that examined the impact of early algebra learning and teachers’ beliefs on U.S. and Chinese students’ thinking. The first study examined the extent to which U.S. and Chinese students’ selection of solution strategies and representations is related to their opportunity to learn algebra. The second study examined the impact of teachers’ beliefs on their students’ thinking through analyzing U.S. and Chinese teachers’ scoring of student responses. The results of the first study showed that, for the U.S. sample, students who have formally learned algebraic concepts are as likely to use visual representations as those who have not formally learned algebraic concepts in their problem solving. For the Chinese sample, students rarely used visual representations whether or not they had formally learned algebraic concepts. The findings of the second study clearly showed that U.S. and Chinese teachers view students’ responses involving concrete strategies and visual representations differently. Moreover, although both U.S. and Chinese teachers value responses involving more generalized strategies and symbolic representations equally high, Chinese teachers expect 6th graders to use the generalized strategies to solve problems while U.S. teachers do not. The research reported in this paper contributed to our understanding of the differences between U.S. and Chinese students’ mathematical thinking. This research also established the feasibility of using teachers’ scoring of student responses as an alternative and effective way of examining teachers’ beliefs.     

Science as a Learner and as a Teacher: Measuring Science Self‐Efficacy of Elementary Preservice Teachers

Academic science achievement of U.S. students has raised concerns regarding our ability as a nation to compete in a global economy. Additionally, research has shown that many elementary teachers have weak science content backgrounds and had poor/negative experiences as students of science, resulting in a lack of confidence regarding teaching science. However, efforts to increase science self‐efficacy (SE) in preservice teachers can help to combat these issues. This study looked at a sample of preservice elementary teachers engaged in a semester‐long science content course, using Bandura's concept of SE as a conceptual framework. Our quantitative data showed significant increases in science SE on both subscales (personal efficacy and outcome expectancy). Our qualitative data showed that students communicated an increased sense of confidence with regard to the discipline of science. In addition, students reported learning science pedagogy through the instructor's modeling. Combining our findings resulted in several meta‐inferences, one of which showed students growing as both confident learners of science and teachers of science simultaneously. We created a construct new to the literature to describe this phenomenon: “teacher‐learner,” for students are both learning science and learning to teach science simultaneously through the content course experience, resulting in increased science SE.     

The Japanese perspective on TIMSS

The International Association for the Evaluation of Educational Achievement (IEA) undertook three international mathematics studies: the first, FIMS, in 1976, the second, SIMS, in 1980, and the third, TIMSS, in 1995, 13-year-old Japanese students were included in these studies. The purpose of this paper is to discuss the level of mathematics achievement and attitudes toward mathematics among Japanese students and the situation of mathematics education in Japan, based on the results of TIMSS and previous studies. From our analysis of results, we can indicate the following points: although achievement in the fundamental techniques of calculation can be viewed in general as satisfactory, the attainment levels cannot be regarded as acceptable for problems which require a high degree of thinking and comprehension. And from the consequence of international comparison about interest and attitude, it became evident that a smaller fraction of Japanese students have a favorable opinion than in other countries.     

Validating a concept inventory for measuring students’ probabilistic reasoning: The case of reasoning within the context of a raffle

Assessing students’ conceptions related to independence of events and determining probabilities from a sample space has been the focus of research in probability education for over 40 years. While we know a lot from past studies about predictable ways students may reason with well-known tasks, developing a diagnostic assessment that can be used by teachers to inform instruction demands the use of familiar and unfamiliar contexts. This paper presents the current work of a research team whose aim is to create a formative concept inventory with strong evidence of validity that uses a psychometric model to confidently predict whether a student exhibits one or more misconception across many items. We illustrate this process in this paper using a particular item with a context of a raffle aimed to measure whether a student reasons with misconceptions related to independence or equiprobability. The results of two aspects of the validity process: cognitive interviews to assess response processes on individual items, and a large-scale administration to examine internal structure of the concept inventory revealed difficulties in assessing students’ reasoning about these key probability concepts and trends in the prevalence of misconceptions across grades. Results can provide guidance for others aiming to develop assessments in mathematics education and also support further possibilities for research into understanding students’ reasoning about independence and sample space.     

U.S. and Chinese Teachers' Constructing,Knowing, and Evaluating Representations to Teach Mathematics

This study examined U.S. and Chinese teachers' constructing, knowing, and evaluating representations to teach mathematics. All Chinese lesson plans are very similar, because they are all based on the Chinese national unified curriculum in mathematics. However, the U.S. lesson plans are extremely varied, even for those teachers from the same school. The Chinese teachers' lessons are very detailed; the U.S. teachers' lesson plans have exclusively adopted the "outline and worksheet" format. In the Chinese lesson plans, concrete representations are used exclusively to mediate students' understanding of the concept of average. In U.S. lessons, concrete representations are not only used to model the averaging processes to foster students' understanding of the concept, but they are also used to generate data. The U.S. teachers are much more likely than the Chinese teachers to predict drawing and guess-and-check strategies. For some problems, the Chinese teachers are much more likely than are the U.S. teachers to predict algebraic approaches. For the responses using conventional strategies, both the U.S. and Chinese teachers gave them high and almost identical scores. If a response involved a drawing or an estimate of an answer, the Chinese teachers usually gave a relatively lower score, even though the strategy is appropriate for the correct answer, because it is less generalizable. This study contributed to our understanding of the cross-national differences between U.S. and Chinese students' mathematical thinking. It also contributed to our understanding about teachers' beliefs from a cross-cultural perspective.     

U.S. and Chinese Teachers' Constructing, Knowing, and Evaluating Representations to Teach Mathematics

This study examined U.S. and Chinese teachers' constructing, knowing, and evaluating representations to teach mathematics. All Chinese lesson plans are very similar, because they are all based on the Chinese national unified curriculum in mathematics. However, the U.S. lesson plans are extremely varied, even for those teachers from the same school. The Chinese teachers' lessons are very detailed; the U.S. teachers' lesson plans have exclusively adopted the “outline and worksheet” format. In the Chinese lesson plans, concrete representations are used exclusively to mediate students' understanding of the concept of average. In U.S. lessons, concrete representations are not only used to model the averaging processes to foster students' understanding of the concept, but they are also used to generate data. The U.S. teachers are much more likely than the Chinese teachers to predict drawing and guess-and-check strategies. For some problems, the Chinese teachers are much more likely than are the U.S. teachers to predict algebraic approaches. For the responses using conventional strategies, both the U.S. and Chinese teachers gave them high and almost identical scores. If a response involved a drawing or an estimate of an answer, the Chinese teachers usually gave a relatively lower score, even though the strategy is appropriate for the correct answer, because it is less generalizable. This study contributed to our understanding of the cross-national differences between U.S. and Chinese students' mathematical thinking. It also contributed to our understanding about teachers' beliefs from a cross-cultural perspective.     

Relationships between emotional dispositions and mathematics achievement moderated by instructional practices: analysis of TIMSS 2015

ABSTRACT

This research is a secondary analysis with Korean students’ data collected in the TIMSS 2015 to describe the moderation effects of instructional practices on the relationships between students’ emotional dispositions toward mathematics and mathematics achievement. From the TIMSS 2015 database, we collected mathematics achievement scores, a student-level contextual scale for students’ emotional disposition, and teacher-level contextual scales representing teachers’ instructional practices. We applied hierarchical linear modelling to construct multilevel models. The findings showed that the achievement gap between emotional dispositions – like and dislike – became smaller when teachers more frequently implemented certain instructional practices like asking students to complete challenging exercises, decide their own problem-solving procedures, and express their ideas in class. Students who disliked mathematics were likely to have higher scores as their teachers implemented each of those practices more frequently. Findings provide important implications to teachers regarding: It is important to encourage students to reason through instructional practices like asking them to decide their own problem-solving procedures and to solve challenging problems.     

Fundamental Fraction Knowledge of Preservice Elementary Teachers: A Cross‐National Study in the United States and Taiwan

The purpose of this paper is to show the similarities as well as the differences of fundamental fraction knowledge owned by preservice elementary teachers from the United States (N= 89) and Taiwan (N= 85). To this end, we examined and compared their performance on an instrument including 15 multiple‐choice test items. The items were categorized into four different types of fundamental fraction constructs, including part–whole relationship, quotient, equivalence, and meanings of operations. Each item was embedded in the area, linear, or set model except for the items constructed out of the meaning of operations. Several items were featured with a pictorial illustration. Quantitative analysis showed that U.S. preservice teachers were significantly outperformed by their Taiwanese counterparts overall. The difference between the two groups was statistically significant on 12 of 15 items. Findings suggest that preservice elementary teachers from both countries need to be better prepared in their understanding of the meaning of fraction multiplication or division operations. Findings also suggest that U.S. preservice elementary teachers need to be more knowledgeable in dealing with fraction problems embedded in a linear model. Further research is suggested to study the issues raised from the findings.     

Performance Assessment in Science as a Tool to Enhance the Picture of Student Learning

The Iowa Assessment Project was funded by the National Science Foundation to explore the feasibility of combining the expertise of science teachers, science educators, and test developers to build innovative performance assessments that complement traditional, norm-referenced, multiple-choice science tests. The science teachers, graduate students, and science educators designed and tested performance assessment tasks to enhance the picture of science understanding in students through multiple points of evidence. This paper describes the design of four science performance tasks for Grade 9 students and the relationship between their performance on these tasks and multiple-choice items in the Iowa Tests of Educational Development. Students and schools used to develop the tasks were not included in the verification sample.     

Content Analysis of Gender‐related Differential Item Functioning TIMSS Items in Mathematics in Jordan

The purpose of this study is to analyze items that exhibit gender‐related Differential Item Functioning (DIF) in Mathematics in Jordan. Data was taken from the TIMSS 1999 of Jordan, which includes responses of 5,299 eighth grade students. Mantel‐Haenszel (MH) DIF procedure was applied to 124 multiple‐choice items. The results showed that 37 items exhibited gender‐related DIF. The analysis of the content of these items reflected some patterns that need further investigation. All the DIF items in measurement content favored male students while most of the DIF items in algebraic and data analysis contents favored female students. Most of the DIF items that negatively impacted on females were unfamiliar items that required some risk taking such as estimation, expectation, or approximation. On the other hand, most of the DIF items that favored females were familiar items which have one specific correct answer. Some implications for both research and teaching practice are provided.     

Cognitive level in problem segments and theory segments

Problems play an important role in mathematics instruction and are therefore frequently seen as central points of application for measures of instructional development. The research project “Quality of instruction and mathematical understanding in different cultures” examines the cognitive level of practice problems and theory problems in a three-lesson unit on the Introduction to Pythagorean theorem¹: Analogously to the TIMSS 1999 video study, a differentiation was made between the cognitive level of problem statement and the cognitive level of problem implementation. Additionally, the lesson time was also divided into practice and theory segments. The results show that teachers with a high proportion of connection activities in practice segments do not necessarily also spend a greater proportion of time on an analogous level for theory.     

Mathematical Thinking Involved in U.S. and Chinese Students' Solving of Process-Constrained and Process-Open Problems

This study examined U.S. and Chinese 6th-grade students' mathematical thinking and reasoning involved in solving 6 process-constrained and 6 process-open problems. The Chinese sample (from Guiyang, Guizhou) had a significantly higher mean score than the U.S. sample (from Milwaukee, Wisconsin) on the process-constrained tasks, but the sample of U.S. students had a significantly higher mean score than the sample of the Chinese students on the process-open tasks. A qualitative analysis of students' responses was conducted to understand the mathematical thinking and reasoning involved in solving these problems. The qualitative results indicate that the Chinese sample preferred to use routine algorithms and symbolic representations, whereas the U.S. sample preferred to use concrete visual representations. Such a qualitative analysis of students' responses provided insights into U.S. and Chinese students' mathematical thinking, thereby facilitating interpretation of the cross-national differences in solving the process-constrained and process-open problems.