United States

We designed a rubric to assess free-response exam problems in order to compare thinking skills evidenced in exams in classes taught by different pedagogies. The rubric was designed based on Blooms taxonomy (revised version) [1-3]. The rubric was then used to code exam problems. We analyzed exams from different sections of the algebra-based physics course taught the same semester by the same instructor with different pedagogies. One section was inquiry-based and the other was taught traditionally. We discuss the instrument, present results and present plans for future research. The inquiry-based students demonstrated all of the thinking skills coded more often than the traditional students.

PERC SM 20

A Rubric for Assessing Thinking Skills in Free-Response Exam Problems

thinking skills

inquiry

assessment

The 2020 AAPT Virtual Summer Meeting brought together physics educators from around the globe to learn about the latest in physics theory, research methodology and education from leaders in their field. The program featured plenary speakers, sessions, workshops and many special activities.

For more information please visit the 2020 AAPT Virtual Summer Meeting page at: https://www.aapt.org/Conferences/SM2020/index.cfm.


AAPT SM 20

The 2020 AAPT Virtual Summer Meeting brought together physics educators from around the globe to learn about the latest in physics theory, research methodology and education from leaders in their field. 

poster

Join us in exploring emergent themes in PER as a scholarly endeavor and as a research community!  Since its inception, PER has served as a confluence of discipline, theory, research methodology, practice, and application.  How have these facets and the interplays among them been fruitful for physics education globally, regionally, locally, or otherwise?  And, how have our successes and interactions influenced the very nature of PER itself?
Following the lead of AAPT for the summer meeting, PERC 20/20 was virtual. Enjoy browsing the content.

Since its inception, PER has served as a confluence of discipline, theory, research methodology, practice, and application.  How have these facets and the interplays among them been fruitful for physics education globally, regionally, locally, or otherwise?  And, how have our successes and interactions influenced the very nature of PER itself?

BEMA (Brief ElectroMagnetism Accessment) tests were conducted for undergraduate students in introductory electromagnetism courses at Texas A&M University (TAMU) from 2014 - 2017 to assess students performances. A total of 3480 students participated and more than 6000 test scores are collected. The present study used a part of the data to investigate the BEMA performance with these scores taken prior to and after the class using the statistical measures based on the classical test theory. Item difficulty index, item discrimination index, point biserial coefficients and so on obtained from the TAMU data agree with the results reported by various research groups in past, and showed the test is statistically reliable. Thus, it is confirmed that administering the BEMA test is beneficial for assessing students learning of undergraduate electromagnetism at university classes.

New measurements of BEMA performance based on classical test theory

We collaborated with our universitys office of institutional research to assess pedagogical changes in our calculus-based introductory physics courses using different sets of student data. We found that switching to a competency-based grading system in these classes reduced the drop/fail/withdrawal rates and course-associated grade penalties of under-represented minority, first generation, and female students. We separately performed a longitudinal study to identify how changing the curriculum of these physics courses affected student performance in downstream engineering courses. We found that increasing the calculus content in introductory physics correlated with higher grades earned in subsequent engineering courses, and that these downstream benefits were largest for students with lower math abilities. Taken together, these results demonstrate how instructors can use educational data sets to make improvements in their courses that specifically target improving the performance and retention of traditionally underserved populations.

Changing pedagogy to help traditionally under-served populations

Building on prior analyses of how introductory physics experiences affect student attitudes, preliminary evidence suggests that IPLS students, more so than their counterparts in traditional introductory physics courses, express the attitude that physics is relevant to their primary biological interests. We report on the durability of these attitudes. We present results from interdisciplinary attitude surveys given to students a year (or more) after their initial experience in IPLS, as well as immediately after that experience. By tracking the evolution of student attitudes over time, we assess whether attitude improvements due to IPLS are in fact stable and long-lasting. We also explore how students subsequent coursework in biology or other disciplines may influence these attitudes.

Exploring the Durability of Student Attitudes Toward Interdisciplinarity

An instructional intervention consisting of students interacting with eight short videos on scientific topics was introduced to a general education physics class. The effects of these videos on student attitudes about science and their scientific reasoning skills were assessed by comparison of pre- and post-test data from the CLASS and Lawsons Test of Scientific Reasoning for two control and three treatment semesters. Initial findings showed improved student attitudes toward science and improved scientific reasoning skills. With continued use of the videos in additional semesters, similar pre-post gains in scientific reasoning skills were observed, however, less improvement in student attitudes toward science was measured. This decrease in improvement of student attitudes as the new course activity became standard has implications for the number of semesters a new curriculum aimed at improving student attitudes should be studied. Most new curricula only report a semester or two of results before wide implementation occurs.

Changes in Student Attitudes and Curricular Benefits as a New Course Activity Becomes Standard (PERC)

Many researchers have studied aspects of problem-solving. Recent work has characterized problem-solving in terms of the decisions made by and practices of an expert scientist as they solve authentic problems in their field. One crucial decision involves the assumptions that are made in the process of defining and solving an authentic problem. While making assumptions is required for simplifying authentic problems, students are provided with little opportunity to practice making simplifying assumptions and check those assumptions. Results from two studies show that one of the students' shortcomings is making erroneous assumptions and failing to effectively reflect upon accuracy of their assumptions. Indeed, unchecked assumptions can interfere with problem-solvers' ability to solve different types of physics problems despite the use of other effective practices. The results suggest that, for effective problem-solving, we should give students more training in making simplifying assumptions and reflecting upon them as they are solving problems.

Implicit and unchecked assumptions interfere with problem-solving in physics

We describe an investigation focusing on advanced students and faculty members reasoning about two questions related to the double-slit experiment with single particles. One question posed was a standard double slit question while the other question was more speculative. First, students in advanced quantum mechanics courses were asked the questions in written form and six students were interviewed individually in which they were asked follow up questions to make their thought processes explicit. We also interviewed five faculty members who had taught modern physics, quantum mechanics and/or solid state physics to understand their reasoning and thought processes. The faculty members responses shed light on what it means to think like a physicist. Student responses varied greatly in their correctness and sophistication of reasoning, suggesting that even advanced students need scaffolding support in order to learn to think like a physicist.

Advanced Students’ and faculty members’ reasoning about the double-slit experiment with single particles (PERC)

Students enter classrooms with an accumulation of experiences. This case study examined the effect of multiple factors, including undergraduate students prior knowledge and attitudes, on their (G)FCI scores and final class grades. The participants were 43 students in an introductory mechanics class for engineers. The students completed either the FCI or GFCI to test for any differences due to the context of questions. We found that students GPA, self-reported gender, prior advanced physics classes, and negative attitudes towards physics to be the most predictive variables of their scores. Male students scored on average almost twice as high as their female counterpart regardless of the test taken. Though past research has shown no significant difference between the FCI and GFCI, we did detect a difference in the perceived ease of the questions. Notably, 53% of the students did not meet the 60% threshold which indicates the initiation of Newtonian thinking

Contributing Effects to Students’ Performance on the FCI as a measure of Physics Knowledge

Research group meetings can often function as a space for members to share their own research, receive constructive feedback, and stay informed on what others in the group are working on. This poster will introduce an activity that aims to use the PER group meeting time to reflect on the work that is being done and the values that researchers bring to their research. We will discuss how the existing group norms supported the creation and execution of this activity. The poster will discuss our reasoning and process for creating the activity and will include the activity itself. Perspectives from different participants of the activity will be shared along with how these group members interacted with the activity. Poster attendees would be invited to think together with us on how they might want to adopt/adapt the activity for their own groups.

Scaffolding Collective Reflection in a Physics Education Research Group

This project models the complexity of how students and faculty think about change within university STEM departments. We argue that how people think about change can affect the change process. Our model includes the personal experiences they draw on, the way they logically connect these experiences to their conclusions about change, and their beliefs about how change happens. Viewing interviews with students and faculty in university STEM departments, we developed a change complexity codebook which suggests the potential relationship between logic, complexity, and deep-rooted change. We will argue that engaging in sensemaking is a necessary process for enacting deep-rooted change.

Modeling the Complexity of Change Perspectives and the Implications of Sensemaking

This article outlines survey results seeking to understand USA Physics Education Research (PER) Ph.D. programs. The survey explored research group composition, the number of graduates, courses taken and more. The survey was sent to a list of PER research group leaders created by crowdsourcing from the PER community. Of the 46 PER Ph.D. programs identified and invited to the survey, 25 usable responses were received. The majority of programs were in departments of physics with fewer in schools of education or institutes of science education. Most programs required graduate physics course work, with fewer requiring research methodology courses. Only five required a course in PER. The majority of graduates into academic careers. However, a robust minority pursued careers in the private sector. The survey results allow us to estimate the size of the PER community in research groups at PhD granting institutions to be n=487 people with n=156 P.h.D. students. Graduate Programs in Physics Education Research

Premium content

A Rubric for Assessing Thinking Skills in Free-Response Exam Problems

Next from PERC SM 20

New measurements of BEMA performance based on classical test theory

Similar lecture

3 Dimensional Learning in Student Work: Developing and Using Models

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES