United States

This study advances the work of previous studies that have applied machine learning algorithms to identify students who are at risk of failure in introductory calculus-based physics. Those prior studies found it very difficult to predict the outcome of students who received a D, and F, or W (withdrawal from the course). This study applies ensemble machine learning—using two or more machine learning algorithms in tandem—to identify students at-risk of failing introductory calculus-based physics. These algorithms are applied using institutional variables such as college GPA and in-class variables such as homework grades. Non-cognitive variables gathered through in-class surveys, such as self-efficacy, are also used to improve identification accuracy.

AAPT SM 21

Using Ensemble Machine Learning to Identify At-Risk Physics Students

identifying

classifier

machine learning

The 2020 AAPT Virtual Summer Meeting brought together physics educators from around the globe to learn about the latest in physics theory, research methodology and education from leaders in their field. The program featured plenary speakers, sessions, workshops and many special activities.

For more information please visit the 2020 AAPT Virtual Summer Meeting page at: https://www.aapt.org/Conferences/SM2020/index.cfm.


AAPT SM 20

The 2020 AAPT Virtual Summer Meeting brought together physics educators from around the globe to learn about the latest in physics theory, research methodology and education from leaders in their field. 

poster

Join AAPT and Physics Educators worldwide in developing and strengthening your professional connections by attending the 2021 AAPT Virtual Summer Meeting. AAPT meetings are a great way to reflect on your teaching, recommit to best practices and re-energize your career with new research, products, and techniques.

Listen to internationally known plenary speakers, attend sessions, workshops, and special activities. Attending the virtual summer meeting is an excellent opportunity to share your expertise and learn from and network with leaders in the field.

Please log in with your AAPT account or register for the event if you haven't already.

If you are registered please note that you need to be logged in with your AAPT email / credentials in order to get access.

An instructor may often wonder what their students understand from their lessons or may want to know what students already understand before coming to class. While formative assessments can help, they can also be stressful and time consuming. During the fall and spring semesters from 2018 to 2021 in Physics I and Physics II, students reported their level of understanding of the material before and after each class. This information helped to inform the instructor on how to teach the class as well as how well the class went and what material might need to be reviewed. We graphed the average level of understanding for each class session to compare the students’ understanding before and after class, between semesters, and between years. Each semester is analyzed for trends, and we plan to share the interesting results that were found.

Measuring students’ level of understanding in introductory physics courses

We report on the development of a written assessment designed to analyze pedagogical content knowledge in the context of questioning (PCK-Q). We discuss the process of analyzing and coding classroom videos and writing and testing a written assessment. The classroom observations have first been coded using a coding scheme that analyzes levels of questioning in an inquiry-based classroom. A written instrument was then developed based on the classroom videos, administered to student teaching assistants (TAs and LAs) and compared with the video results to validate the written instrument. We report on the coding scheme and the validation of results as part of this ongoing project.

Development of an Instrument for Analysis of Student Assistants’ PCK-Q

This poster analyzes two examples of whiteboard meetings from a college calculus-based introductory physics course taught using University Modeling Instruction. In this pedagogy, students work in small groups to create a solution to the same problem on 2′ × 3′ whiteboards. They then sit in a large circle with their whiteboards held facing in and conduct a student-led whole-class discussion (“board” meeting) to reach a consensus. One example is given of a conversation where students overcame sharp disagreements to eventually reach whole-class consensus and another example is given where they did not. We examine how social positioning contributed to students either successfully examining and resolving different ideas or failing to do so. Our analysis suggests that the way students position themselves in discussions may open or close the collaborative space to productive sense-making.

Social-positioning correlates with consensus building in two contentious large-group meetings

In his well-known paper Hake states that “a consistent analysis over diverse student populations with widely varying initial knowledge states […] can be obtained by taking the normalized average gain <g> as a rough measure of the effectiveness of the course in promoting conceptual understanding.” The practice of measuring course learning gains is common, despite the fact that variation in <g> on the FCI has also been tied to population effects, such as SAT scores, along which diverse populations are spread [Coletta, 2007]. In our data set of thousands of introductory mechanics students over more than a decade, we see a strong correlation between FMCE gain and math preparation level. We present the lessons we have learned about using normalized gain to study physics instruction, and we suggest how PER researchers might move beyond it to better compare learning in courses that reflect the diversity of the student population.

Looking Beyond the Normalized Gain

Computer simulations have been used to support student learning in physics, not only to boost conceptual understanding but to make labs more widely accessible to different populations. However, their impact on student learning has shown mixed results. To better understand this impact, HTML5-based computer simulations for topics in mechanics were investigated in a large, algebra-based, studio physics course for life science students at a private, research-intensive institution. Over a period of three years, we compared learning outcomes of students using traditional hands-on equipment only, a simulation only, or a hybrid combination of both for activities on energy, momentum, simple harmonic motion, and rotational dynamics. We will present our findings of this study through an equity lens, both from the experience of online learning due to the global pandemic and from consideration of the limited access to lab equipment for many students.

Use of physics simulations in supporting equity and student learning

Calls to reform to science teaching have elicited numerous responses over the past decade. One avenue of reform, 3 Dimensional Learning (3DL), focuses on what students learn, instead of how students learn. 3DL achieves this by organizing science learning around the same 3 dimensions that experts organize their knowledge: Scientific Practices, Disciplinary Core Ideas, and Crosscutting Concepts. To aid transformations toward 3DL, the 3DL4US group has created protocols to help characterize when 3DL may occur on assessments and in instruction. In this talk, we present initial results that extend this effort to student engagement with the Scientific Practice of Developing and Using Models. We will look at results from student problem solving interviews to see how the practice manifested in their work. These results will help inform us about the effectiveness of instructional efforts to engage students with the Practice of Using Models as we attempt to create 3DL environments.

Student Engagement in the Practice of Developing and Using Models

Reasoning across different contexts consistently is a challenge when learning physics. Despite demonstrating correct conceptual understanding and reasoning on one task, students often fail to use the same knowledge and skills on similar tasks. Dual-process theories can explain these inconsistencies, asserting that cognition involves two thinking processes: the heuristic, which is fast and automatic, and the analytic, which is slow and deliberate. Inconsistent responses arise when the heuristic process produces an intuitively appealing (but incorrect) response that the analytic process fails to reject. We hypothesize making the dual nature of human cognition explicit and more visible to the students could help them recognize and evaluate intuitive responses more productively. Results of a study designed to probe the impact of such instruction on student learning in introductory calculus-based mechanics courses are presented here.

The effect of dual process theory instruction on student learning*

In Summer 2020, an accelerated, 9-week version of the full-year introductory physics laboratory sequence was offered in the emergency remote teaching (ERT) environment. The three term sequence covers mechanics in the first course, electromagnetism in the second, and finally wave, optics and thermodynamics in the last course. Students had to complete three labs each week. We discuss the process for transitioning the traditional laboratory curriculum of the courses to ERT. We also describe changes to the third course in the sequence that were informed by preliminary results from a study of students’ attitudes conducted during the rapid remote implementation of the Spring 2020 instance of the course. Finally, we show changes in students’ attitudes over the three course sequence.

Students’ Self-Efficacy in an Accelerated Remote Introductory Lab Sequence

Advisors and graduate students are sometimes stuck and not moving their research forward. We use Deliberate Innovation(DI) techniques to understand the specific situations centered around research that graduate students and their advisors find themselves in that may be problematic. A critical element of this is determining whether advisors and advisees show indifference in particular situations. We understand indifference by noticing what people are and are not drawn to in the research setting, and will be focusing on ways in which we have seen it show up. Finding this indifference allows us to know where to focus our attention, with the ultimate goal of finding situations in which we can help graduate students and advisors get unstuck and move their research forward.

Using Deliberate Innovation to Understand Indifference in Graduate Advisor/Advisee Relationships

Motion is used in a variety of ways in introductory mechanics contexts. Given the foundational importance of related ideas such as force and acceleration, it might behoove instructors to understand how their students are interpreting this all-encompassing word. In this study, we explored how beliefs and conceptions about motion are refined or used by undergraduate, upper-division physics students in a series of think-aloud interviews with coupled-multiple response (CMR) items developed from free-response data and with two adapted example questions from the Force and Motion Conceptual Evaluation (FMCE).

Premium content

Using Ensemble Machine Learning to Identify At-Risk Physics Students

Next from AAPT SM 21

Measuring students’ level of understanding in introductory physics courses

Similar lecture

Applying Machine Learning to Automatically Assess Middle-School Students’ Argumentation

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES