United States

Likert-style surveys are a widely used research instrument to assess respondents&#39; preferences, beliefs, or experiences. This poster showcases a new methodology in which network analysis can be used to evaluate the interconnectedness of items in Likert-style surveys. The unique features of this approach are explored through an application of the methodology to the Aspects of Student Experience Scale (ASES) data set and a comparison of the results to principal component analysis. I show how a meaningful network based on student survey item response similarity is successfully created. I also demonstrate how modular analysis can be used in order to identify larger themes built from the connections between particular survey items. Our proposed methodology is widely applicable and provides a new way to investigate phenomena assessed by Likert-style surveys.

AAPT SM 22

Network analysis of Likert-style surveys

AAPT meetings are a great way to reflect on your teaching, recommit to best practices and re-energize your career with new research, products, and techniques. Listen to internationally known plenary speakers, attend sessions, workshops, and special activities. This meeting is an excellent opportunity to share your expertise and learn from and network with leaders in the field.



Likert-style surveys are a widely used research instrument to assess respondents' preferences, beliefs, or experiences. This poster showcases a new methodology in which network analysis can be used to evaluate the interconnectedness of items in Likert-style surveys. The unique features of this approach are explored through an application of the methodology to the Aspects of Student Experience Scale (ASES) data set and a comparison of the results to principal component analysis. I show how a meaningful network based on student survey item response similarity is successfully created. I also demonstrate how modular analysis can be used in order to identify larger themes built from the connections between particular survey items. Our proposed methodology is widely applicable and provides a new way to investigate phenomena assessed by Likert-style surveys.

poster

Significant attention in the PER community has been paid to student cognition and reasoning processes in undergraduate quantum mechanics. Until recently, however, these same topics have remained largely unexplored in the context of emerging interdisciplinary quantum information science (QIS) courses. We conducted exploratory think-aloud interviews with 24 students in an upper-division quantum computing course at a large R1 university crosslisted in physics and computer science, as well as 5 graduate students in a similar graduate-level QIS course offered in physics. We classify and analyze students' responses to a pair of questions regarding the fundamental differences between classical and quantum computers, interpreting students' responses as a possible window into the underlying ontologies of quantum mechanics students develop through such coursework.

Investigating student interpretations of differences between classical and quantum computers

The application of various machine learning algorithms and techniques to the prediction of student performance in an introductory mechanics class is discussed (N = 2061 students). The dataset used for prediction includes institutional variables (GPA, credit hours) and in-class variables gathered from the first two weeks of class (homework and clicker scores). These variables are used to predict which students are at risk of failing the course (receiving a D or F). The effectiveness and implementations of random forest, logistic regression, and voting ensemble classification models are discussed. Various model optimization methods are explored including decision threshold tuning, model regularization, synthetic upsampling, and variable selection techniques. The best performing models achieved a balanced accuracy of around 80%.

Machine Learning Techniques for Classifying Physics Performance

This work is part of a multi-institution collaboration developing a fair, valid, and reliable research-based conceptual fluids assessment, the Fluids Conceptual Evaluation (FCE). The FCE will utilize two-tier multiple-choice items covering both fluids statics and fluids dynamics. Rasch analysis will be used to create different equivalent versions of the FCE, allowing instructors to give just the statics questions, just the dynamics questions, or a combination of both. In this poster we will describe the development of student-based distractors for the FCE. IPLA instructors interested in serving as a pilot test site should contact Project Lead Dawn Meredith.

Development of questions for the Fluids Conceptual Evaluation (FCE)

This work is part of a multi-institution collaboration developing a fair, valid, and reliable research-based conceptual fluids assessment, the Fluids Conceptual Evaluation (FCE). The FCE will utilize two-tier multiple-choice items covering both fluid statics and dynamics. Initially we developed 101 tier-1 questions covering 8 statics constructs and 9 dynamics constructs, with the intent of including 3 questions per construct in the final assessment. In the spring of 2021, we conducted 15 clinical interviews asking student volunteers selections from those questions. Analysis of interviews, and feedback from the interviewers, led to the revision, removal, and addition of several questions, which were used in the second round of interviews during the 2021-2022 academic year. In this poster we describe selected takeaways from this process, including some “dos and don’ts” for constructing robust questions. IPLS instructors interested in serving as a pilot test site for the assessment should contact Project Lead Dawn Meredith.

Refining Assessment Questions Based on Clinical Interviews

Creating a more equitable scientific community is an important goal in physics. To this end, one approach is addressing students’ views about the socially and culturally embedded nature of science (NOS) and increasing their awareness about the influence of subjectivity in physics. Identifying ways personal experience and values affect the culture of physics, may promote student self-efficacy and empower them to work towards greater social justice in physics. Prior studies reveal that without explicit instruction about the NOS, students tend to describe physics without reference to human influence. Adding to this, we explored undergraduate students’ ideas about the presence of subjectivity in physics in an introductory calculus-based physics sequence and with prospective elementary teachers in a physics course. We collected written responses on whether students viewed physics as objective or subjective. In this talk, we discuss how student responses describe subjectivity in physics and argue for explicit NOS instruction.

Addressing Subjectivity in Physics for Equity in Physics Education

The PhysPort Data Explorer (www.PhysPort.org/DataExplorer) is an online tool for physics instructors to analyze their research-based assessment data (including FCI, FMCE, BEMA, CSEM, CLASS, MPEX, and many more). Instructors upload their students' responses using our secure interface. The Data Explorer matches their pre/post data, scores it, compares it to national data, and graphs it in an interactive and intuitive manner. Instructors can look at results over time, breakdown by question or cluster, and comparisons between courses. Instructors can also upload data related to student demographics, academic record, or background, and look at their assessment results based on this information. We present the Data Explorer and discuss equity issues around analysis and interpretation of research-based assessment results, in order to support faculty in using data to reveal and address inequities, to more effectively support minoritized groups of students, and to minimize and eliminate harm.

Using the PhysPort Data Explorer to analyze research-based assessment results

Force symbols in University Modeling Instruction explicitly represent forces as detailed descriptions of interactions to facilitate (i) coordinating force with the system schema and (ii) learning force. For example, represents the gravitational force by Earth on ball, where “g” represents gravitational (i.e. the type of interaction), “E” represents Earth, represents “by” and “on”, and “B” represents ball. Although students are taught to say as “gravitational force”, audio data from student-led whole-class discussions shows that more than 40% percent of the time was referred to as “force gravity” instead. Analogous results were obtained for contact force symbols as well. Because language plays such a crucial role in learning physics, several years ago, as an experiment, the notation was changed from to to make it more closely match how it is to be read. Student use of “force gravity” and “force contact” dropped to less than 5% with this notation switch. Energy.

Changing Notation That Represents Force Changes How Students Say It

With the growing ubiquity of computation in STEM fields, understanding how to teach computational thinking (CT) practices has become an active research area in the last two decades, with particular emphasis on developing CT frameworks. In this paper, we apply one of these CT frameworks and correlate the results with a task analysis to examine how CT practices relate to specific design features of an in-class problem. We have analyzed video data from two separate groups working on one computational class period, which utilizes a minimally working program to model magnetic field vectors. While still in the initial stages of the study, our preliminary results indicate that what is left out of the minimally working program will impact the CT practices students use, particularly around building computational models. Ultimately, we hope this work will help instructors to design activities that can target & build specific CT practices

Relating Computational Thinking Practices and Problem Design Features

We’re interested in ways that students collaborate to solve conceptual physics problems in the context of spherical unit-vectors in upper-division E&M, especially ones that are difficult for individuals to solve, but which small groups are more successful at. Using a think-out-loud protocol, we interviewed students in small groups, asking them to solve together problems from this context on a large whiteboard. Using the resources framework and an emergent coding analysis of interview video, we observed one common mechanism in all three group-interviews whereby students collaborated effectively: first one student activated a conceptual resource and expressed it, then another student took up that idea, then finally the whole group together used that idea to move forward with the problem. In this poster, we report our method of analysis and results from one group interview to illustrate a resource activated and used in this way, which we call a shared resource.

Shared Resources in Student Problem-Solving of Spherical Unit Vectors

This poster analyzes two examples of whiteboard meetings from a college calculus-based introductory physics course taught using University Modeling Instruction. In UMI, student small groups create a solution to the same problem on 2? × 3? whiteboards. They then sit in a large circle with whiteboards held facing in and conduct a student-led whole-class discussion to reach consensus. In the first example students overcame sharp disagreements to reach consensus, but in the second example they don’t. We examine how social positioning contributed to students either successfully examining and resolving different ideas or failing to do so. Results from these two examples support the idea that meetings where “experts” soften their position by “hedging” more frequently are better able to overcome sharp initial disagreements to reach consensus on their own. Our analysis suggests that the way students position themselves in discussions may open or close the collaborative space to productive sense-making.

Premium content

Network analysis of Likert-style surveys

Next from AAPT SM 22

Investigating student interpretations of differences between classical and quantum computers

Similar lecture

Causal mapping analysis of Universal Design for Learning-aligned instructional changes

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES