Rationale
Our mission as a primarily undergraduate institution is to “provide our students with the world's best undergraduate science, engineering, and mathematics education in an environment of individual attention and support.” To provide further support for our research students and their faculty mentors, we developed a summer Undergraduate Research Community (URC) that provided supplemental workshops to the traditional research experience. These workshops, taught by faculty and staff experts, included topics around general research skills, scientific communication skills, and impostor syndrome. The workshops included lunch in many cases and were designed to promote both social interaction and skill development.
It is generally known from the assessment literature that high-impact practices like undergraduate research experiences can benefit student success (Johnson & Stage, 2018; Kuh, O’Donnell, & Schneider, 2017), and that frequency, intensity, and quality of the experience matter (Hernandez, Woodcock, Estrada, & Schultz, 2018). Furthermore, historically underrepresented students are less likely to participate in undergraduate research and more likely to benefit than majority groups (Kuh, 2008). We wanted to understand the likelihood of participation in the URC program, as well as the impacts of the program on student success indicators to identify areas to broaden participation and increase retention of our underrepresented students.

Design
We used data from 2016-2019 to conduct a quasi-experimental comparison study, comparing students who participated in the URC program with a matched group who did not participate. The overall sample included summer URC participants (n = 43 in 2017, n = 45 in 2018, and n = 35 in 2019, total n = 123 with 6 students who participated during more than one summer) and a control group of all active undergraduate students from Fall 2016-Fall 2019 (n = 1941 unique students) at a Midwestern, undergraduate-focused STEM institution. 
Variables used to model the propensity to participate in the program included sex, race/ethnicity, research program, transfer status, major, year in program, cumulative year-end GPA prior to respective summer term, and first term math grade. The model is adapted from variables established by Hernandez et al. (2018) and includes additional variables of interest (e.g., first term math grade, which is a known predictor of retention at our institution). 
Inverse propensity score weighting was used to control for observed confounding. Inverse weighting was preferred to propensity score matching (Hernandez et al., 2018; Lin, Ninon, & Byrd, 2019) in order to reduce fuzzy matching and retain all treatment samples within the prediction model (Austin, 2011; Rosenbaum & Rubin, 1983; Rubin & Thomas, 1996). We removed students from the model who had a nearly 0% or 100% chance of participating in the URC to reduce weighting bias. 
Student success outcomes included four dichotomous variables: 1) graduation rate less than 6 years, 2) final cumulative GPA greater than 2.0, 3) plans to pursue graduate school or a research position, and 4) plans to join the STEM workforce.

Results/Program Outcomes 
We will share results concerning the likelihood of participation in the URC, as well as how URC participation impacts student success outcomes like graduation rate, final GPA, and placement after graduation. 

Limitations
Other variables of interest include prior research experience, number of URC workshops attended, and prior exposure to course-based research experiences. In the absence of reliable data for these variables, we recommend more systematic data collection to improve the predictive ability of our proposed model. 

References
Austin, P. C. (2011). An Introduction to Propensity Score Methods for Reducing the Effects of Confounding in Observational Studies. Multivariate Behavioral Research, 46(3), 399–424. https://doi.org/10.1080/00273171.2011.568786

Hernandez, P. R., Woodcock, A., Estrada, M., & Schultz, P. W. (2018). 
Undergraduate Research Experiences Broaden Diversity in the Scientific Workforce. BioScience, 68(3), 204–211. https://doi.org/10.1093/biosci/bix163

Johnson, S. R., & Stage, F. K. (2018). Academic Engagement and Student Success: Do High-Impact Practices Mean Higher Graduation Rates? The Journal of Higher Education, 89(5), 753–781. https://doi.org/10.1080/00221546.2018.1441107

Kuh, G. (2008). High-Impact Educational Practices: What They Are, Who Has Access to Them, and Why They Matter. Retrieved from Association of American Colleges & Universities website: https://secure.aacu.org/imis/ItemDetail?iProductCode=E-HIGHIMP&Category=

Kuh, G., O’Donnell, K., & Schneider, C. G. (2017). HIPs at Ten. Change: The Magazine of Higher Learning, 49(5), 8–16. https://doi.org/10.1080/00091383.2017.1366805

Lin, W., Ninon, S., & Byrd, A. (2019, October). Using Propensity Score Matching to Assess Student Success. Conference presentation presented at the 2019 Assessment Institute, Indianapolis, IN. Retrieved from https://assessmentinstitute.iupui.edu/overview/institute-files/2019-institute/handouts-2019.html

Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41–55. https://doi.org/10.1093/biomet/70.1.41

Rubin, D. B., & Thomas, N. (1996). Matching Using Estimated Propensity Scores: Relating Theory to Practice. Biometrics, 52(1), 249. https://doi.org/10.2307/2533160


Speaker bio:Dr. Jordan Trachtenberg is the Director of Assessment in the Office of Institutional Research, Planning, and Assessment at Rose-Hulman.

Predictors of Participation and Outcomes for a Summer Undergraduate Research Community (URC)

Exploring the Undergraduate Research Experience

technical paper

UI is excited to announce the rescheduled 12th Annual Conference on Understanding Interventions December 10 & 12, 2020! As you may recall, the original conference scheduled back in March 2020 had to be cancelled due the worsening global conditions, and further declaration of the COVID-19 pandemic. Stay tuned for details regarding the virtual conference.

On December 10, 2020 from 12:00-5:00pm (EST), and December 12, 2020 from 11:00am-5:00pm (EST), UI will host the Virtual 12th Annual Understanding Interventions Conference powered by Underline. Attendees will experience plenaries, workshops, symposia, and poster presentations on hypothesis-based research on interventions and initiatives that broaden participation in STEM careers. The conference will continue to facilitate the dialogue among education, behavioral, and social science researchers, evaluators, and faculty in STEM fields who participate in intervention programs. Graduate students in these fields are strongly encouraged to attend. Workshops, sessions, and posters will communicate effective strategies from successful STEM intervention programs, present results from empirical research studies, and synthesize the translation of research findings into practice on STEM learning, undergraduate research, graduate education, and student retention.

12th Conference on Understanding Interventions 2020

12th Annual Understanding Interventions Conference continues to facilitate the dialogue among education, behavioral, and social science researchers, evaluators, and faculty in STEM fields who participate in intervention programs. Workshops, sessions, and posters communicate effective strategies from successful STEM intervention programs, present results from empirical research studies, and synthesize the translation of research findings into practice on STEM learning, undergraduate research, graduate education, and student retention.

Jordan Trachtenberg

1

3

SHORT BIO

Presentations

Predictors of Participation and Outcomes for a Summer Undergraduate Research Community (URC)

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES