REST PAPERS

An Exploratory Investigation of Undergraduate Students’ Values and Experiences with Breaks, Leisure, and Rest

Shaw, S. T., McReynolds, A., Neer, E., Zhu, T., & Givvin, K. (2025) - Link to Publication | Link to Preprint

In the face of worsening college student mental health, calls for greater rest and leisure are abundant. However, without an understanding of student attitudes and experiences with rest, improved mental health through rest are limited. The current study surveyed this critical context in 280 college students across two institutions by examining their attitudes, challenges, and experiences with rest. Results showed students report valuing rest as important to their productivity as students but also report feeling guilty when resting and not working. Students reported that the most common way to recognize it is time to take a break was because of physiological symptoms, including headache, eye fatigue, stomachaches, racing heart, headaches, and exhaustion. Despite students reporting a range of leisure and rest activities, including socializing, watching TV, creative hobbies, and walking, they also reported multiple barriers to resting. This included schoolwork, employment, and extracurriculars, plus feelings of guilt and worries about unfinished work. Our results indicate that, while students may believe that rest is important, they need support managing negative emotions, intrusive thoughts of unfinished work, or guilt during their rest periods to receive the full benefits of rest and leisure

Rest and Its Challenges in College Life: Results from 100 Survey Items

Shaw, S. T., Egorova, A., Vanacore, K., McReynolds, A., Neer, E., & Givvin, K. (2025) - Link to Preprint

This preprint provides a summary overview of a study that administered 100 survey items about rest to college students, with several findings of interest. First, students reported a high value of rest for their productivity and wellbeing, but also reported struggling with rest. Second, students reported that they turn to mindless distractions after work and use social media when it’s time to relax, or when they don’t feel like working and students who report using the two most popular forms of social media – TikTok and Instagram – reported that they do not generally experience feelings of relaxation, restoration, or being energized afterward. Finally students largely reported that they do not reflect on their rest periods or think about how to optimize their rest periods. This is an interesting finding, especially as students place high value on rest periods for their wellbeing and productivity and report struggling to engage in rest periods.

STATISTICS EDUCATION PAPERS

Teaching Statistics in the Age of AI: Leveraging Learning Sciences Principles with AI Tool Use to Support GAISE

Rostkowski, E., Rushton, N., Smith, H., & Shaw, S. T. (2025) - Link to Publication | Link to Preprint

In this article, we offer recommendations on how students and educators can leverage ChatGPT and other AI tools in conjunction with principles of learning sciences in introductory statistics courses. Specifically, we focus on how these tools can help meet aspects of the American Statistical Association (ASA)’s Guidelines for Assessments and Instruction in Statistics Education (GAISE).

Teaching Statistics and Data Analysis with R

Tucker, M. C., Shaw, S. T., Son, J. Y., & Stigler, J. W. (2023) - Link to Publication 

We developed an interactive online textbook that interleaves R programming activities with text as a way to facilitate students’ understanding of statistical ideas while minimizing the cognitive and emotional burden of learning programming. In this exploratory study, we characterize the attitudes and experiences of 672 undergraduate students as they used our online textbook as part of a 10-week introductory course in statistics. Students expressed negative attitudes and concerns related to R at the beginning of the course, but most developed more positive attitudes after engaging with course materials, regardless of demographic characteristics or prior programming experience. Analysis of a subgroup of students revealed that change in attitudes toward R may be linked to students’patterns of engagement over time and students’ perceptions of the learning environment.

The Better Book Approach for Education Research and Development

Stigler, J. W., Son, J. Y., Givvin, K. B., Blake, A., Fries, L., Shaw, S. T., & Tucker, M. (2020)  

Link to Publication

Despite advances in the learning sciences, a persistent gap remains between research and practice. In this project, we develop and try out a new approach to education research and development in which researchers, designers/ developers, and instructors collaborate to continuously improve an online interactive textbook. Using a “learn by doing” strategy, we first created a highly instrumented online textbook for introductory statistics. The design of our online book is based on the practicing-connections hypothesis: Instead of learning individual “bits” of information and then hoping that learners end up with transferable knowledge, we designed a curriculum to engage students in repeated practice of the connections—between core concepts, representations, and the world—that make knowledge transferable. This article describes the motivation for the textbook, the use of improvement science in developing and refining the textbook, as well as how the textbook was developed and how it is being used in statistics courses.  

MATHEMATICAL CREATIVITY & EDUCATION PAPERS

Diversity in Mathematical Insight Experiences in the Wild: Evidence of Opportunistic Assimilation.

Shaw, S. T. (2024) - Link to Publication

As evidence grows supporting the importance of non-cognitive factors in learning, computer-assisted learning platforms increasingly incorporate non-academic interventions to influence student learning and learning related-behaviors. Non-cognitive interventions often attempt to influence students’ mindset, motivation, or metacognitive reflection to impact learning behaviors and outcomes. In the current paper, we analyze data from five experiments, involving seven treatment conditions embedded in mastery-based learning activities hosted on a computer-assisted learning platform focused on middle school mathematics. Each treatment condition embodied a specific non-cognitive theoretical perspective. Over seven school years, 20,472 students participated in the experiments. We estimated the effects of each treatment condition on students’ response time, hint usage, likelihood of mastering knowledge components, learning efficiency, and post-tests performance. Our analyses reveal a mix of both positive and negative treatment effects on student learning behaviors and performance. Few interventions impacted learning as assessed by the post-tests. These findings highlight the difficulty in positively influencing student learning behaviors and outcomes using non-cognitive interventions.

Mathematical Creativity in Elementary School Children: General Patterns and Effects of An Incubation Break

Shaw, S. T., Luna, M. A. Rodriguez, B., Vilalta, N., & Ramirez, G. (2022) - Link to Publication

Research from the general field of creativity demonstrates that in the realm of problem-solving, breaks from the task at hand, known as incubation breaks, can improve idea generation and creative thinking. This study investigated whether a brief incubation break during a mathematical strategy generation task could improve elementary students’ ability to generate strategies and think more creatively. Over 200 elementary school students (grades 1–5) were asked to continuously generate mathematical strategies to solve the problem 36 – 18 for 10 min, with half randomly assigned to receive a 1-min incubation break after 5 min. Results showed that children assigned to the incubation break showed a statistically significantly higher number of strategies generated in the second block of the working period compared to students who received no break, but there were no differences in rated creativity of their strategies. Further exploratory analyses found that across grades, the number of strategies students could produce on average increased with each grade. However, when it came to the creativity of strategies, a linear trend emerged only from first through fourth grade, but fifth-grade students showed a drop in creativity.

The Mathematical Flexibility of College Students: The Role of Affective and Cognitive Factors

Shaw, S. T., Pogossian, & Ramirez, G. (2020) - Link to Publication

The current study explored the mathematical flexibility of college students who completed their K-12 education in the United States, and investigated how affective and cognitive factors contributed to flexible thinking. Participants were 128 undergraduate students at a competitive U.S. public university. Mathematical flexibility was measured through a novel task that asked participants to generate as many strategies as they could for a simple arithmetic problem. These strategies were coded to create scores of fluency (number of strategies) and flexibility (number of unique strategies). On average, participants were only able to provide little more than three unique strategies beyond the primary strategy taught in K-12 classrooms. Measures of math anxiety, math identity, need for cognition, and working memory were all unrelated to flexibility. These results provide evidence for a change-resistance account and provide further evidence that math flexibility is a unique construct. 

Math Anxiety: Past Research, Promising Interventions, and a New Interpretation Framework

Ramirez, G., Shaw, S. T., & Maloney, E. A. (2018) - Link to Publication 

Mathematics anxiety is a pervasive issue in education that requires attention from both educators and researchers to help students reach their full academic potential. This review provides an overview of past research that has investigated the association between math anxiety and math achievement, factors that can cause math anxiety, characteristics of students that can increase their susceptibility to math anxiety, and efforts that educators can take to remedy math anxiety. We also derive a new Interpretation Account of math anxiety, which we use to argue the importance of understanding appraisal processes in the development and treatment of math anxiety. In conclusion, gaps in the literature are reviewed in addition to suggestions for future research that can help improve the field's understanding of this important issue. 

COLLEGE STUDENTS AFFECT & EXPERIENCE PAPERS

Status in a Psychological Statistics Class: The Role of Academic and Status-Based Identities in College Students’ Subjective Social Status

Rahal, D., Shaw, S. T., Tucker, M., & Stigler, J. (2024) - Link to Publication

The present study assessed how perceptions of social status within the classroom—termed subjective social status—aligned with objective course performance and differed by sex, first-generation status, work status, and race/ethnicity among 713 students enrolled across three introductory statistics classes. When asked to explain how they evaluated their standing in the course, students reported five main themes, including both academic achievement with respect to exam scores and their understanding of course content. When examining differences by status-based identities in subjective social status, we found that female and first-generation students had lower subjective social status compared to their male and continuing-generation peers, although results were less robust for first-generation status. Likewise, working students reported lower subjective social status relative to non-working students, despite showing no difference in final exam score. Taken together, results suggest that factors beyond course performance may relate to students’ subjective social status, and subjective social status may contribute to disparities in academic performance, especially by sex and work status.

“Do I Really Belong Here?”: The Stigma of Being a Community College Transfer Student at a Four-Year University

Shaw, S. T., Spink, K., & Chin-Newman, C. (2019) - Link to Publication 

Students who begin their educational journeys in community college face many obstacles trying to complete their bachelor’s degrees. Much research has been dedicated to identifying academic factors that predict successful transfer and degree attainment, but relatively little research investigates how the community college experience affects these students once enrolled at the four-year university. Here, we present the results of a qualitative study that explored the challenges faced by 14 community college students during and after transfer. Specifically, we focus on student reports of a sense of stigma from having attended community college and how students overcame these feelings. Recommendations are provided for how community colleges and four-year universities can better equip their students with the knowledge and resources to combat this perception of stigma.

Lower Socioeconomic Status is Related to Higher Negative Emotional Well-Being Prior to Academic Exams

Rahal, D., Shaw, S. T., & Stigler, J. (2023) - Link to Publication 

The present study assessed whether lower social status was related to greater emotional responses in anticipation of a naturalistic stressor: academic exams among college students. College students in an introductory statistics class completed two course exams as part of this naturalistic prepost-experimental design. They provided four reports of positive, depressive, and anxious emotion – one the day before and one immediately after each exam. As hypothesized, multilevel models (ratings nested within participants) predicting emotion indicated that students with lower mother’s education had less positive emotion, more depressive emotion, and more anxious emotion the day prior to academic exams than students with higher mother’s education (proportional reductions in variance [PRV] = .013–.020). Specifically, lower mother’s education was associated with poorer well-being before but not after the exam. Exploratory models revealed that differences in emotion by mother’s education were strongest for students with lower exam scores (PRV = .030–.040). Conclusions: Socioeconomic status may influence college students’ anticipatory distress prior to academic exams, which may impact health and academic performance.

Impacts of the COVID-19 Transition to Remote Instruction for University Students

Rahal, D. & Shaw, S. T. (2023) - Link to Publication | Link to Preprint

The present study assessed how perceptions of social status within the classroom—termed subjective social status—aligned with objective course performance and differed by sex, first-generation status, work status, and race/ethnicity among 713 students enrolled across three introductory statistics classes. When asked to explain how they evaluated their standing in the course, students reported five main themes, including both academic achievement with respect to exam scores and their understanding of course content. When examining differences by status-based identities in subjective social status, we found that female and first-generation students had lower subjective social status compared to their male and continuing-generation peers, although results were less robust for first-generation status. Likewise, working students reported lower subjective social status relative to non-working students, despite showing no difference in final exam score. Taken together, results suggest that factors beyond course performance may relate to students’ subjective social status, and subjective social status may contribute to disparities in academic performance, especially by sex and work status.

META SCIENCE RESEARCH

Investigating the Analytical Robustness of the Social and Behavioural Sciences

Aczel, B., Szaszi, B., Kósa, L., Torma, Z., Kumpel, H., Clelland, H. T., Hoffmann, S., Kovacs, M., Ahnström, L., Holzmeister, F., Nilsonne, G… Shaw, S. T., … Nosek, B. (2026) - Link to Publication

In this crowd initiative, we investigated the degree to which research findings in the social and behavioural sciences are contingent on analysts’ choices. We examined a stratified random sample of 100 studies published between 2009 and 2018, in which, for one claim per study, at least five reanalysts independently reanalysed the original data. The statistical appropriateness of the reanalyses was assessed in peer evaluations, and the robustness indicators were inspected along a range of research characteristics and study designs. We found that 34% of the independent reanalyses yielded the same result (within a tolerance region of ±0.05 Cohen’s d) as the original report; with a four times broader tolerance region, this indicator increased to 57%. Of the reanalyses conducted, 74% reached the same conclusion as the original investigation, 24% yielded no effects or inconclusive results and 2% reported the opposite effect.

How to Open Science: Promoting Principles and Reproducibility Practices Within The Educational Data Mining Community

Haim, A., Shaw, S. T., & Heffernan, N. T. (2023)  - Link to Publication

Across the past decade, open science has increased in momentum, making research more openly available and reproducible. Educational data mining, as a subfield of education technology, has been expanding in scope as well, developing and providing better understanding of large amount of data within education. However, open science and educational data mining do not often intersect, causing a bit of difficulty when trying to reuse methodologies, datasets, analyses for replication, reproduction, or an entirely separate end goal. In this tutorial, we will provide an overview of open science principles and their benefits and mitigation within research. In the second part of this tutorial, we will provide an example on using the Open Science Framework to make, collaborate, and share projects. The final part of this tutorial will go over some mitigation strategies when releasing datasets and materials such that other researchers may easily reproduce them. Participants in this tutorial will gain a better understanding of open science, how it is used, and how to apply it themselves.

How to Open Science: Debugging Reproducibility within the Educational Data Mining Conference

Haim, A., Gyurcsan, R., Baxter, C., Shaw, S. T., & Heffernan, N. T. (2023)  - Link to Publication

Despite increased efforts to assess the adoption rates of open science and robustness of reproducibility in sub-disciplines of education technology, there is a lack of understanding of why some research is not reproducible. Prior work has taken the first step toward assessing reproducibility of research, but has assumed certain constraints which hinder its discovery. Thus, the purpose of this study was to replicate previous work on papers within the proceedings of the "International Conference on Educational Data Mining" to accurately report on which papers are reproducible and why. Specifically, we examined 208 papers, attempted to reproduce them, documented reasons for reproducibility failures, and asked authors to provide additional information needed to reproduce their study. Our results showed that out of 12 papers that were potentially reproducible, only one successfully reproduced all analyses, and another two reproduced most of the analyses. The most common failure for reproducibility was failure to mention libraries needed, followed by non-seeded randomness. [For the complete proceedings, see ED630829. Additional funding for this paper was provided by the U.S. Department of Education's Graduate Assistance in Areas of National Need (GAANN).]

How to Open Science: A Principle and Reproducibility Review of the Learning Analytics and Knowledge Conference

Haim, A., Shaw, S. T., & Heffernan, N. T. (2023)  - Link to Publication

Within the field of education technology, learning analytics has increased in popularity over the past decade. Researchers conduct experiments and develop software, building on each other’s work to create more intricate systems. In parallel, open science — which describes a set of practices to make research more open, transparent, and reproducible — has exploded in recent years, resulting in more open data, code, and materials for researchers to use. However, without prior knowledge of open science, many researchers do not make their datasets, code, and materials openly available, and those that are available are often difficult, if not impossible, to reproduce. The purpose of the current study was to take a close look at our field by examining previous papers within the proceedings of the International Conference on Learning Analytics and Knowledge, and document the rate of open science adoption (e.g., preregistration, open data), as well as how well available data and code could be reproduced. Specifically, we examined 133 research papers, allowing ourselves 15 minutes for each paper to identify open science practices and attempt to reproduce the results according to their provided specifications. Our results showed that less than half of the research adopted standard open science principles, with approximately 5% fully meeting some of the defined principles. Further, we were unable to reproduce any of the papers successfully in the given time period. We conclude by providing recommendations on how to improve the reproducibility of our research as a field moving forward.

How to Open Science: Promoting Principles and Reproducibility Practices Within The Artificial Intelligence in Education Community

Haim, A., Shaw, S. T., & Heffernan, N. T. (2023) - Link to Publication

Across the past decade, open science has increased in momentum, making research more openly available and reproducible. Artificial Intelligence (AI), especially within education, has produced effective models to better predict student outcomes, generate content, and provide a greater number of observable features for teachers. While completed, generalized AI models take advantage of available open science practices, models used during the actual research process are not made available. In this tutorial, we will provide an overview of open science practices and their benefits and mitigation within AI education research. In the second part of this tutorial, we will use the Open Science Framework to make, collaborate, and share projects - demonstrating how to make materials, code, and data open. The final part of this tutorial will go over some mitigation strategies when releasing datasets and materials so other researchers may easily reproduce them. Participants in this tutorial will learn what the practices of open science are, how to use them in their own research, and how to use the Open Science Framework.