Indiana University SPEA Edward J. Bloustein School of Planning and Public Policy University of Pennsylvania AIR American University

Panel Paper: Bias in Online Classes: Evidence from a Field Experiment

Friday, November 13, 2015 : 1:50 PM
Tuttle Center (Hyatt Regency Miami)

*Names in bold indicate Presenter

Thomas Dee1, June Park John1, Rachel Baker2 and Brent Evans3, (1)Stanford University, (2)University of California, Irvine, (3)Vanderbilt University
Previous research has demonstrated that gender and racial discrimination exists in a variety of educational contexts, but little is known about discrimination in online classes. We explore this phenomenon in the discussion forums of Massive Open Online Courses (MOOCs). Analogous to face-to-face interactions in traditional classrooms, discussion forums are the primary setting for student-to-student and student-to-instructor interactions in MOOCs. The efficacy of this new educational medium depends upon engaged interactions in the discussion forums; hence identifying potential obstacles to those interactions, such as discrimination, is critical.

Forum participation is one important component of the small but growing literature on online classrooms. A small fraction of students enrolled in MOOCs participate in the discussion forums by posting comments, but there is evidence that many more students participate by viewing rather than posting comments (Breslow et al., 2013). Outside of the MOOC context, researchers have found aspects of online discussion positively associated with student perceptions and learning in online courses (Swan, 2002; Arbaugh & Benbunan-Fich, 2007). Participation in discussion forums is clearly an important component of online learning.

We address the following questions in our study:

  1. Is there evidence of bias in the rates of views, replies or comment points in MOOC discussion forums based on gender or race?
    1. Are there differential rates of views, replies or comment points by race and gender of commenter?
    2. Do instructors have differential rates of replies by race and gender of commenter?
  2. Is there evidence for in-group bias: are MOOC users more likely to respond to those who are more like them (same gender/same race)?

To answer these questions, we conducted a randomized experiment by posting MOOC forum comments using randomly assigned names. The comments we used come from a list of 32 generic discussion forum comments that are neither course- nor content-specific and were modified from actual forum comments. We randomly selected eight comments from the list to post in each course, and we spaced the timing of comments in regular intervals throughout the course. Comments were randomly paired with a name that evokes a particular race and gender combination. The racial categories are White, African American, Indian, and Chinese; in each course, one male and one female name from each racial category posted one of the eight comments. Applying this procedure across 126 MOOCs, we examined the number of views, responses, and comment votes each comment received to determine whether discrimination exists. We also captured the names of the respondents to our comments and estimate their gender and race to determine homophily (in-group bias) effects.

Results confirm the existence of instructor bias as they are more likely to respond to forum posts by ostensibly White Male students; however, we discover no evidence of racial or gender bias from MOOC participants as a whole. We also explore whether there is in-group bias, that is, whether MOOC users are more likely to respond to comments from usernames with the same race and gender as themselves.