Association for Public Policy Analysis & Management

Background: Executive function (EF) skills have been linked to educational outcomes, including academic skills, school engagement, and self-regulated classroom behaviors. However, the conventional approach to EF assessment is to measure children's performance on standard EF tasks in a highly controlled, laboratory-like setting, typically with a ratio of one child to one assessor. This approach lacks the ecological validity of assessment in a classroom setting and does not scale well for collecting data from a large number of students. We developed a new procedure to simultaneously assess EF skills in all students in a classroom using standard EF tasks administered on tablet computers.

Purpose: We validate this new group assessment (GA) procedure by: (1) examining convergent validity with conventional individual assessment (IA) procedures; and (2) investigating the predictive validity of EF skills assessed in two settings.

Method: Socioeconomically and ethnically diverse students from 33 third, fourth, and fifth-grade classrooms (N = 269; 52% female) participated in GA and IA of EFs. We developed a classroom procedure that enabled three research assistants (RAs) to administer GA using tablet computers. EFs were assessed using: (1) Hearts and Flowers task (H&F) Digit Span Backwards (DSB), and (3) Multi-Source Interference Test (MSIT). Percent accuracy for the H&F incongruent and mixed blocks, the DSB, and the MSIT incongruent block were composited (IA α = .64; GA α = .62). Classroom behavior was assessed using the Teacher-Child Rating Scale (TCRS; Hightower et al., 1986) of task orientation and frustration tolerance. State-administered standardized test scores were used to measure English/language arts (ELA) and mathematics skills one year before and year after EF assessments.

Results: The IA and GA accuracy composites were strongly positively correlated (r = 0.59, p < 0.001). Using multi-level modeling, we found that both IA accuracy and GA accuracy composites were independently predictive of teachers' reports of self-regulated classroom behavior (bs = 0.23 and 0.51, p = 0.025 and p < 0.001), as well as ELA (bs = 27.97 and 41.14, ps < 0.001) and math achievement (bs = 30.70 and 34.96, ps < 0.001). However, only GA was significantly related to ELA (b = 19.64, p = 0.007) and math (b = 22.73, p = 0.004), after controlling for prior achievement. In all of these models, we accounted for the nesting of children within classrooms and included school fixed effects, child age, gender, and ethnicity, and parent education as covariates.

Discussion: Our findings demonstrated that EF skills measured in both contexts are relevant for students' ability to control their attention, behaviors, and emotions in the classroom, as well as for their performance on standardized academic tests. However, only EF performance in a classroom environment emerged as a significant predictor of changes in students' achievement test scores, empirically corroborating the notion that improved ecological validity of classroom assessment may also improve predictive validity of directly assessed EF skills for school success. The new GA protocol represents a pragmatic, cost-effective way to obtain direct assessment of children’s EF skills at scale.