Impact of the Thinking Reader Software Program on Grade 6 Reading Vocabulary, Comprehension, Strategies and Motivation (TRSP)

Improving adolescent literacy is a critical step toward improving adolescent academic achievement (Kamil , Borman, Dole, Kral, Salinger, & Torgesen, 2008). “Adolescent literacy” commonly refers to the skills that students in Grades 4–12 need in order to successfully learn by reading, as opposed to learning how to read, which is emphasized in earlier grades (Kamil, 2003; Kamil et al., 2008; National Governors Association, 2005). Recent policy reports emphasize the need to build students’ reading vocabulary and comprehension skills to meet the increased literacy demands that begin in Grade 4 (Carnegie Council on Advancing Adolescent Literacy, 2010; Meltzer, Smith, & Clark, 2001). Experts who drafted the Common Core State Standards for English Language Arts have emphasized that students must show a steadily increasing ability to discern more from text to become successful readers (National Governors Association & Council of Chief State School Officers, 2010). The current study evaluates an intervention (Thinking Reader®) designed to improve middle school students’ reading vocabulary and comprehension (Tom Snyder Productions, 2006a). It responds to an interest expressed by stakeholders to the Regional Educational Laboratory Northeast and Islands in improving literacy outcomes for students beyond elementary school. Thinking Reader is a software program for students in Grades 5–8 that incorporates elements commonly identified in policy reports as being key components of effective adolescent literacy instruction. These reports prioritize elements such as instruction in comprehension strategies (Biancarosa & Snow, 2004; Carnegie Council on Advancing Adolescent Literacy, 2010; Meltzer et al., 2001); attention to motivation and self-directed learning (Biancarosa & Snow, 2004; Meltzer et al., 2001; National Council of Teachers of English, 2006); ongoing formative assessment (Biancarosa & Snow, 2004; Carnegie Council on Advancing Adolescent Literacy, 2010; National Council of Teachers of English, 2006); and inclusion of technology as an instructional tool (Biancarosa & Snow, 2004; Kamil, 2003; National Council of Teachers of English, 2006). When using Thinking Reader, students read novels on computers and respond to prompts. The software aims to teach students to use comprehension strategies through a reciprocal teaching approach in which the strategies are taught while the teacher and students explore the meaning of text (Palincsar & Brown, 1984). Students’ progress is assessed regularly. Thinking Reader also aims to motivate students to read and to make self-directed use of strategies. The software has a limited but positive evidence base, including statistically significant findings of one quasi experimental study (Dalton, Pisha, Eagleton, Coyne, & Deysher, 2002) and empirical evidence supporting instruction on comprehension strategy use (for example, RAND Reading Study Group, 2002), particularly on strategy instruction through reciprocal teaching approaches (for example, Rosenshine & Meister, 1994). The current study is the first rigorous, randomized controlled trial on the program. Research Questions and Measures This evaluation of the impact of Thinking Reader use by Grade 6 students focused on two confirmatory research questions about the effect of the program on two measures of students’ reading achievement: 1 1. What is the effect of Thinking Reader on students’ reading vocabulary? 2. What is the effect of Thinking Reader on students’ reading comprehension? A statistically significant impact on either outcome measure would signal the program’s success. The study also examined whether Thinking Reader has an effect on two ancillary, but important, measures of students’ approaches to reading: 1. What is the effect of Thinking Reader on students’ use of reading comprehension strategies? 2. What is the effect of Thinking Reader on students’ motivation to read? The answers to these questions provide information that may be useful to educators who see these factors as being important precursors or supplements to improved achievement. But without a direct, measurable effect on reading achievement itself (vocabulary, comprehension), such effects would be insufficient to judge the program’s effectiveness. The vocabulary and reading comprehension subtests of the Gates-MacGinitie Reading Tests (GMRT) (MacGinitie, MacGinitie, Maria, Dreyer, & Hughes, 1999) served as the achievement measures for the primary research questions. Two self-report student surveys—the Metacognitive Awareness of Reading Strategies Inventory (MARSI; Mokhtari & Reichard, 2002) and the Motivation for Reading Questionnaire (MRQ; Wigfield & Guthrie, 1997)—served as the measures for the ancillary research questions. Each was collected as both a pretest fall baseline measure before the start of the intervention and as a posttest spring outcome measure. This study also addressed four exploratory research questions. These questions investigate whether the impact of the Thinking Reader intervention on students’ reading achievement varied across subgroups of students formed on the basis of baseline reading vocabulary, baseline reading comprehension, and baseline motivation to read measures: 1. Does the effect of Thinking Reader on students’ reading vocabulary vary according to their baseline reading vocabulary scores? 2. Does the effect of Thinking Reader on students’ reading comprehension vary according to their baseline reading comprehension scores? 3. Does the effect of Thinking Reader on students’ reading vocabulary vary according to their baseline reading motivation scores? 4. Does the effect of Thinking Reader on students’ reading comprehension vary according to their baseline reading motivation scores? The outcomes of interest for these exploratory research questions are the vocabulary and comprehension subtests of the GMRT that served as the achievement measures for the primary research questions. The study collected student, teacher, and school data as covariates for the analyses and collected data from classroom observations and electronic report data from Thinking Reader to help understand program implementation. Student data included age, gender, and ethnicity, as well as English language learner and special education status. Teacher data included information about years of teaching experience, educational attainment, and certifications or endorsements held. School data included type of school (elementary or middle school), state, enrollment size, and the poverty level and ethnicity of students. For a subset of classrooms, structured observations in the winter and spring in one intervention and one control classroom at each school provided descriptive information about instruction. To examine fidelity of implementation, electronic report data were collected from the Thinking Reader program at the end of the year. These data provided information about students’ exposure to the software (such as number of books started and completed, total number of minutes using the software, and number of weeks spent on each book), as well as students’ program levels.