Ron Cole, Barb Wise, Sarel van Vuuren, Nattawut Ngampatipatpong, Jariya Tuantranont,
Gordon Golding, Lynn Snyder, Taylor Streumph, Pam Cole, Tim Weston, Donna Caccamise
Previous Project Team Members:
Isidoros Doxas, Bill Gaven, Scott Schwartz, Linda Corson, Dave Wade-Stein
FtL is a comprehensive computer-based reading program that has been designed to teach beginning and early readers to read accurately, fluently and with good comprehension. It provides the pedagogical foundation for effective learning experiences for ELL students through the following features:
a) Marni, an engaging, lifelike animated character, instructs and supports the student through a sequence of learning activities. Marni produces natural (recorded) speech accompanied by accurate lip movements, head movements and facial expressions. Marni is helpful and supportive-providing instruction, hints, feedback and encouragement. Student interviews (see below) reveal that students “bond” with Marni and feel that she behaves like a real teacher.
b) The program proceeds through a set of learning exercises that build on each other, and the sequencing of learning tasks are based on scientific research. The program teaches foundational reading skills until accuracy is assured, and then presents speeded tasks so these skills become automatic and unconscious.
c) The program emphasizes phonological awareness and decoding skills, which research shows are just those skills that require extra intensive practice for many children with reading challenges.
d) The program’s study plan continually modifies pacing, presentation of material, and progress through the program based on the student’s performance on the learning tasks. Thus, students who learn quickly can move rapidly through the program, with periodic “reviews” to assure that skills have been retained and generalize to new stimuli, while students who have difficulty learning will receive continued practice on a focused set of skills within a variety of complementary learning exercises and books. The program continuously adjusts the pacing of activities and the number of response alternatives provided based on the student’s performance.
e) Children spend significant time listening to and reading interesting books that are aligned with the learning exercises in terms of both vocabulary and content, so that skills learned in the exercises are practiced and reinforced in books.
FtL incorporates the key principles of scientifically-based reading research. The Report of the National Reading Panel (NRP, 2000) and a comprehensive review article (Rayner et al., 2001) summarize the implications of scientific reading research for instruction. This research suggests that balanced reading instruction covers five domains of reading with sequenced instruction that is intensive, explicit, structured and direct. The domains include phonological awareness, phonics (alphabet, decoding, and spelling), fluency (automaticity and reading with natural expression), vocabulary, and comprehension. Most reading professionals agree that these domains are important in balanced reading instruction, and Foundations to Literacy is designed to cover these five domains explicitly, systematically and intensively in carefully sequenced ways.
All FtL exercises aim for simplicity, engagement, and empowerment of success. The experience of success is encouraged by presenting the sequenced items in a “scaffolded” manner (Vygotsky, 1978), ensuring that children start with “comfort level” items from a previously successful level, proceed to supported instructional levels, and end again in “comfort level.” This scaffolding is done seamlessly by the program, as it would be by an expert teacher. We also encourage a full cycle of learning from discovery and practice to competence. After students attain competence with a skill or concept, the program assigns speeded practice with it to get it automatic, applies it in reading in context, and places students in books designed to transfer these skills to independent reading and writing.
Managed Learning Environment (MLE): The MLE controls each student’s individual study plan, administers retention tests periodically within the program, adapts the student’s study plan automatically based on his or her performance in learning exercises (e.g., by reducing the number of response choices, or moving the student to a lower level in the program), and presents the student with rewards and progress graphics. The MLE also provides an interactive interface for educators to enrol students within the program, to examine progress within the program in various ways (including replaying individual sessions), and to measure progress for individuals or groups in terms of district or state learning goals. In the context of the proposed work, the MLE will be used to enrol students in the program, to construct different study plans for the different computer-based intervention conditions describe below, and to log, organize, analyze and display student learning data in the different treatment conditions.
After students are placed within the MLE based on estimates of their current reading level, the program adapts to the student’s performance to optimize learning. The program can focus the student on critical skills while challenging the student to maintain an optimal learning pace. Based on the adaptive study plan, students can move at their own pace through a set of exercises to more advanced exercises and books, or be kept at the same level, or taken to lower levels for remediation and practice on specific core competencies. An important feature of the program is that it engages and empowers students by providing them with choices about what to do next. When the student completes an exercise, he or she returns to the virtual homeroom, where he or she can then choose to do another exercise, or select a new book to read, or go to their library to reread a book. The homeroom is also where the student is rewarded for completing exercises and books (via an animated progress graphic) so the student is able to understand what level they are in within the program, how much they have accomplished, and how much work is required to achieve the next milestones.
Foundational Reading Skills Exercises: Foundational Reading Skills Exercises are designed to teach core competencies needed to recognize words and read fluently. Exercise domains include alphabet knowledge, phonological awareness, letter-to-sound decoding and spelling regular words and reading and spelling common sight words. Each foundational reading skills exercise was designed by a team of researchers at CSLR using science-based research principles based on recommendations from the National Reading Panel, and participatory design methodology-which brings together researchers, programmers, teachers and students to design, test and modify the software from the initial stages of development. Each foundational reading skill is taught in 3 to 5 different learning games that complement and reinforce each other.
Interactive Books: Interactive books teach students to read and comprehend text. Interactive Books are designed by our project staff using a powerful set of tools (the Interactive Books Authoring Environment) that enables us to design books with different functions appropriate for readers at different levels. In the proposed work, Interactive Books will provide the learning environment for training fluent and expressive reading. New functions will be created and existing functions enhanced for this purpose. Current capabilities of Interactive Books include:
The high levels of engagement and positive experiences reported by students and teachers who used the FtL program during the 2004-2005 and 2005-2006 school years, reported below, are caused in part by the seamless integration (Van Vuuren, in press; Cole et al., 2003) of computer speech recognition and character animation technologies that make Marni believable and likeable. The 3-D character animation technologies in the CU Animate system generate accurate movements of the lips, tongue and jaw synchronized with recorded speech and natural head and face movements. The speech recognition technologies developed at CSLR produce accurate phonetic segmentation and recognition of words in speech. Together, these technologies enable conversational interaction with a virtual tutor and real time feedback to students when they read out loud.
SONIC Speech Recognizer: To track students reading out loud we use a novel speech recognition technology developed at the University of Colorado known as SONIC . SONIC has been benchmarked on a number of comparative databases within the speech recognition community and has been shown to provide efficient, accurate, and state-of-the-art performance, including recognition of children’s speech. SONIC has been ported to over 15 different languages and children’s speech recognition has been enabled in 4 languages (English, Spanish, Italian, and Finnish). SONIC is currently integrated within the FtL platform and enables real-time speech-based interaction with children. During typical reading aloud in context by children in grades 2 through 5, SONIC obtains 93% word accuracy and has the capability to automatically tune to a particular child’s voice across multiple sessions of interaction.
3-D Character Animation: The Virtual Tutor, Marni, “lives” within Interactive Books and Foundational Reading Skills Exercises. During these tasks, Marni gives hints and explanations to help children to figure out answers while learning about phonological awareness, reading, spelling, and comprehension. In the learning activities, Marni pronounces words distinctly, with accurate movements of the lips, tongue, and jaw. Rather than using synthetic speech, which would be easier and faster, we use recorded speech, synchronized automatically with the movements of the visible articulators and natural facial expressions and head movements, to foster social agency. By using a recorded voice, we produce a more natural and effective user experience that benefits from the remarkable range and communication abilities of the human voice. When the character reads a story or interacts with the child, she does so with the appropriate prosody and emotional content in all contexts. In Interactive Books, we record all sentences, prompts, questions, and reading supports produced by the agent. We also record each word pronounced individually in isolation, so the Tutor pronounces it clearly and distinctly when the student clicks on the word. To students, Marni is a believable and credible teacher because she produces natural and expressive speech with appropriate facial expression using the CU Animate system.
Summative evaluation of FtL during 2003-2004 school years produced significant learning gains for letter and word recognition for all measures for Kindergarten students (elision, letter and word recognition, see Wise et al, 2004). Results for the 2004-2005 school years found significant effects for two measures across the three main school districts for kindergarten, first grade and second grade students included in the analysis. The comparison for the WJ-III Letter-Word ID Standard Score was significant at F = 20.6, df (error) = 718 and p < .001. The effect size for this measure equaled d = .34. Large gains were found for Kindergarteners on this variable with an effect size of d= .85. The standardized WJ-III Letter ID measure also showed significant differences between treatment groups (F = 19.8, df = 190, p < .001). The effect size for this measure was d = .62.
Student Experiences with FtL: A student survey was administered to 129 K-2 children who used FtL in 10 schools during the 2004-2005 school year. A revised and expanded survey was administered to 239 K-2 students who used FtL in 7 schools during the 2005-2006 school year. The surveys were administered individually to each student by a member of the FtL project staff. Questions were designed to provide insights about students’ opinions about the program, and to gauge students’ understanding of specific program features. All questions except the first one, “What did you think about the program?”, could be answered by selecting among response choices that were read aloud to the student, or by selecting among different pictures. Opinions were indicated using a 3-point Likert scale. For example, choices to “How much do you trust Marni” were “I trust Marni very much,” “I trust Marni some of the time,” and “I don’t trust Marni.” The survey questions and histograms of student responses to each question can be viewed here.
The survey results demonstrated that students were able to produce informative and discriminative responses to the program, with significant variations in responses to the different questions by individual students. Overall, students were highly enthusiastic about the program and their interactions with Marni. About 75% of all responses fell into the highest approval category. When given three choices, with the first being positive, the second neutral and the third being negative, students reported that Marni is smart (176/47/16; smart / smart sometimes / not smart), is a good teacher (158/74/8), acts like a real teacher (156/55/29), helped them learn how to read (150/66/24), that they trusted Marni (151/70/18) and believe she cares about them (156/40/42).
Teachers’ Impressions of FtL: Thirty six teachers (of the 38 total whose students used FtL) provided written responses to a questionnaire that posed 17 questions about their experiences with and opinions of the program. The written responses were coded for analysis; survey questions and histograms of the coded responses can be viewed here. In general, teachers were positive about the program. Of the 36 teachers who responded to the question “What did you think about the program?” 1 did not like it, 12 said it was OK, 11 said they liked it when it worked well (referring to intermittent problems that caused the program to freeze), and 12 said they loved it. All teachers reported that the program taught skills they wanted their students to learn, and that the program complemented their classroom activities. All but 2 teachers said they would like to use the program next year, and all but one said they would recommend it to other teachers.
FtL has been featured in over forty presentations at conferences and workshops and in invited talks in the U.S., Europe and China. Recent talks by R. Cole are listed here.
Articles that incorporate summaries of FtL include:
The Foundations to Literacy (FtL) project was initiated in 2000 when CU was awarded a 5 year National Science Foundation (NSF) Information Technology Research (ITR) grant with Ron Cole as the Principal Investigator (PI). The project was subsequently supported by a second NSF ITR grant (Walter Kintsch, PI) an Interagency Education Research Initiative (IERI) grant from the NSF (Walter Kintsch, PI) and an IERI grant from the National Institute of Health’s National Institute for Child Health and Development (Ron Cole PI). Additional support to demonstrate the feasibility of extending FtL to students with mild to moderate cognitive disabilities (e.g., Autism Spectrum Disorder or Mental Retardation) was provided by the Coleman Institute for Cognitive Disabilities. Recently, an NIH Center grant was awarded to CU (Richard Olsen, PI), with a project component dedicated to using FtL as a means for studying Response to Intervention (Barb Wise, project PI). Foundations to Literacy is a component of the Colorado Literacy Tutor, a collaboration between universities and public schools that aims to improve student achievement through development of educational software that helps students learn to read and comprehend text. The research grants that have or continue to support FtL and the Colorado Literacy Tutor projects include:
The following research grants provide direct support for the CSLR Reading Project:
NIH: 1542620- Center Director, Olson, R. “Differential Diagnosis of Reading Disabilities,” $7,415,061 12/01/05-11/30/2010
Project V PI: Wise, B.; Van Vuuren, S.; Cole, R., Byrne, B. “Response to Computer-Assisted Instruction for Reading Difficulties,” $1,327,284, NIH, 4/01/06 – 11/30/2010
NSF/ITR: REC-0115419 – Kintsch, W., Landauer T., Caccamise, D., Cole, R., “ITR/PE: Latent Semantic Analysis Theory and Technology,” $2,400,000, NSF, 09/01/01 – 08/31/06.
NSF/IERI: EIA-0121201 – Kintsch, W., Caccamise, D., Cole, R., Olson, R., Snyder, L., “IERI: Scalable and Sustainable Technologies for Reading Instruction and Assessment,” $5,997,404, NSF, 07/01/01- 06/26/06.
NSF/ITR: IIS-0086107 – Cole, R., van Santen, J., Movellan, J., “ITR: Creating the Next Generation of Intelligent Animated Conversational Agents,” $4,000,000, NSF, 09/01/00 – 08/31/05.
NSF/IERI : 1R01HD-44276.01Cole, R., Barker, L., Schwartz S., Snyder, L., Wise, B., “IERI: Scaling up Reading Tutors,” $1,000,000.00, NIH. 9/27/02 – 9/30/04.
Coleman Institute for Cognitive Disabilities: Schwartz, S., Cole, R., Wise, B., Doxas, I., “Coleman Foundation Grant: Participatory Design for Creating Computer Based Learning Tools,” $8184, Coleman.
NSF ITR Supplement Research Experience for Undergrads: REU/ITR: Creating the Next Generation of Intelligent Animated Conversational Agents (supplement to NSF 0086107), $45,500.00, NSF, 06/01/02 – 8/31/03.
NSF ITR Supplement Research Experience for Teachers: RET/ITR: Creating the Next Generation of Intelligent Animated Conversational Agents (supplement to NSF 0086107) $40,000.00, NSF, 09/01/02 – 08/31/03.
FtL owes its existence to the support and patience of the administrators, principals, teachers and technical staff who believed in the program and showed remarkable patience as worked each year to make it stable enough to support daily classroom use. Special thanks to Jean Riordan and Judy Skupa in the Boulder Valley School district, who have steadfastly supported our efforts to improve and assess FtL in Boulder Valley schools, and to Sister Elizabeth Youngs, Assistant Superintendent of the Denver Archdiocese schools. These administrators helped identify schools and recruit principals and teachers who used the program. And special thanks to Dr. Donna Caccamise for her tireless efforts in administering the FtL project during the past five years. The most gratifying outcome of their efforts is that principals and teachers are now demanding that we show up with the program at the beginning of each new school year, since they are convinced that it benefits their students.