During year two of the study, grade 2 students improved from 45% of students performing below grade level in word-reading in Fall 1999 to 97% at or above grade level in word-reading in Spring 2000. By the spring assessment, 94% of the students were reading 1-5 years above grade level. These results demonstrate continued effect of the program despite students entering at higher achievement levels due to their participation in the first year of the program.

Grade 3 students went from an average 20% below grade level in word-reading in Fall 1999 to 85% at or above grade level in word-reading in Spring 2000, with 80% 1-4 years above grade level.

Grade 4 students went from 34% below grade level in word-reading in Fall 1999 to 94% at or above grade level in word-reading in Spring 2000, with 80% 1-5 years above grade level.

Grade 5 students had 23% below grade level in the pre-test in word-reading in fall 1999; and 96% of the students at or above grade level in the post-test in word-reading in Spring 2000; with 85% 1-6 years above grade level.

2. Farragut Elementary School went from 61% below grade level to only 15% below grade level in 9 months. Reading level growth rose from 39% at or above grade level to 85% at or above grade level in 9 months. Reading level growth rose from 7% at 1-7 years above grade level to 70% in 9 months. Reading level growth rose from 1% at 2–7 years above grade level to 43% in 9 months.

3. Results from Manteno School District, September 2000 – May 2004. Measures based on improvements in reading levels. Results positively correlated with gains in nationally norm-referenced tests.
Grade 1          98% raised 1-8 grade levels
Grade 2          95% raised 1-7 grade levels
Grade 3          99% raised 1-7 grade levels
Grade 4          95% raised 1-8 grade levels
Grade 5          89% raised 1-6 grade levels
Grade 9          99% rose 1-2 grade levels
Grade 10         100% rose 1-2 grade levels
Grade 11         99% rose 1-2 grade levels
Grade 12         100% are at or above grade level

4. Results from Lincoln Middle School, Chicago, Illinois. 1997-1998.

Grade 6: In Fall 1997, 69% of 6^th grade students were below grade level in reading before using Keys to Reading Success™ program. (Thirty-six percent began the year 2 or more years below reading level.) This 69% below reading level corresponded to the state reading assessment which showed 2/3 of the students in grade 6 were below state standards in reading before use of Keys to Reading Success™.  In the Spring of 1998, after using the program for 8 months, 2/3 of the students in grade 6 exceeded state standards in reading.

Grade 8: In Fall 1997, 70% of the students in 8^th grade were reading below grade level, with 47% two or more years below grade level. This corresponded to the state reading assessment which showed 2/3 of the students in grade 8 were below state standards in reading before use of the reading program. In the Spring of 1998, after using the program for 8 months, 2/3 of the students in grade 8 exceeded state standards in reading.

Proven Validity and Reliability -- Keys to Reading Successä Technical Report -
Michigan State University, August 2002, Steven G. Viger and Edward W. Wolfe
From the Executive Summary:

“Keys to Reading Successä is a reading program that provides for screening, diagnostic and formative assessment as well as instruction and reporting in Phonics and Comprehension skills. Assessment data from 1,658 students, grades K-12, were collected on several measures. The census code designates the subject district is on the urban fringe of a large mid-western city. The data of primary interest were collected using the Keys to Reading Successä comprehension and word reading tests. Word reading accuracy is assessed in both the phonics and passage reading portions of the tests. This technical report uses the data collected to evaluate the reliability of the Keys to Reading Success scoring system. Additionally, validity evidence was evaluated for the Keys to Reading Success instruments by way of relating the Keys to Reading Success scores to student level data obtained using other measures beyond the Keys to Reading Success assessments. These measures include the California Test of Basic Skills (CTBS) comprehension and total scores and the ACT.

Test-retest reliability estimates were computed using Cronbach’s alpha coefficient, a standard in the field of instrument evaluation. Cronbach’s alpha essentially evaluates the degree to which participants are rank ordered by the instrument across differing data collection periods. In order to accomplish that task, data must be available for a given instrument for at least two data collection instances. In the evaluated data set, such information was available for both Keys to Reading Success assessment types as well as the CTBS. In practice, it is desirable to have coefficients of at least .80. The evaluated data demonstrated coefficients above .90 for both Keys to Reading Success assessments. With the exception of the CTBS comprehension scores, coefficients for the other CTBS measures exceeded .90 with the CTBS comprehension evaluations demonstrating reliabilities of .87 and .88 for grade equivalent and normal curve equivalent respectively.

Evidence for concurrent validity, evidence that the Keys to Reading Success assessments are correlated with tests measuring the same or similar constructs collected within the same time frame, was gathered for both the word reading and comprehension assessment of the Keys to Reading Success evaluations. Due to the high reliability of the aforementioned instruments, averages per student were used to compute the correlation coefficients. Correlation coefficients range from -1.00 to +1.00 and in this case suggest the amount of linear relatedness between the correlated measures. A negative coefficient in general suggests that as scores on one measure increase, scores on the measure it is being correlated with tend to decrease. The same type of logic suggests that a positive correlation is present in relationships where both measures tend to increase together. Since the point of establishing concurrent validity is to demonstrate that each instrument is measuring approximately the same construct, positive correlations are desirable. Additionally, coefficients greater than .40 suggest moderate to strong linear relatedness.

For the Keys to Reading Success word reading assessment, concurrent validity evidence was found to exist. Average word reading scores for the sampled students were related to ACT scores (r = .47), CTBS comprehension scores (r = .55), and CTBS total scores (r = .58). These relationships suggest that the Keys to Reading Success word reading test has a positive linear relationship with the aforementioned measures. Specifically, as word reading scores increased, the other measures tended to increase as well.

A similar, yet slightly stronger and more convincing pattern was observed with respect to the Keys to Reading Success comprehension scores. The CTBS total score was most strongly related to the Keys to Reading Success comprehension scores (r = .60), followed by the CTBS comprehension scores (r = .53). Again, these observed coefficients suggest positive linear relationships.

Finally, predictive validity evidence was found by correlating the Keys to Reading Success comprehension and recognition data collected in the fall of 2000 with the ACT score obtained by those individuals in the spring of 2001. Coefficients were .43 and .42 for the comprehension and recognition scores respectively. This evidence suggests that information on Keys to Reading Success evaluations is useful in predicting performance on future assessments such as the ACT.

Overall, the evidence suggests that scores generated by the Keys to Reading Success assessments are reliable as evidenced by test-retest estimates. The scores from the assessments are also valid. Validity evidence was noted by concurrent and predictive types of evidence.

How can I learn more? You can arrange to have an on-line demo from any computer with an Internet connection. Without leaving your school or home, see how this time saving, cost-effective, and proven program can help your students succeed.

Keys Learning: (630) 717-4221 or email: info@keyslearning.com