• Support
• Contact Us

Research

All five components of READI are grounded in theoretical research and practice. The five components of READI are:

• Individual Attributes - motivation, procrastination, willingness to ask for help, etc.
• Learning Styles – based on the multiple intelligences model
• Technical Competency
• On-screen Reading Rate and Recall
• Typing Speed and Accuracy

The providers of READI encourage schools to do research with READI data regarding their own students. When schools plan to do an analysis of their READI data they often plan first to correlate READI scores to student’s grades in the course. This is a welcomed analysis and typically results in statistically significant findings. The 2008 study conducted by Atanda Research analyzed the READI scores of 2,622 random students representing over 300 schools. Correlations significant at the .05 level or higher were found with 11 of the 15 READI scores variables and student’s grades. However, this analysis is really not the most appropriate way to measure the validity of READI scores because student’s grades are impacted by a myriad of variables (prior academic experiences, IQ, etc.). READI is not designed to be an indicator of academic success. There are several tools such as the ACT, SAT, and GRE which serve this purpose. READI does not measure any constructs of content knowledge in areas such as math, science, history, etc. So to use READI solely as a predictor of academic success is not the most appropriate application.

In addition to correlating READI scores to grades, we recommend three other types of analysis which are a more valid measurement of the applicability of READI. (1) Identify students who dropped out of the online courses and compare the means of their READI scores to the means of the READI scores of the students who persisted in the courses. The intent of READI is to identify students who are “at-risk” of not being a good fit for distance learning and it is these students who are more likely to drop out. The real benefit of READI is when schools can identify “at-risk” students then provide the encouragement, remediation and support that the students need to remain in the course and be successful. (2) After students have completed their first online course, do a survey of the students asking them to report the goodness of fit of distance learning for them. Ask them questions about how they did keeping up with the volume of reading in the online course, the degree to which they could find time to participate in online course activities, the level of frustration they had with their computer and the Internet, etc. Then correlate their responses on these questions back to their READI scores. This type is study is very appropriate because READI is intended to be a predictor of goodness of fit of distance education. In the 2008 study conducted by Atanda Research, of the 90 correlations calculated between measures of goodness of fit and READI scores 63 of the 90 correlations were statistically significant at the .05 level or higher. (3) The third type of analysis that we encourage is a qualitative study in which you interview individually or in a focus group students who persisted in online courses and those who withdrew. Compare the factors that influenced their decision to remain or withdraw to the means of the READI scores from your students.

Not only does the provider of READI support additional analysis of READI data, but we will support you in the effort. If your school would like to construct a study like this contact Dr. Mac Adkins for assistance in designing the study, exporting the correct data, and conducting the statistical analysis.

Three major studies have been conducted by external research agencies to measure the reliability and validity of READI. Information about these studies is presented below.

Construct Validity

Construct validity refers to whether an assessment measures a theorized psychological construct. In the case of READI, construct validity is a measurement of the degree to which READI is an indicator of a learner’s level of readiness for studying online. Results from the two studies described below indicate that READI has strong construct validity in that it is an indicator of the goodness of fit for distance learning as is evidenced by multiple correlations that are statistically significant at the .01 level.

It should be noted that READI is not designed to be a predictor of academic success. There are a myriad of variables which impact academic success in online courses ranging from the student’s intelligence to the level of interactivity of the online faculty member. READI is an indicator of the degree to which online courses are a good fit for a student. READI does not make a value judgment indicating that a student should or should not take online courses. Rather it informs the student of their strengths and opportunities for growth in areas related to taking online courses. If a student is indicated to be deficient in a certain area and then if the school provides appropriate remediation and/or support, then READI can serve as a retention tool by helping students succeed as they learn in the context of online courses.

In 2007 an external research firm (Atanda Research, Alexandria, VA) was commissioned to analyze the data gathered during a study concerning the relationship of READI scores and measures of academic success and goodness of fit of distance education as a measure of construct validity. The major findings of this report were that there were forty-two statistically significant correlations between READI variables and measures of academic success and goodness of fit. Of the five constructs measured by READI, the construct with the most correlation to academic success and goodness of fit was Individual Attributes. The variable of the participant’s individual attributes scores were statistically significant at the .001 level with all measures of academic success and goodness of fit. The variable with the strongest correlation in the study was relationship between Grade Point Average and Reading Comprehension. Click here to view a copy of this report.

In 2008 the study conducted by Atanda Research was replicated as a part of a learner’s dissertation research which involved 2,622 students who had taken READI representing over 300 schools. This replication yielded even stronger results than the original study. Of the possible 105 correlations measured, 74 were found to be statistically significant. The factor measured by READI that had the strongest correlations to measures of goodness of fit and academic success was individual attributes which yielded correlations in each of the seven categories which were statistically significant at the .01 level. This finding mirrored the finding from the 2007 study which also indicated that individual attributes were the strongest indicator of goodness of fit of distance education.

The following correlation matrix presents the results of the statistical analysis from this study:

Correlations between student success variables (READI scores) and measures of goodness of fit for online learning and measures of academic success.

Item Reliability

In statistics, reliability is the consistency of a set of measurements used in an assessment. It is a measurement of whether the items of an instrument give or are likely to give the same measurement upon multiple attempts.

In 2008 Applied Measurement Associates of Tuscaloosa, Alabama was commissioned to conduct reliability coefficient calculations for the questions in READI. An expected range for Cronbach Alpha reliability coefficient values is expected to be from .70 to .95 to indicate a reliable assessment.

It should be noted that for the areas of READI which showed a lower reliability coefficient that the scale type was 0,1. This scale type resulted in lower levels of variability among the possible answers thus reducing the measurement of reliability.

One of the useful features of READI is that school leaders (faculty and/or administrators) can view READI scores through a dashboard which allows them to at-a-glance identify students who might be at risk of not doing well in an online course based on their READI scores. Then based on these findings the school can provide remediation and support as appropriate. This serves as a valuable student service which can increase the retention rates among online learners. Because the student population of each school is unique, one of the features of READI is that schools can set the grading thresholds to determine what level of READI scores should classify their students as “failed,” “questionable,” or “passed.” In July, 2008 an analysis was conducted based on the 108,423 students who had taken READI in the prior twelve months. Based on this analysis recommendations were made regarding the setting of the grading threshold values in the administrative dashboard of READI. Click Here to view a copy of this report.

This analysis revealed the following distributions of READI scores:

Learning Styles

The learning styles instrument embedded into READI is adapted from the larger (70 item) learning styles instrument administered by http://www.memletics.com. The instrument was reduced to 35 items in READI and some of the wording was modified to apply to online courses.

The Memletics learning styles instrument is grounded in the multiple intelligences research by Dr. Howard Gardner of Harvard University.

Read more about learning styles ...

Individual Attributes

The component of READI which measures individual attributes is based on the dissertation research of Dr. Julia Hartman. Dr. Hartman served as the Manager of the Alabama Online High School. In 2001 she received her Ph.D. from the University of Alabama in Instructional Leadership with emphasis in Instructional Technology (minors in Educational Research and Educational Computer Technology). Her dissertation was titled ATFY-R: Psychometric properties and predictive value for academic performance in online learning.

In her dissertation she identified the individual attributes which are significant predictors of success in an online learning environment. These are variables such as motivation, procrastination, time availability, and willingness to seek help. The individual attributes section of READI measures these variables which are indicators of success in an online course environment.

READI has partnered with the world's leading learning company, Pearson Education, to provide student success resources at a discounted rate. Through using these resources students can learn how to enhance their opportunities for success in higher education. These resources are available through the READI Student Success Bookstore.

On-Screen Reading Rate and Recall

The on-screen reading rate and recall section of READI was developed by the consultants of DECADE Consulting in cooperation with LiteracyWorks.org which is a project of the National Institute for Literacy and with ReadingSoft.com.

Both reading rate and recall are measured in READI because students should realize that they must not too rapidly read on-screen course content because they may be assessed on the content in their courses.

READI is used by secondary schools, technical colleges, community colleges, universities and corporations. To best fit the needs of the learners of each of these organizations, several reading passages are available. Institutions using READI may select per login group the reading passage that is most developmentally appropriate for that group of learners. The following passages are available:

The Flesch/Flesch–Kincaid Readability Tests are designed to indicate comprehension difficulty when reading a passage of contemporary academic English. The two tests are the Flesch–Kincaid Grade Level and the Flesch Reading Ease. Although they both use the same core measures (word length and sentence length), they have different weighting factors, so the results of the two tests correlate imperfectly: a text with a comparatively high score on the Reading Ease test may have a lower score on the Grade Level test. Both systems were devised by Rudolf Flesch.

The "Flesch–Kincaid Grade Level Formula" translates the 0–100 score to a U.S. grade level, making it easier for teachers, parents, librarians, and others to judge the readability level of various books and texts. It can also mean the number of years of education generally required to understand this text. The result is a number that corresponds with a grade level. For example, a score of 8.2 would indicate that the text is expected to be understandable by an average student in 8th grade (usually aged 13-14 in the U.S.).

In the Flesch Reading Ease test, higher scores indicate material that is easier to read; lower numbers mark passages that are more difficult to read. For comparison the Readibility Index of the Reader’s Digest is about 65, Time Magazine is about 52 and the Harvard Law Review is in the low 30s.

The degree to which the learner can recall the information in these passages is measured by ten questions. There are two of each of the following types of questions: sequence of events, factual, inferential, cloze, and main idea.

Participants are not allowed to view the reading passages while taking the quiz. As such READI provides an assessment of reading recall, not reading comprehension. The intention of this component of READI is to measure the degree to which a person can read academic information on-screen and then recall that information on a quiz. This is a task that is frequently replicated in online courses.

It should be noted that the reading rate and recall section of READI should not be used as an exhaustive reading skills inventory. Rather, it should be used as a screening device to identify learners who may be having difficulty recalling what they have read on-screen. If a learner is identified as having opportunities for growth in this area, the school can then inform the student about the resources for remediation and support which they provide. Communicating these resources can be automated through the feedback mechanisms of READI.

Technical Competency

The technical competency and typing components of READI was developed by Dr. Mac Adkins. Dr. Adkins holds an Ed.D. from Auburn University in Educational Leadership with an emphasis on instructional technology. Dr. Adkins was one of the authors of the Alabama Course of Study in Technology used by all public schools in Alabama. He was also a participating writer for the National Education Technology Standards (NETS) for Teachers document published by the International Society for Technology in Education. Dr. Adkins also teaches Administration and Leadership of Distance Learning Programs online for Capella University.

The premise of the technical competency section is that if students do not possess basic technical competencies, they will quickly become frustrated and may drop out of the online course. The tasks measured in the technical competency section are basic technology skills which a learner should possess to begin studying online.

Typing Speed and Accuracy

The formula for calculating typing speed and accuracy is based on the research of Teresia R. Ostrach, President, Five Star Staffing, Inc., Orlando, FL. With over twenty-seven years in the professional staffing industry, Teresia has expertise on calculating and comparing average typing speeds. In 1997 she conducted research on the average typing speeds of 3475 clerical workers. This research produced the deciles for average typing speeds which are used in READI. View the paper resulting from this research.

The formula used is to divide the number of words by the number of seconds and subtract for the number of errors.

READI Usage Patterns

Schools use READI in a variety of ways. A common model is that schools embed READI as an assignment into their orientation course. Some schools use it in the orientation course which is specific to online learners while other schools use it in the general orientation course which all students take. READI is a useful student service tool not only for students who will be taking fully online courses, but also hybrid courses, video conferencing courses and even face-to-face courses which use the Internet for communication in the course. Several schools make READI available to prospective students through their website. In May, 2009 schools which use READI were asked to describe how that READI is beneficial to their students and how they utilize READI. Click here to view this report.

2009 Online Student Readiness Report

eLearningToolBox, the provider of READI, annually analyzes the READI data in aggregate of all of the students from the prior year who have taken READI. No data specific to individual students or individual schools is made publically available. Data in the 2009 report was taken from 150,065 unique students from 272 higher education institutions who took the READI assessment from June 1, 2008 to May 31, 2009. Highlights in the report include the following statistically significant differences between the means of the variables of gender, ethnicity, institution type, age range and number of prior online courses taken as they relate to student readiness for online learning:

• Gender – Females scored higher on the constructs of individual attributes, learning styles, typing speed, typing accuracy, and reading recall. Males scored higher on technical competency and technical knowledge.
• Ethnicity – Caucasian/White reported higher means for learning styles, technical competency, and reading recall. African American scored the highest on individual attributes. Asian or Pacific Islander had the highest means for technical knowledge, typing speed and typing accuracy.
• Age Range – For constructs related to personal maturity, older students had the highest means. For constructs related to technical matters younger students had the highest means.
• Institution Type - Doctorate Granting Universities reporting the highest means for learning styles, technical competency, technical knowledge, and typing speed. Special Focus Institutions (schools preparing students for specific careers) reported the highest means for typing accuracy and reading recall. Baccalaureate Colleges reported the highest means for individual attributes.
• Number of Prior Online Courses – The results strongly demonstrated that with online learning experience matters. The greatest difference in means from students with no prior online course experience and those who had taken five or more courses was in the area of technical knowledge. This indicates that with experience students can learn to use the technology required for online courses.

A full copy of the report is available at http://www.readi.info/documents/2009_Online_Student_Readiness_Report.pdf.

Brief Review of Literature on the need for READI

With the shift toward online learning, it is important to explore the adoption of online education. Previous studies found that among academic leaders, 64 percent believe that it takes more discipline for a learner to succeed in an online course (Sloan Consortium, 2006); therefore, placing additional responsibility on students to be self-directed learners. Before the start of an online program or course, it should be determined if a learner’s instructional need can be resolved through a distance education approach (Willis & Lockee, 2004). Assessing the pre-requisite skills of the distance learner is critical (Hsiu-Mei & Liaw, 2004; Simonson et al., 2003). Learners need to have enough pre-requisite skills of technological proficiency and a strong motivation to learn by technology (Hsiu-Mei & Liaw, 2004). Because of the difficulty in accommodating a group of learners with a wide range of acquired skills, requirements for pre-requisite skills should be set (Falvo & Solloway, 2004). A researched method of examining the notion of online readiness is listed using three aspects: (a) Student’s preference for online form of instructional delivery as compared to traditional face to face instruction; (b) Student confidence in using electronic communication for learning and competence and confidence in the use of Internet and computer-mediated communication; and (c) Ability to engage in autonomous learning (P. J. Smith et al., 2003).

Pamela Dupin-Bryant of Utah State University – Toole conducted a study which was published in The American Journal of Distance Education titled “Pre-entry Variables Related to Retention in Online Distance Education.” This study identified pre-entry variables related to course completion and non-completion in university online distance education courses. Four hundred and sixty-four students who were enrolled in online distance education courses participated in the study. Discriminant analysis revealed six pre-entry variables were related to retention, including cumulative grade point average, class rank, number of previous courses completed online, searching the Internet training, operating systems and file management training, and Internet applications training. Results indicate prior educational experience and prior computer training may help distinguish between individuals who complete university online distance education courses and those who do not. READI measures all of the variables that this study indicated as indicators of success except for class rank.

Click here to view a paper presented at the 2009 Distance Learning Administration conference about the usage of READI.

Developmental Students

When developmental students enroll in distance classes, they bring with them the same need for support that they have in a conventional classroom (Caverly and MacDonald, 1998; Rhoda and Burns, 2005), and surprisingly little research has been done on how best to facilitate the progress of underprepared students in an online class (Perez and Foshay, 2002). Distance education requires more self-directed learning and higher levels of personal motivation, independence and self-discipline (Sampson, 2003), in addition to the technical skills required for participation in an online class (Caverly and MacDonald, 1998). These are all skills in which underprepared students making be lacking. Fortunately, the same technology that delivers the class can deliver the support systems.

Additional Research Requests

Additional research on READI is welcomed. If you are interested in conducting research on the topic of online student readiness using READI data please send a brief research request to Dr. Mac Adkins (mac@eLearningToolBox.com). In the research request describe the purpose and plan for your research including the proposed subjects, timeline, and plans for the dissemination of the research. All research done using READI data must meet our privacy statement. We never release to third parties any data which identifies individual or other school specific data.

Reference List

• Dupin-Bryant, P. A. (2004). Pre-entry variables related to retention in online distance education. American Journal of Distance Education, 18(4), 199-206.
• Caverly, D., and MacDonald, L. (1998). Techtalk: Distance developmental education. Journal of Developmental Education, 2. Retrieved October 12, 2007 from Academic Search Premier
• Falvo, D. A., & Solloway, S. (2004). Constructing community in a graduate course about teaching with technology. TechTrends: Linking Research & Practice to Improve Learning, 48(5), 56.
• Hsiu-Mei, H., & Liaw, S.-S. (2004). Guiding distance educators in building web-based instructions. International Journal of Instructional Media, 31(2), 125.
• Perez, S., & Foshay, R. (2002). Adding up the distance: Can developmental studies work in a distance learning environment? T H E Journal, 29, pp. 16+. Retrieved May 22, 2007 from Questia.
• Rhoda, K. R. & Burns, C. N. (2005). Developing and online writing center for distance learning courses. Paper presented at 21st Annual Conference on Distance Learning and Teaching. Retrieved October 13, 2007 from http://www.uwex.edu/disted/conference/Resource_library/proceedings/05_1923.pdf
• Sampson, N. (2003). Meeting the needs of distance learners. Language, Learning and Technology, 7, pp.103+. Retrieved June 13, 2007, from Questia.
• Simonson, M., Smaldino, S., Albright, M., & Zvacek, S. (2003). Teaching and learning at a distance. Upper Saddle River, NJ: Pearson Education, Inc.
• Smith, P. J., Murphy, K. L., & Mahoney, S. E. (2003). Towards identifying factors underlying readiness for online learning: An exploratory study. Distance Education, 24(1), 57. United States Distance Learning
• Association, (2004). Retrieved March 10, 2004 from http://www.usdla.org
• Willis, L. L., & Lockee, B. B. (2004). A pragmatic instructional design model for distance learning. International Journal of Instructional Media, 31(1), 9.