“Students performed 20 percent better in the hybrid version of this course compared to the face-to-face sections taught by other instructors.” When I heard this statement during a presentation at the Educause Learning Initiative Annual Meeting in February, I did something I rarely do: I closed my laptop, looked straight at the presenter, and stopped multitasking for a full twenty minutes.
I find most educational-technology conferences are a lot like an episode of the X-Files with a cast made up entirely of Fox Mulders. Everyone wants to believe. There are a lot of technology cheerleaders and a lot of iPad sightings, and no one seems to notice that Dana Scully—the skeptical, pragmatic agent designed to bring Mulder back down to Earth—has gone missing. So when someone offers up a bold promise backed by actual bar graphs, I take notice.
The presenter, Professor T. Warren Hardy from the University of Maryland–Baltimore County (UMBC), stated that his students performed significantly better on their final exam largely due to his use of online self-assessments. Upon hearing this, I immediately put on my Agent Scully trench coat and asked myself why his conclusions could be off.
- Was his final exam easier than the one used in other sections? No, all sections take the same final exam.
- Did he give his students an unfair advantage by using final exam questions in his self-assessments? No, the final exam is designed by other members of the department who are not currently teaching the course. To ensure a level playing field, the instructors have no knowledge of the specific questions that will appear on the final exam.
- What if he’s just a better instructor than the faculty teaching the other sections? That might hold water if it wasn’t for the fact that Professor Hardy’s students scored considerably higher than his own past students after he converted the course to a hybrid format with online self-assessments.
Of course, I’m sure there are other variables that might impact the validity of Professor Hardy’s findings. Yet, after hearing the unique steps that UMBC’s economics department takes to ensure a rigorous and standardized final exam for the five-hundred students who take ECON 122 every year, I felt the 20 percent difference on Hardy’s final exam scores were hard to dismiss.
In addition to praising his students’ performance, Hardy’s co-presenters from UMBC noted that his course was a regular in the University’s list of most-active Blackboard courses. Hardy attributed his students’ extensive and frequent use of Blackboard largely to his course’s reliance on adaptive release. Adaptive release refers to a set of restrictions that can require students to view and interact with certain online content and/or assessments before new instructional materials are made available. In Hardy’s course, students were required to access learning materials and complete quizzes for each module before subsequent modules could be accessed. Hardy and his colleagues believe this approach helped students pace themselves and decreased the odds that they might skip vital content needed to succeed on the final exam.
Perhaps even more impressive than the student performance in Hardy’s initial hybrid offering was the fact that his hybrid students continued to score higher than their peers in subsequent course offerings. In addition, when the course was offered fully online in the summer of 2010, students scored even higher than those in previous hybrid sections.
It’s difficult to pinpoint exactly how much of the improved student performance was due to the online self-assessments, adaptive release, or other unique aspects of Hardy’s online course design and teaching style. However, his findings clearly show that low-stakes knowledge checks and conditional release of content can have a significant impact on student performance. While I still consider myself a skeptic, even Agent Dana Scully had to admit once in a while that supernatural phenomena do exist. Whether it’s the wolf-man, alien abduction, or online courses that prove more effective than face-to-face, the truth is out there and we owe it to our students to keep digging.
Additional Resources
- View a recording of a 2010 presentation by Tim Hardy and another faculty member at UMBC in which they describe the design of their courses.
- View an overview of the UMBC presentation
- Download the PowerPoint slides from the Educause website.
- View publicly visible Blackboard usage statistics on the UMBC Blackboard Reports site.
Interesting outcome, I have found in my previous studies that self assessment for me has assisted because it provided me a true starting point. i was able to see what I did not know and what I needed to learn. Self assessment saved me a lot of time which allowed me to look past the information I already knew and provided more times toward the information needed to be acquired. I from there created tracks for myself to complete. On the other side when a Professor tends to have an abundance of students who are a lot more successful than other presenters then a red flag is warranted. Not that he is taking it easy on his students but many want to know how, what am I not doing. I comparison I use which is outside the classroom is L. A Lakers Coach Phil Jackson. Phil Jackson entered in his first year as the Lakers coach during the 99-00 season. The Lakers record was 67-15 winning the NBA Championship, the previous year the Lakers with the same team and under a different coach and style and only record 31 wins. My personal opinion is in both situations, the presentation and the use of your strengths. The evidence is there, the professor does not create the test, I think the access to additional resource and setting your own tone create an advantage for the students as well.