Monthly Archives: April 2011

Avatar photo

The Instructional Technology X-Files: Enchanted iPads, Magical Clickers, and Online Courses that Beat Face-to-Face

“Students performed 20 percent better in the hybrid version of this course compared to the face-to-face sections taught by other instructors.” When I heard this statement during a presentation at the Educause Learning Initiative Annual Meeting in February, I did something I rarely do: I closed my laptop, looked straight at the presenter, and stopped multitasking for a full twenty minutes.

I find most educational-technology conferences are a lot like an episode of the X-Files with a cast made up entirely of Fox Mulders. Everyone wants to believe. There are a lot of technology cheerleaders and a lot of iPad sightings, and no one seems to notice that Dana Scully—the skeptical, pragmatic agent designed to bring Mulder back down to Earth—has gone missing. So when someone offers up a bold promise backed by actual bar graphs, I take notice.

The presenter, Professor T. Warren Hardy from the University of Maryland–Baltimore County (UMBC), stated that his students performed significantly better on their final exam largely due to his use of online self-assessments. Upon hearing this, I immediately put on my Agent Scully trench coat and asked myself why his conclusions could be off.

  • Was his final exam easier than the one used in other sections? No, all sections take the same final exam.
  • Did he give his students an unfair advantage by using final exam questions in his self-assessments? No, the final exam is designed by other members of the department who are not currently teaching the course. To ensure a level playing field, the instructors have no knowledge of the specific questions that will appear on the final exam.
  • What if he’s just a better instructor than the faculty teaching the other sections? That might hold water if it wasn’t for the fact that Professor Hardy’s students scored considerably higher than his own past students after he converted the course to a hybrid format with online self-assessments.

Of course, I’m sure there are other variables that might impact the validity of Professor Hardy’s findings. Yet, after hearing the unique steps that UMBC’s economics department takes to ensure a rigorous and standardized final exam for the five-hundred students who take ECON 122 every year, I felt the 20 percent difference on Hardy’s final exam scores were hard to dismiss.

In addition to praising his students’ performance, Hardy’s co-presenters from UMBC noted that his course was a regular in the University’s list of most-active Blackboard courses. Hardy attributed his students’ extensive and frequent use of Blackboard largely to his course’s reliance on adaptive release. Adaptive release refers to a set of restrictions that can require students to view and interact with certain online content and/or assessments before new instructional materials are made available. In Hardy’s course, students were required to access learning materials and complete quizzes for each module before subsequent modules could be accessed. Hardy and his colleagues believe this approach helped students pace themselves and decreased the odds that they might skip vital content needed to succeed on the final exam.

Perhaps even more impressive than the student performance in Hardy’s initial hybrid offering was the fact that his hybrid students continued to score higher than their peers in subsequent course offerings. In addition, when the course was offered fully online in the summer of 2010, students scored even higher than those in previous hybrid sections.

It’s difficult to pinpoint exactly how much of the improved student performance was due to the online self-assessments, adaptive release, or other unique aspects of Hardy’s online course design and teaching style. However, his findings clearly show that low-stakes knowledge checks and conditional release of content can have a significant impact on student performance. While I still consider myself a skeptic, even Agent Dana Scully had to admit once in a while that supernatural phenomena do exist. Whether it’s the wolf-man, alien abduction, or online courses that prove more effective than face-to-face, the truth is out there and we owe it to our students to keep digging.

Additional Resources

Add a Pinch of Classroom, a Dash of Online—Blend Well

I had the opportunity to attend the Sloan-C Blended Learning Conference in Oak Brook, IL, last week, and as with most education conferences, it’s left my mind full of questions—some answers, but mostly questions. There was a panel discussion on the second day, and some of the most interesting pieces related to research on blended learning conducted at the University of Central Florida (UCF).

Blender

Joel Hartman, Vice Provost at UCF (and EDUCAUSE Leadership Award winner) related some of the key findings from the study of about two million student evaluations taken over several years.

  • Overall, student satisfaction was higher for blended courses than for either face-to-face or online. (At UCF, blended means “courses that combine face-to-face instruction with online learning and reduced contact hours.”)
  • Regardless of format, a course has a 97 percent chance of getting an excellent overall rating if these three items receive excellent ratings: ability to communicate information, interest in student learning, and concern and care for students.
  • For blended courses, the student success outcomes used to be about 14 percent higher than face-to-face or online. Now that gap is much smaller, most likely because strategies from blended are being used in face-to-face and online courses, blurring the lines between the three types.

A phrase about blended learning that I heard at the session that has really stuck with me is “classroom-enhanced online instruction,” as opposed to “web-enhanced classroom instruction.” As students realize that valuable learning opportunities can occur online—on their own schedule and in their pajamas if they wish—they naturally begin to wonder, “Why am I driving/walking/riding to campus?” With blended courses, the in-class time has to seem “worth it”—full of the types of activities that are best done in person and not the types of activities that are better accomplished online. In each discipline these activities will be a little different, but I would like to think and read more about general principles for how to best take advantage of face-to-face time and online time when designing a blended course.

Even though I didn’t win the Samsung Galaxy tablet, the conference was enjoyable and gave me a fresh perspective on course design and teaching. I highly recommend viewing the slides from the keynote address for some thought-provoking statistics and arguments about higher education today and where it needs to be in the future.

Avatar photo

Ready, Set, Act: Running Your Show in the Classroom

The sound so vibrant and rich, the tone so vivacious, the gesture so pulsating, and the emotion so poignant and touching, it brought tears to the eyes of the audience as they listened to the recitation of the “poem”—or what they thought was a poem.

“It was done in Russian by a renowned actor from Russia,” my father said as he described the performance, conducted by a visiting Russian actor to his theatre troupe in China in the 1950s. Although none of the Chinese audience could understand a single word of it, they were completely mesmerized by the presentation—until one of them raised the question: what is it saying in the poem?

No, it wasn’t a poem. With a short pause, the actor revealed, through an interpreter, what he was reciting—a restaurant menu!

So with all the feelings and passions he could project, he was reciting something like cabbage rolls, fish sautéed with onions and mixed with hard-boiled chopped eggs and, oh, potatoes mashed then mixed with eggs and smetana!

This story came to my mind as I started to plan for this year’s DePaul Annual Faculty Conference with colleagues from the office of Teaching, Learning, and Assessment. The theme we selected for this year’s conference is teaching as an act of body and brain. Inspired by Nancy Houfek’s philosophy and practice of using theatre techniques to enhance teaching (see the video below), we decided to bring her to DePaul as a keynote speaker to talk about the power of acting or how to induce tears by reading a menu.

As Nancy points out, the techniques used by actors to captivate an audience can very well be borrowed by professors to engage students in the classroom. Yet in our daily practice, we as teachers often focus almost solely on the content and leave the delivery of the content to chance.

Content is critical, but without gaining the attention of your audience, it won’t come across. While the story of menu reading is a bit extreme, it does convey a very strong message: sometimes when it is done extremely well, the presentation can overpower the content! Even if we can’t go that far, it can at least help us capture the attention of our students.

There is a common attribute shared by the profession of acting and the practice of teaching: both require a high dosage of passion. We teach largely because we are in love with it. The difference between us and actors is that they seem to know better how to make that passion visible—through their voice, gestures, and body languages. We teachers, on the other hand, rarely make any conscious choice about the nonverbal messages that we convey, especially when we are in the classroom.

On May 6th, following Nancy’s keynote speech, DePaul Theatre School professor Natalie Turner-Jones will lead a practical workshop exploring theatre-based techniques that can be applied to the classroom environment. She will explain why the way we use the classroom space, gesticulate to make a point, move, breathe, or pause all convey a clear message to our students and how making conscious choices in these areas empowers teachers to create an engaging and playful learning environment.

So, if you haven’t yet, please mark your calendar for May 6th’s DePaul Annual Faculty Conference and registers online at http://teachingcommons.depaul.edu/Conference/registration.html

Avatar photo

Strategic Planning: Technology for the Whole University

Now that the transition from Blackboard to Desire2Learn is complete, it is time to start looking forward to what additional functionality or systems faculty need to be effective and efficient in online teaching. To this end, I have been spending some time in the last couple of weeks pulling together a list of faculty to participate in a focus group.

Gathering names is always a tricky business, and one thing that I have become increasingly aware of is that in any strategic-planning process, it is important to listen to all voices, not just those who are the loudest. This is especially true when looking at technology adoption. Often it seems easier to go back to those faculty with the most to say, those who are already strong advocates, or those that are friendly to new technology. While it is easy to listen to the early adopters and technologically savvy, to do so at the exclusion of other voices runs the risk of alienating your primary audience.

Those early adopters are often at what we refer to as being at the bleeding edge of technology. Wikipedeia defines bleeding edge technology as being "so new that the user is required to risk unreliability, and possibly greater expense, in order to use it." While it is important for there to be people out front experimenting with the possibilities, this is not where most of your users (nor the institution) will be, for obvious reasons.

When you add technology into the mix you also find that some faculty are more hesitant to voice an opinion because they don’t feel as though they are as technologically savvy or competent. While these individuals may not be able to recite a list of technological solutions to a problem, they often have real needs that can be expressed regardless of the solution. When gathering requirements from faculty, it is important, therefore, to take the specific technology out of the conversation. For example, instead of asking faculty "what other technology would you like to see the university adopt," ask questions like "what would you like to be able to do with your class that you currently can’t," or "what frustrates you about the technology currently available to you," or even more important, "why are you not using this tool?"

The "why are you not using this" question can be just as important if not more important than the "what do you need” question. For example, if the goal of the university is to have 100 percent participation in the use of the learning management system, it is important to find out why some faculty choose not to participate. Let’s not assume that we know the reasons or that if we build it they will come. Instead, we need to hear from all the voices about what is working, what isn’t, and what the true need is.

It is important to remember that any planning process should always include feedback and input from all stakeholders not just those with the most to gain (or lose) and that planning requires compromise between these same stakeholders—adoption should not alienate the greater population for the benefit of a few or vice-versa. We must always remember to listen to the voices of all users before making our decisions.