Davidson College’s first MOOC, a course on medicinal chemistry, wrapped up last month on the edX platform. Producing and delivering the MOOC required significant effort on the part of several Davidson staff and faculty members. With such an investment comes the natural question… Was it a success?
Evaluating success in education of any type is difficult, and MOOCs are no different. The mountains of (often messy) data from online student activity create assessment challenges for MOOCs. Details such as page views, time of engagement with videos, and discussion board interactions are all available. In fact, each time a student clicks within the platform that interaction is recorded. Despite the opportunities posed by ample data sets, MOOC assessments are often disappointingly reduced to reporting how many students finish a course.
Even the idea of “finishing” a MOOC is not a simple concept. How does one define finished―students who passed the course, took the last test, or viewed content during the end of course? In comparing the outcomes of our course to other schools’ MOOCs, we found our definition of finishing had to adapt to the definitions used by other institutions.
Harvard and MIT recently disclosed data on all their edX courses that had been offered from fall 2012 through summer 2013. Harvard and MIT had a collective MOOC enrollment of 841,687 students. Attrition was reportedly highest in the first week at 50 percent but then slowed to 16 percent in the second week. A total of 29,551 (5.1 percent) students merited certificates of achievement at the end of the course. For our first edX course (14,183 students), attrition was higher in the first two weeks at 61 and 32 percent, respectively. The percentage of students receiving a certificate, however, also was slightly higher at 5.8.
Another institution that has been active in the MOOC area is Duke University. Duke’s spring alumni magazine reported enrollment and outcome data for all of Duke’s 2013 MOOCs on the Coursera platform. Of the 876,354 enrolled students, 12.4 percent attempted to answer the first homework set of the course. Approximately 3.4 percent attempted the last assessment task in the course. In our course, 18.5 percent of all students attempted the first homework set, and 4.3 percent worked the last task. Fortunately, the description of Duke’s criteria was very clear and could be matched directly to accessible data for our course.The data in the table are not easily compared. The Harvard/MIT study bases percentages upon “student activity”, which is never explicitly defined. The activity is presumably related to video or page views in the various courses. For the Davidson data, attrition estimates are based upon student views of homework problems at the beginning of each week. Class-wide homework viewing information is readily available from within the edX system.
Based off of various categories of completion, I consider our course a success. Relative to MOOC offerings at Harvard, MIT and Duke, our course fared well in two metrics for course completion―both the percentage of student certification and percentage of students who attempt the final exercise.
Are these two metrics the best measures of course success? Probably not. Assessment should factor in student expectations and course goals, as well as involve rich student activity data. Ideally, institutions offering MOOCs should work with MOOC platforms toward the creation of measures for reporting course and student outcomes. Such endeavors would certainly facilitate the evaluation and comparison of MOOCs while perhaps also lending credibility to MOOCs as an emerging means of education. In other words, some standards in the Wild West world of MOOCs would benefit both the offering institutions as well as education consumers.
Although best practices for assessing MOOCs are still in their infancy, I am very comfortable with celebrating Davidson’s first MOOC as a success. Maybe someday I will be able to support my opinion with generally accepted data.
My journey with the DavidsonX Medicinal Chemistry course began last fall, across the ocean and many latitudes north in Uppsala, Sweden. With the semester winding down and sunlight at a premium, I was looking for a side-project for those dark afternoons. What more natural way to entertain oneself than β-test an advanced chemistry course? And so, as darkness blanketed Uppsala at 14:30 each day, I would curl up in a café and watch the sunset while drawing morphine’s pharmacore and calculating IC50 values.
What began as a cozy afternoon companion soon became an opportunity to be a teaching assistant (TA) for Davidson’s first edX course. The prospect was daunting—little ol’ undergraduate me, who hadn’t even completed biochemistry (pssst, my major is biochemistry), fielding questions from pharmaceutical chemistry doctorates. Erm, aren’t you supposed to be notified somehow if you’ve pulled a Freaky Friday with a professor?
And so, accompanied by two parts excitement and one part terror (am I smart enough for this?!), the course launched, and I went with it. Doubt soon gave way to only enthusiasm when I logged on for my first shift and saw the discussion board alight with introductions:
Hi, I’m from Egypt and studying for my masters in medicinal chemistry.
I’m a high school senior from California.
Hello! I’m an undergraduate in South Africa hoping to go into the pharmaceutical industry.
I’ve been working in the world of higher education for the past seven years as a video media technologist. While video technologies have been advancing at amazing rates, one thing hasn’t changed: You still have to look into that emotionless lens when speaking to a camera audience. There’s no getting around it, and the first time is always a struggle.
Trying to connect with an audience you can’t see is a skill that is hard to master. Most of us just aren’t cut out for it. As the video producer for Davidson College’s venture into the world of MOOCs, one of the biggest challenges I face is trying to capture on camera the energy that is present in a classroom setting.
As a small liberal arts college, Davidson’s faculty pride themselves in their ability to connect with their students and form close academic relationships. So, how do you take professors who excel in a small classroom environment, remove them from it, place them in a studio in front of a camera, and then ask them to connect with students they cannot see?
I came to MOOCs by way of copyright. I am not a lawyer (a phrase many around me are probably tired of hearing), but I do have some expertise in the use of copyrighted materials for educational purposes, which is why I joined the DavidsonX team. It was clear from the beginning that we would need to tackle many intellectual property (IP) issues as we created our MOOCs. As I set out to explore what those issues might be, I quickly realized that I had entered a whole new world, much of which hasn’t yet been thoroughly mapped.
Davidson’s library is relatively small, especially in comparison to many of the edX institutions. We don’t have in-house counsel like some larger libraries and haven’t been given any additional resources for MOOC support. The college’s general counsel is wonderful and always willing to answer questions, but it’s been up to me to foresee and to tackle the IP challenges DavidsonX might face. As I began thinking about these issues, I did what any good librarian would do and started to research.
Your typical small liberal arts college does not possess a wealth of resources focused on online course development that can be leveraged to produce a MOOC. But what we lack in resources we more than make up for in collaboration and creativity. Partly out of necessity and partly by design, the DavidsonX team jumped into our MOOC course development process feet first. We are figuring this out as we go, much like a startup.
This learn-by-doing approach is not without some pain points, but the exercise in collaboration and creativity epitomizes a new academic model that we as academic technologists strive toward in our work with faculty. It is also reflective of what I believe are future models of teaching and learning, whether that is within a residential classroom or a fully online course. I will touch on the pain points in future posts, but for the moment, I want to focus on the promise of MOOCs for the academic technologist at a small liberal arts institution.
*Professor Erland Stevens wrote this post in November, three months into his MOOC production process. We hope these early reflections are useful for faculty beginning the course production process.
“What to Expect When You Are Expecting a MOOC”
Davidson College recently entered into an agreement with edX to develop four massively open online courses (MOOCs). I am the professor for the first MOOC, a course on medicinal chemistry modeled after the course I teach on our campus. As I write this sentence, we are in the middle of our Quality Assurance (QA) testing for the first two weeks of the course. We are halfway through assembling the entire course, with the team logging approximately 950 hours of planning, recording, editing, and instructional design. Everyone of this six-member team is new to the MOOC experience. I’d like to share some of the lessons we’ve learned over the past few months.
Video Killed the Radio Star (or Less is More)
When I proposed to teach a MOOC, I envisioned the course to be much like the course I teach at Davidson. I give approximately forty 50-minute lectures for a total of 2,000 minutes of lecture during a semester. I assumed I would record around 2,000 minutes of video for the MOOC. In the early planning stages, it became clear that 2,000 minutes of video would be far too great of a burden for me or the video production team at Davidson to produce given our timeline. We settled on a far more manageable 400 minutes, and a 7-week course that is more typical of MOOCs than the traditional 12-week residential course at Davidson. That works out to about 60 minutes of video for each of the seven weeks of the course. 50-minutes lectures were pared down to 7-8 minute clips focused on specific topics.
I cannot go from 2,000 minutes of delivery down to 400 minutes without losing content. The least vital material in my Davidson course—and I admit that some material is not 100 percent vital—has been cut completely. Other materials will not make it to video, but it will be included in web-based exercises. Case studies are particularly good for web-based supplemental content. Because my video delivery is more scripted, the lecture is more efficient than in a live setting. I estimate that my course only lost around 25 percent of its original content.
Davidson College, edX and the College Board are participating in a new collaboration to provide Advanced Placement (AP) teachers and their students with access to a suite of online instructional modules to improve educational outcomes in calculus, physics and macroeconomics.
National data show that many students struggle to complete such independent, college-level coursework. This new, blended learning model will use exam data to pinpoint challenging concepts in the three disciplines, and then develop and present concept-based resources using the edX platform for teachers and students.
The New York Times featured the new partnership in the Education section of the paper. Read the article.
The college will offer the first of several Massive Open Online Courses (MOOCs) taught by Davidson faculty beginning March 10. Professor Erland Stevens, chair of the chemistry department, will present the seven-week course “Medicinal Chemistry: The Molecular Basis of Drug Discovery” through DavidsonX, a nonprofit online learning initiative hosted by edX.
In “Medicinal Chemistry,” Stevens will address how pharmaceutical drugs move from concept to market, and how a drug’s chemical structure relates to its biological function. Students should be able to identify organic chemistry functional groups and read line-angle chemical structures. DavidsonX courses are free and open to the public; register now.Medicinal chemistry is an area that touches the public more personally than most fields of chemistry. Stevens explained, “Students want to see where the rubber meets the road, and in medicinal chemistry we get to do that. Everyone knows someone with a good or bad encounter with medicine, and the process of developing drugs, their place in the health care system, and their price is constantly in the news. Medicinal chemistry is highly applied to our daily lives.”
Rather than sit on the sidelines and wait to see how Massive Open Online Courses (MOOCs) might affect Davidson, the college is actively exploring online education so it can adopt best practices and avoid the pitfalls. “Our interest is in determining how an online platform will allow us to go further in ways we don’t imagine now,” said Mur Muchane, executive director of information technology.
Last summer President Carol Quillen announced that Davidson has joined the edX consortium for online learning. The non-profit alliance was founded by Harvard University and MIT just a year ago, but is already recognized as a leader in the field. EdX enrolls 1.2 million students through 29 affiliated institutions of higher education worldwide. Most at this point are large universities. Davidson and Wellesley College are the first highly selective edX liberal arts institutions.
New technology will enhance student learning, expand opportunities for personal contact with talented faculty
Davidson College announced today a partnership with edX, a non-profit enterprise founded by Harvard and MIT, to produce a series of Massive Open Online Courses (MOOCs) over the next several years. Davidson is one of the first leading liberal arts colleges to collaborate with edX on the XConsortium platform, which is now comprised of 26 global educational institutions.
“This is an exciting day for Davidson as we realize our commitment to exploring new technologies that can enhance student learning and expand opportunities for contact with our talented faculty,” Davidson President Carol Quillen said. “Like the founders of edX, we seek to understand how MOOCs can best enable more and more people to advance their education, and we believe that this collaboration will extend Davidson’s reach and impact for good around the world.”
Davidson’s agreement with edX gives it the opportunity to offer four DavidsonX courses over the next three years.