Assessment Design Tips for MOOCs and other Open Courses


MOOCAssessmentDesignI’ve participated in dozens of open courses over the years, and I can appreciate many designs and approaches.  With that said, I have a hard time with the use of traditional grading systems in MOOCs.  As I wind down my role as a host and facilitator for Learning Beyond Letter Grades, I’m starting to participate in a few new MOOCs.  Given that I spent the last six weeks co-learning and facilitating in an open course on current and emerging models of assessment, I likely have a more critical eye with these courses.  Nonetheless, it is a struggle when I am taking a course on educational innovation only to discover largely traditional practices that lack solid support from an assessment standpoint.  So, I offer the following ten assessment suggestions for those who are designing MOOCs.

  1. Focus on Formative Assessment – The majority of feedback that people get in their course should be formative.  That is the feedback that helps us discover how we are progressing toward one or more goals.  This can be self-assessments, peer-assessments, instructor feedback, computer-generated feedback, feedback from mentors and people outside of the course.  However, allow it to be formative…a checkup and not an autopsy.  Allow the participants to use this feedback to improve and refine their work. This means not making it summative by simply using it as a way for participants to accumulate points that count toward an overall grade or certificate. Of course, you are welcome to browse the assessment options at Beyond Letter Grades as well.
  2. Make the Assessments Authentic – In most cases, we are talking about adult learners who will be your participants.  They are taking this to connect with others and/or to learn something of value. “Of value” usually means something that they can use in their work or avocations.  Design assessments where they actually build, create, or design something that they can use in other aspects of their lives.  If they can’t use it, make it authentic enough that they can easily transfer the tasks completed in the assessment to a similar task in their work or avocation(s). This usually (but not always) means setting aside or minimizing the use of things like true and false or multiple choice question quizzes and tests. Performance on such assessments do not transfer to post-course life nearly as well as more authentic assessments.
  3. Beware of Using a Grading System to Punish – If you are going to use a grading system, make it a measure of what students have or have not learned. This is lost if you start removing points for late work, penalizing for behaviors that you want to discourage, etc.  It may work to do these things, but it turns your grading system into something other than a measure of student learning. If you want a system to track or encourage certain behaviors, then build a second and separate system for that, maybe a special badging system that publicly recognizes certain contributions.
  4. Consider Using an Alternative to a Letter Grade System – These are innovative, digital age open learning experiences and the letter grade system is a largely outdated mode of assessment.  If we are in school systems, many of us will still need to use them (unless there is adequate support for alternatives).  However, why fall back on such a system with a MOOC? There is no need, and we have plenty of more helpful systems.  Why not a badging system with clear criteria, a series of rubrics related to specific concepts that are important, or even a mini standards-based report card that gives more granular and helpful feedback?  Or, depending upon the size of the class, what about trying out a narrative assessment plan where peers provide the feedback in narrative form, using a checklist and a little guidance from you (They will likely need help learning to give good narrative feedback to one another.)?
  5. Design for a Pick-and-Choose Mindset - Most people will not complete everything in your course. A MOOC is not a pre-made meal.  It is more like a buffet.  People will wander around and pick and choose what they want to do and what they want to avoid. Some may go through it to earn a certificate or something else, and you may have more specific expectations to accomplish that.  However, many will not.  Don’t forget about them. Design an assessment plan that welcomes or even invites people to do some picking and choosing while also getting good and helpful feedback.
  6. Revisit Your Assessment Vocabulary - Instead of terms like quizzes, tests, and assignments; add some fun and interest to the course by using a different vocabulary.  It might be something that is more authentic to out-of-school life.  It might be some action words or metaphor-driven concepts that draw interest and curiosity (challenges, missions, mind mirrors, action plans, etc). You get the idea.  Be creative and have some fun with it.
  7. Don’t Let the System Design Your Course – Many MOOCs are in learning management systems that lead you toward first thinking about more traditional approaches to assessment. Don’t let the system drive your MOOC assessment design.  Instead, consider designing your entire plan outside of the MOOC. Plan it out in Google Docs, your favorite mind mapping tool, or even sketch it out in your idea notebook.  Whatever you do, don’t design it in the system before you’ve had a chance to create your ideal blueprint.  This will be more challenging at first, but it will allow you to be more creative, to pursue your true vision for the course, and to make it more of a reality.  You can always revise and tweak the design if you have to fit it in a certain learning management system.  Or, you might be able to run parts of it beyond the system so that you can bring your course vision to life. This approach will also give you a great chance of creating some wonderfully interesting, valuable and outside-the-box approaches to assessment.
  8. Reconsider the Teacher / Student Dichotomy – I continue to be grateful to Howard Rheingold for introducing me to the term “co-learner” as a way to refer to myself in a course. Again, we are in a new learning environment and this gives us a chance to play, experiment, and try out new roles. If you want a little help thinking about different ways to approach the relationships, you might appreciate my visuals on 9 Roles of the Online Teacher and What is Your Teaching Metaphor? This will directly impact the way you think about and approach assessment in your courses.
  9. Strive for a Design That Promotes a Culture of Learning, not Earning – It is ironic that I am writing this given that I am finishing a course right now that has the earning of badges as a significant part of the design.  However, I tried to design it so that the badge was a visual recognition, but that people most wanted the feedback. Based upon many interactions with participants in my most recent MOOC, that was very much the case.  People sought feedback from others and the “badge hosts”…and they also worked on cultivating more skill in self-assessment and the idea of assessment as learning. How do you promote a culture of learning?  That is the topic of the last live session (last night) from Beyond Letter Grades, but here is the shortened version. a) Design learning experiences that are likely to result in what Mihály Csíkszentmihályi, refers to as flow. b) Consider using Martin Seligman’s positive psychology idea of PERMA to shape the culture of your course. As you design the course, simply brainstorm ways that your design can encourage or foster each of these five elements in the participants. c) Aim for cultivating a “community of purpose.” This comes in part from how you write, talk about and represent the course. A community of purpose is a community of practice that is purpose-driven.  It is driven by “why” questions.  It recognizes that there is nothing more motivating than a deep, strong sense of purpose. Don’t be afraid to talk about that purpose often and in varied ways.  That will help learners think about the course as something more than a course…as a purpose-driven community.  As a result, it doesn’t become about earning a certificate, grade, or other accolade.  It becomes about pursuing that purpose.  After all, people don’t usually need your MOOC.  Most are taking it because they choose to do so.  Help them discover, remember, and reflect upon why this is meaningful and purposeful, and you will make huge strides toward a culture of learning.
  10. Leave Room for Student-Initiated Feedback Loops and Assessment Plans – Invite, encourage and create spaces for students to self-organize.  This may well include self-organized ways to get feedback on their progress like blogging and getting comments from others, or perhaps posting questions or artifacts to a community and asking for informal tips or feedback.

MOOC are still a new frontier in learning experience design. Give yourself permission to experiment and be playful in your design.  While it is not “wrong” to use school-speak and traditional models, why not try something new?


3 comments

  1. […] I’ve participated in dozens of open courses over the years, and I can appreciate many designs and approaches. With that said, I have a hard time with the use of traditional grading systems in MOOCs.  […]

  2. […] I've participated in dozens of open courses over the years, and I can appreciate many designs and approaches. With that said, I have a hard time with the use of traditional grading systems in MOOC…  […]

  3. […] "… I have a hard time with the use of traditional grading systems in MOOCs. As I wind down my role as a host and facilitator forLearning Beyond Letter Grades, I’m starting to participate in a few new MOOCs. Given that I spent the last six weeks co-learning and facilitating in an open course on current and emerging models of assessment, I likely have a more critical eye with these courses. Nonetheless, it is a struggle when I am taking a course on educational innovation only to discover largely traditional practices that lack solid support from an assessment standpoint. So, I offer the following ten assessment suggestions for those who are designing MOOCs…."  […]