The Death of Testing and the Rise of Learning Analytics

I know that it is sad news for some, but more than a few of us have assessed the situation, and the prognosis is not good for our friend (or perhaps the arch enemy to others of us), the test. We might be witnessing the death of testing. Tests are not going away tomorrow or even next year, but their value will fade over the upcoming years until, finally, tests are, once and for all, a thing of the past. At least that is one possible future.

Tests are largely a 20th century educational technology that had no small impact on learning organizations around the world, not to mention teachers and students. They’ve increased anxiety, kept people up all night (often with the assistance of caffeine), and consumed large chunks of people’s formative years.

They’ve also made people lots of money. There are the companies that help create and administer high-stakes tests. There are the-the companies that created those bubble tests and the machines that grade them. There are the test proctoring companies along with the many others that have created high-tech ways to prevent and/or detect cheating on tests. There are the test preparation companies. There are even researchers who’ve done well as consultants, helping people to design robust, valid and reliable tests. Testing is a multi-billion dollar industry.

death of testingGiven this fact, why am I pointing to the death of the test? It is because of the explosion of big data, learning analytics, adaptive learning technology, developments around integrated assessments in games and simulations and much more. These technologies are making and will continue to make it possible to constantly monitor learner progress. Assessment will be embedded in the learning experiences. When you know how a student is making progress and exactly where that student is in terms of reaching a given goal, why do you need a test at the end? The student doesn’t even need to know that it is happening, and the data can be incredibly rich, giving insights and details often not afforded by traditional tests.

Such embedded assessment is the exception today, but not for long. That is why many testing companies and services are moving quickly into the broader assessment space. They realize that their survival depends upon their capacity to integrate in seamless ways with content, learning activities and experiences, simulations and learning environments. This is also why I have been urging educational publishing companies to start investing in feedback and assessment technologies. This is going to critical for their long-term success.

At the same time, I’m not convinced that all testing will die. Some learning communities will continue to use them even if they are technically unnecessary. Tests still play a cultural role in some learning contexts. My son is in martial arts and the “testing day” is an important and valued benchmark in community. Yes, there are plenty of other ways to assess, but the test is part of the experience in this community. The same is true in other learning contexts. Testing is not always used because it is the best way to measure learning. In these situations, testing will likely remain a valued part of the community. In some ways, however, this helps to make my point. Traditional testing is most certainly not the best or most effective means of measuring learning today. As the alternatives expand and the tools and resources for these alternatives become more readily available, tests will start the slow but certain journey to the educational technology cemetery, finding a lot alongside the slide rule and the overhead projector.

Meet Breakout EDU: Proof That Learning & Fun Go Together

In 2005, James Sanders started as a classroom teacher in South Los Angeles, where he continued for the next five years. In his words, he was “lucky enough to be one of the first teachers to use Chromebooks in the classroom” as a pilot with Google. Of course, there must have been more than luck at work as he went on to be head of innovation for KIPP Bay Area Schools, worked with education projects at Google, was co-founder of ClassBadges, became a Presidential Innovation Fellow, and more recently, James became the founder of Breakout EDU.

From their web site:

Breakout EDU creates ultra-engaging learning games for people of all ages. Games (Breakouts) teach teamwork, problem solving, critical thinking, and troubleshooting by presenting participants with challenges that ignite their natural drive to problem-solve.

James explained that the idea for Breakout EDU came when he and Michael Wacker were in Canada at an escape room with a group of teachers and students. If you haven’t experienced an escape room, you get locked in a room and work with others to solve a series of challenges/puzzles/riddles to get out. With his background in the classroom, James already knew that play was a powerful tool for learning. Combine the experience of a classroom teacher and that of a rich game design like an escape room and you get the birth of an education startup that is building an impressive community of educators who are excited about mixing games and learning.

Of course, there are problems with locking students in a room, but you can still use the game and puzzle part, and that is just what Breakout EDU does. They provide you with the starter kit or the instructions to build your own. It consists of a wooden box, a series of different types of combination locks, an invisible ink pen, and a few other items to get you started. Then there are the games. Among the 70+ existing Breakout EDU games, only a small portion were developed by the Breakout EDU team. The rest are built by the wonderfully creative and engaged Breakout EDU teacher community.

Richard Feynman Quote of PuzzlesJames is quick to note that the power isn’t in the box and locks but in the power of play, and putting these tools into the hands of great teachers. When you have that combination (pardon the pun), you get a community of students who are deeply engaged and working incredibly hard to solve problems. Along the way they are developing skills like collaboration and problem-solving, and they are gaining some content area knowledge as well.

At this point, Breakout EDU has not done any traditional marketing or promotion. They are still in beta and believe strongly in that, using this as a time to refine their ideas with the help of a vibrant community. Of course, the word is getting out, and the interest is growing, as evidenced by the robust Facebook group and the fact that I’m seeing presentations about Breakout EDU at EdCamps, regional education conferences, and in the online chatter among those in the larger educational technology community.

If you read my blog, I’m often talking in grand terms, speaking of potentially disruptive innovations or the wide scale impact of a new technology or company. James is cautious, even skeptical of such language, claiming that Breakout EDU is not a solution to any great educational problem. For James, “it is just a tool.” However, he clearly has a simple but compelling mission in Breakout EDU. He sees this as an opportunity to show people that learning and fun are not mutually exclusive.

Breakout EDU is also completely open. It is an open community. They provide everything that you need to build your own kit, so anyone with less than a hundred dollars for supplies and a local Target or Home Depot has pretty much everything that they need to get started. Anyone can design a game or use the other games, and there is nothing proprietary.

In an education startup space where many are designing entirely virtual products and services, Breakout EDU maintains a keen interest in the physical element. Many of the existing games certainly have a blend of the online and physical, but James believes that, “there is something powerful in getting them to move around and work in groups.”

Such an approach in the classroom is also an interesting way to reconsider the role of failure. Where failure on a test or assignment is no laughing matter and often even a cause serious concern, failure in a game becomes part of the learning experience. A group might have 75-100 failed attempts at solving a puzzle before they find the right solution. This is a good thing in a game, and failure becomes just part of the journey to success.

In the contemporary education startup space of people positioning for large market share and massive financial support, Breakout EDU is a refreshing exception. They are not seeking venture capital. They are not quick to rush out of the beta phase. They truly embody the spirit of Nail It Then Scale It, but they seem far more driven by building a great model that helps teachers than creating some world-renowned educational gaming company. As much as I could tell from James, they truly are a mission before margin outfit, a fine example of social entrepreneurship in the education space.

In the past, I’ve written about how I see potential in being a humble radical in the education space, but Breakout EDU is a good example of what I mean. They have a compelling vision, but they are more grounded in the present than grand dreams of the future. They embrace the fact that what they have is a work in progress, and they invite a community of educators to help with that progress. They are providing something of potentially great value to educators, but they refuse to overstate the potential impact of what they’ve provided.

As such, I’ll give the Breakout EDU community the last word in this article. As our chat was coming to an end, I asked James about what sort of results teachers were finding from their games. James simply posted that question to the Facebook group, and following is how the community members responded.

Mindy C. I have found that my students earning D/F’s are often the ones solving clues! It’s awesome to see them participating and finding success.

Sara B. Yes! Students that don’t do well in “traditional” school assignments seen to thrive here. Maybe it says something about their thinking…

Bailey C. I know as a new teacher I’ve been really having a tough time getting my students on board with the idea that learning isn’t always a “show me on a quiz” process. It finally clicked with them now that I am doing content based breakouts and they are having fun exploring topics rather than focussing on “what’s going to be on the test.”

Jody M. I have seen a student who, in regular math class, refuses outright to do any math at all (very bright, autistic spectrum), but in a breakout sat twice with pencil & paper to show another student how she felt something should be solved.

Jessica G. The first two breakouts I did, I had one student who would completely give up because it was “hard” and he was “stupid.” Then, after a pep-talk about perseverance, this same student KILLED the next breakout, solving clues left and right. The smile on his face was priceless when he realized he could do it if he set his mind to it…..Attitude is everything! smile emoticon

Michelle J. Some students (who feel they are “not smart enough”) have been able to appreciate their own special gifts when they think differently and are able to help the group of solve a problem. The feedback process for me has been very important, as well as being able to give students the opportunity to design their own games.

Maria G. One of my favorite things I heard in the debrief after a game with 8th graders showed me what a big role inference plays in BreakoutEDU games. They were saying that they loved not being told what they were going to learn and having to figure it out by themselves. You can see a sense of empowerment that they felt. This to me was such a powerful positive example of what these games bring to learning.

Donna W. We played one game with 5th graders during their Media Center pullout time. None of the groups were able to breakout in the time allotted. However, every time they see the Media Coordinator they ask when they will have another opportunity to Breakout. They were engaged, thinking, collaborating during the activity. They didn’t give up and they want to try again- calling that a win-win!

Toni H. During a debrief session, a student(special education) identified that he became obsessed with the lock rather than using clues to solve it. He plans to use the clues in his next Breakout session. His teacher couldn’t believe he identified the issue and planned for improving.

Heather L. I found it interesting how my high level kids struggle with both the breakout box lessons and the weekly homework challenges. The lower level kids excelled during the breakout box lesson. I watched the advanced students get frustrated by the thought an…

Bailey C. Agree! I’m seeing similar things. However, my gifted students do not fit well into that mold. I’ve found mine get pretty competitive and proprietary over their solutions.

Learning as Work or Play

I’ve learned so much more outside of school than in it. For every book that I’ve read for a school assignment over the years, I’ve likely read 20 outside of school. I’ve conducted more interviews, written more, observed more, experienced more, and learned more. I’ve also surfaced far more insights outside of school than inside it. They’ve led to meeting and connecting with fascinating people; changing my beliefs, behaviors and convictions more than anything that happened amid my formative or higher education experiences. I’ve also enjoyed these activities immensely. I’ve lost track of time on late Friday nights. They’ve driven me to travel thousands of miles for a single conversation or a few hours of a new experience. They’ve left me  falling asleep at night with a sense of accomplishment and joy about a life of discovery and learning. They’ve also kept me from falling asleep, wanting to write or read just one more page, wildly scribbling out a new idea, chatting with a new friend, or dreaming of the possibilities. I had some wonderful experiences in formal schooling as well, but they just don’t compare to what I’ve learned beyond the walls of those buildings. Why?

In The Most Productive Ways to Develop as a Leader, Herminia Ibarra wrote the following:

In contrast, no matter what you’re up to, when you’re in “play” mode, your primary drivers are enjoyment and discovery instead of goals and objectives. You’re curious. You lose track of time. You meander. The normal rules of “real life” don’t apply, so you’re free to be inconsistent — you welcome deviation and detour. That’s why play increases the likelihood that you will discover things you might have never thought to look for at the outset.

This blog is play more than it is work. This is the place where I log and experience new discoveries. I am free to debate with myself from one article to another. I’m not trying to write like an academic. My thoughts are serious and I strive for substance, but this fun for me too. I don’t try to sell myself as much as I play with thoughts and experiences, exploring the possibilities and inviting others to join me in this play. Wonderful outcome emerge. I build new connections. The play extends. It often turns into “work” in the sense that money is exchanged, goals and planning emerge, tasks are accomplished, programs are developed, and agreements are signed; but for me it is still driven more by a mindset of play than work.

This leads me to wonder, if play is such a powerful lever for learning, why not take greater advantage of it in our learning organizations? I recognize that there are times when play might not work or it might not even be appropriate, but so much of what is done in school could happen through a culture of learning by play, as so powerfully and whimsically championed by the Institute of Play. Groups like the Institute of Play represent a movement in modern learning (not just schooling) and work that:

  • invites us to accept the challenge of addressing the engagement crisis in schools and workplaces;
  • helps us take advantage of our human propensity for play and discovery;
  • sees teachers as game-designers and architects of a culture of engagement;
  • invites students to participate in quests, challenges, adventures, and experiments;
  • and helps students learn to apply principles of games and play to direct their own learning throughout life.

Doesn’t that sound fun?

But how does a school full of games and play prepare people for the real world?

First, it helps them learn. Second, it helps them maintain that inquisitive, engaged, exploratory, adventurous spirit of their childhood. Third, it helps them chang the real world into a place with more curious, engaged, playful people. As Lincoln is quoted as saying, “The philosophy of the school room in one generation will be the philosophy of government in the next.” Could the same thing be true for the communities and workplaces of the future?

10 Gamification Challenges for Educators

What can educators learn from games?  For one thing, games have a longstanding track record of keeping people engaged for extended periods of time, even to the point that they lose track of time.  This is one of many reasons that a growing number of educators are exploring things like game-based learning, the gamification of learning, or simply what principles we can extract from games and apply to learning environments.  With that in mind, if you are involved with education, consider taking one or more of the following “gamification” challenges.

  1. Make learning “pleasantly frustrating.”  This is a powerful attribute of games.  In many games, we fail much more than we succeed and yet the game has a way of keeping us engaged, coming back for more, and persisting through the failures in pursuit of that one success.  How might we create a learning experience that replicates this?
  2. Instead of giving assignments, consider giving learners challenges or quests.  Which one invites you to step up and give your best?  An assignment is something to complete, but a challenge or quest is something to conquer!
  3. Create a plan for constant and frequent feedback.  Games are amazing at this.  Rarely a turn or second goes by that players are not getting some sort of feedback about their progress.  What would it look like to do the same thing in a lesson or unit of instruction?
  4. Instead of traditional points and grades, consider giving experience points, badges, or even titles that recognize achievement.  When learners achieve, celebrate it with these things and let the learner know.  Yes, this is an extrinsic reward, but so are traditional points and grades.  The difference is that this gives you a fresh perspective on the learning environment and a chance to build distinct affirmations related to specific student performance.  It allows you to create a culture that recognizes goal-oriented achievement and not just letter grades.
  5. Reward success with a more difficult challenge.  In games, the reward is often access to the next level or the next challenge; typically more difficult than the previous one.  The fascinating part is that, within a game, the players value such a reward. How can we create learning environments and experiences where this same thing is true?
  6. Reward risk-taking, creativity, experimentation, and strategic thinking. Most of us in education like to think that we do this, but if we are not careful, it is easy to reward compliance over creativity, following the rules over exploration and divergent thinking, and playing it safe instead of taking calculated risks.  What set of skills will best help the learners thrive in the world?
  7. Give the learner/player the ability to customize the learning experience to match their needs, preferences and style.  In many games, players have a good measure of freedom to customize how they go about accomplishing the goals of the game.  When proper, why not give learners this same ability?
  8. Require the learner/player to take constant action.  In games, if you do not act, there is no progress.  There is very little passive time for the learners/players. Even if you are waiting for your turn, you are strategizing, monitoring the behavior of others, etc.  How can you create learning culture where this same constant action is at play?
  9. Call upon learners to perform their way toward competence.  Games do this, but many learning environments do the opposite. They require competence before you get a chance to perform.  Putting performance first calls for constant action, provides ongoing and immediate feedback, and much more.  While there are certainly some situations where this might not work, consider situations when it is worth trying out.
  10. Aim for flow.  Flow (as explained by Mihaly Csikszentmihalyi in the book by that name) is that situation where you lose yourself in the experience, losing track of time, personal needs, and your full attention is devoted to the task at hand.  What would it take for your next lesson or unit to draw learners into flow?

There is no need to agree with or apply all of these.  However, if you are up for the challenge, why not try out one of them and see what happens?