How do you Measure Success of Innovative Models in Education?

How do you measure the success of an educational innnovation? In The Same Thing Over and Over: How School Reformers Get Stuck in Yesterday’s Ideas, Frederick Hess wrote the following:

How would you respond if asked for a plan to transform America’s schools into a world-class, twenty-first-century system? Now imagine that there is one condition; you must retain the job descriptions, governance arrangements, management practices, compensation strategies, licensure requirements, and calendar of the existing system. Hopefully, you would flee just as fast as you possibly could and if so, you would be way ahead of the rest of us, who have spent decades slogging through that dismal scenario. p. 1

Somehow I overlooked this 2010 book until recently, but much of what I’ve written and said about education reform aligns with at least a few key ideas in this book. Of course, there are plenty of differences as well. I suspect that I will have other posts prompted by some of the ideas that Hess presents in the book, but for now, I’d like to focus on the quote above. I suspect that one of the reasons why there are so many limitations to our innovations is that we have created rules and measures that are directly tied to existing practices.

For two decades, I’ve been studying innovations, school models and learning contexts that break new ground in the education space. As I often tell people, when I decide whether to visit a school, I’m far less interested in visiting if the school has desks in rows, bells, or letter grades. I don’t mean that literally (although all three of those are often absent in the schools that visit). What I mean is that I am interested in learning from models of schooling that have moved away from the 19th and 20th-century molds. They are not just tweaks on the existing system or practices. They are genuine alternatives or sometimes completely new.

Yet, as Hess and many others point out, there is much that has changed since many of the common features in contemporary schools were initiated. Where high school graduates were a small fraction of people in the United States a century ago, it is the majority today. Expectations about what should be learned in school have changed. The diversity of people served has changed. The demands of life and work have changed. . .drastically in some instances.

Some argue that the longstanding and dominant schooling model can be adapted to meet these changing needs, but others, including myself, believe that the changes are significant enough that we are wise to explore a much broader and more diverse selection of learning contexts and approaches. Even if the current model is able to adapt enough to meet the changes and contemporary needs, it is certainly not the only (or best in any widespread and demonstrable way) option.

One point that Hess makes in his book, however, is that people championing for greater and broader educational options are sometimes labeled as enemies of public education. People mistakenly equate “public education” with specific roles, policies, practices, political structures, models, frameworks, approaches and balances of power. To argue for a new way of thinking about teacher professional development, teacher accountability, or new roles and job descriptions is too often labeled as an attack on public education. I’ve seen this happen in many instances where the people exploring these alternatives were genuinely seeking to improve and strengthen public education. The problem, as I see it, is that people are defining something like “public education” or more broadly “school” by a group or person’s preferred form and not by its purpose and mission. Itt could also be that they are just using “attack on public education” rhetoric to demonize people who hold positions different from them, to protect their personal interests (but I like to give people the benefit of the doubt).

One person argues that it is too extreme and potentially irresponsible to overhaul the entire system, to imagine something entirely new. The next argues that this is the only way, that we are on a sinking ship. You can do all the remodeling that you like to that ship but it is still sinking. It is maybe just a little more attractive or comfortable while it is sinking, but it is irresponsible to convince people that they should trust you and stay on the ship.

I’m not convinced by either of these positions, at least not in many contexts. Instead, I am confident that we are wise to resist censoring some possibilities because of personal agendas and political interest. Exploring the possibilities and creating a diverse ecosystem of educational models and learning contexts strikes me as the wisest and safest way to face the nature of life in a connected world. This is why I am a champion for mission-minded experimentation with rich feedback loops and constant adaptation.

The mission foundation is important because, without that, we don’t have a means of measuring our success. We just use the existing or dominant measures which were never designed for an innovation with a distinct mission or vision. The traditional measures were designed under certain assumptions about a system. When that system changes, the measures often need to be revised or changed as well. If not, they fail to measure what matters most or is most promising about the new effort.

Too often public voices of education critique one approach to schooling on some one-dimensional basis (or maybe a short list that serves a given group’s agenda). Then they write and talk about their assessments as if they should be the universal measure of quality schools. Starting with mission invites us to consider an alternative. We measure each school according to its distinct mission. There is certainly room for a small set of more broadly accepted measures, but we must force ourselves to be incredibly careful about what goes into such as list, as history shows us that it is often too tempting to adding ideas and wording in these more widely used measures that create unnecessary limitations and boundaries, that unintentionally (or sadly, in some cases, intentionally) create winners and losers among the students.

Using existing, sometimes reductionist, oftentimes biased (toward traditional efforts) measures will not give us adequate insight on a given innovation’s promise. If we want that, then we must differentiate our measures on the basis of the mission, vision, values and goals of that specific innovation. Those with the greatest power for core measures are well aware that the person who controls the tests has immense control of the system.

Oddly, I’ve even seen well-known critics and figures in contemporary education critique new innovations and models (even entire systems like the charter school system. . .despite the fact that “charter” has many different meanings depending upon the state and context, and that each charter has a distinct mission and vision) on the basis of measures established for more legacy schools and practices. And, of course, many of our governmental policies are written with narrow conceptions of education and schooling in mind, which intentionally (or I suspect more often, unintentionally) limit the extent or nature of innovation in education or at least sets them up for failure or reductionist analyses of their value and impact.

The widely used and well-known measures make it easier for people to compare, but they also too often disregard the distinctions. Remember that cartoon which critiques one-size-fits-all testing, assessing an elephant by how they can climb a tree? This same point applies to how we measure new education models and innovations. More often than not, we need differentiated measures for differentiated models. When we do opt for a set of universal standards and policies, it takes great care to design them in ways that do not unnecessarily limit promising future innovations.

What Compels People to Pursue Radical Innovations in Education

What compels people to pursue more radical innovations in education? It has now been almost two decades since I started to more seriously and systematically study innovations in education and innovative learning organizations. Many of the musings about that show up in the chapters of my book on Missional Moonshots (not to mention the many articles on this blog), but since my exploration started, I can’t think of a single day that has passed without some thought experiment or reflection about educational innovation. In that sense, it has become a consuming passion for me because I see educational innovation as an important social good, and I have immense respect for those who tap into the courage, creativity and hard work necessary to pursue revolutionary or radical innovations in education.

As such, I’ve spent plenty of time thinking about what compels people to pursue such innovations. What is it that happens inside or outside of people that draws, drives or inspires them to get off the paved roads of legacy education models and frameworks and do the hard work of helping to create completely new roadways? Under what conditions is this more likely to happen for a person? While some of this has to do with how people are wired (both genetically wired and wired through a longstanding set of life experiences), there are other aspects at work as well. That is what leads me to start to put into words some of what I’ve seen. Amid many observations, conversations, formal and informal interviews, and my study of educational innovators and entrepreneurs, the following six consistently show up as conditions that often catapult people into trying something more radical in the education space.

When there is nothing to lose or you have little stake or loyalty to the established system.

This doesn’t need to be an objective statement. You might, from many perspectives, have a great deal to lose. What matters, however, is that you believe that you have little to lose, or perhaps that you do not have a strong sense of loyalty to the existing system. You might (or might not) be extremely loyal to the broader mission or goals, but not necessarily the system or current methods. Perhaps the system failed you. Perhaps it was never that important to you. Perhaps you are coming from outside of the system and looking at it with fresh eyes. Regardless, this is a significant entry point for some who pursue what others might consider more radical or revolutionary innovations.

While some critique educational innovators who don’t have longstanding experience in the classroom, it is sometimes this outsider-ness that allows them to think and act in what others might consider more radical ways. In fact, some don’t even see or think that their innovation is all that radical. Feeling like an outsider might be unpleasant for some of us or a source of pride for others. Either way, it can drive us to look at the context from a unique (or at least less common) perspective. We are willing and able to consider possibilities censored or disregarded by insiders. We are open to possibilities that others reject because they would have too much to lose by such possibilities.

When there is no other option but the mission is still important to you.

“Necessity is the mother of invention”, right? Or, as John Kotter points out in much of his work, a “sense of urgency” can be a powerful lever for change and innovation. If there is no other option and you lack a compelling mission, innovation is less likely. Or, if you have mistakenly glued the mission and your current practices together, no longer able to see that they are indeed separate elements, you may rather shut down, learn to live in persistent failure, or use denial to avoid the intense pain of current failure instead of looking for alternatives and innovations. Yet, when one sees that the mission is compelling and separate from what is currently being done, and the option of staying the course is no longer an option, this is enough to move some people to lead or embrace revolutionary innovations in education.

When you experience a compelling alternative.

Sometimes people are stuck in educational ruts simply because they are not aware of the alternatives. Yet, when they see them, when they experience them firsthand and work through some of their doubts and questions, this is enough for some to venture into more radical changes. It is why I advocate so strongly that people at least take the time to get informed about the possibilities, even if they don’t choose to embrace any of them.

When your passion for the goal and/or mission far exceeds your fear of loss, discomfort or failure.

There are plenty of us who have many radical or more revolutionary ideas. It is just that our fear keeps us in check, we are not willing to take the associated risks, or the pain and discomfort associated with the change is not tolerable to us at the time. Yet, for many who do embrace a more radical educational pathway, it happens when their passion for the goal or mission grows to such a level that it overshadows these others. Or, we find ourselves in a life circumstance where we’ve been able to minimize some of these risks enough that we are then willing to venture out into the less knoswn or unknown.

When you are deeply connected to or convinced of the minority opinion, situation or a specific need.

There are winners and losers in the dominant education system. When you are connected to those in the system who are on the losing end and you care deeply about those people, this can be enough to move to you to bold and new actions. It is not a coincidence that many parents are active innovators in the charter school system throughout the United States. Interview founders of innovative charters and independent schools and you will find compelling stories, often about their own children. Love and concern for another person (family member or not) is a fuel for more radical innovations in education.

When the vision or dream is too strong to deny or delay.

I’ve heard from this from many educational innovators. They sometimes thought, planned and dreamed for years or decades. Finally, at some point, the conditions were right but they also got to a point where they just could not wait. They had invested so much of themselves into the idea that they just had to do something about it. So they acted. Sometimes they have a vision for the impending doom if we continue down the standard path and they’ve reached a point where it is so urgent (internally), that they just need to do something. In other cases, it is just that they want it to happen so much and all the years of thought and emotion create a tipping point toward action.

Interview innovators in education, and you are likely to find one or more of these six answers at work. There are many others as well, but these six are among the more common and transparent. These are the kind of things that compel people to what the rest might consider more radical innovations in education.

The Death of Testing and the Rise of Learning Analytics

I know that it is sad news for some, but more than a few of us have assessed the situation, and the prognosis is not good for our friend (or perhaps the arch enemy to others of us), the test. We might be witnessing the death of testing. Tests are not going away tomorrow or even next year, but their value will fade over the upcoming years until, finally, tests are, once and for all, a thing of the past. At least that is one possible future.

Tests are largely a 20th century educational technology that had no small impact on learning organizations around the world, not to mention teachers and students. They’ve increased anxiety, kept people up all night (often with the assistance of caffeine), and consumed large chunks of people’s formative years.

They’ve also made people lots of money. There are the companies that help create and administer high-stakes tests. There are the-the companies that created those bubble tests and the machines that grade them. There are the test proctoring companies along with the many others that have created high-tech ways to prevent and/or detect cheating on tests. There are the test preparation companies. There are even researchers who’ve done well as consultants, helping people to design robust, valid and reliable tests. Testing is a multi-billion dollar industry.

death of testingGiven this fact, why am I pointing to the death of the test? It is because of the explosion of big data, learning analytics, adaptive learning technology, developments around integrated assessments in games and simulations and much more. These technologies are making and will continue to make it possible to constantly monitor learner progress. Assessment will be embedded in the learning experiences. When you know how a student is making progress and exactly where that student is in terms of reaching a given goal, why do you need a test at the end? The student doesn’t even need to know that it is happening, and the data can be incredibly rich, giving insights and details often not afforded by traditional tests.

Such embedded assessment is the exception today, but not for long. That is why many testing companies and services are moving quickly into the broader assessment space. They realize that their survival depends upon their capacity to integrate in seamless ways with content, learning activities and experiences, simulations and learning environments. This is also why I have been urging educational publishing companies to start investing in feedback and assessment technologies. This is going to critical for their long-term success.

At the same time, I’m not convinced that all testing will die. Some learning communities will continue to use them even if they are technically unnecessary. Tests still play a cultural role in some learning contexts. My son is in martial arts and the “testing day” is an important and valued benchmark in community. Yes, there are plenty of other ways to assess, but the test is part of the experience in this community. The same is true in other learning contexts. Testing is not always used because it is the best way to measure learning. In these situations, testing will likely remain a valued part of the community. In some ways, however, this helps to make my point. Traditional testing is most certainly not the best or most effective means of measuring learning today. As the alternatives expand and the tools and resources for these alternatives become more readily available, tests will start the slow but certain journey to the educational technology cemetery, finding a lot alongside the slide rule and the overhead projector.

Want to Display Your Digital Badges? Here are Some Options.

Updated on 4/1/2017

In 2014 I created an article about the services that exist to issue open badges. There are many examples of how groups are issuing badges. There are, however, fewer interesting examples of how people are actually displaying their badges or using badges as part of their online identity. That is the purpose of this article. The only exception is that, instead of my collecting and reviewing the list of display services/options, I thought I would experiment with inviting the larger badge community to share their ideas. As such, the bottom of this article includes a simple form for you to suggest a service to be added. As services are submitted, I will add them to the list below (which starts with none). Let’s see if we can crowdsource a near-exhaustive list. Please note that many of the services mentioned in the badge issuing article also provide a means of displaying badges, but I will hold off on adding those, as it would be great to get a description of each service from people who either run the service or have used it a great deal and know it well.

Scroll down (below the form) to see verified submissions.

Name of the Badge Display Service / Option: Open Badge Factory

Name of the person submitting and connection with the service (founder, user, etc.): Don Presant

Website of the service (if relevant):

Description (please consider any description that would help readers understand its role/function/benefits/limitations.): Open Badge Passport is a free, easy to use service, where you can receive and store your Open Badges safely and share them with whomever you like and wherever you like.

Your free Passport account is a secure place for you to:

• Accept and upload Open Badges from any service that supports the Mozilla Open Badge standard

• Store and manage your badges for future use

• Display your badges on Pages, or “micro-portfolios” with other files and multimedia content

• Share your badges on social media such as LinkedIn, Twitter and Facebook

Name of the Badge Display Service / Option: Badgr

Name of the person submitting and connection with the service (founder, user, etc.): Nate Otto (director of this software project at Concentric Sky)

Website of the service (if relevant):

Description (please consider any description that would help readers understand its role/function/benefits/limitations.): Badgr is a backpack service that allows users to create collections of earned Open Badges issued to any of their verified email addresses. Users may generate a share URL for a collection and use that to spread the word over email, social media, or in job applications. Badgr also provides HTML iFrame embed code for badge earners who wish to embed their collections directly onto their blog or website.

Name of Service: Bestr

Website of the Service (if still active):

Description of Service (or any information about why it is no longer active…if you have that information): Bestr is the first Italian badging platform developed by CINECA, the leading consortium of Univerisities (73) and the Ministry of Education and Research.

It is a portal that connects learners to Employers and Learning providers based on Open Badges

Name of Service: Makeawaves

Website of the Service (if still active):

Description of Service (or any information about why it is no longer active…if you have that information):

Makewaves is the safe social learning platform for children to share what they make, challenge themselves with Missions and show their achievements with badges. Helping young people realise their full potential, by surfacing, capturing and communicating their growing skills. For Young People (Makers) Young people can take part in fun learning missions created by Teachers and high profile partners and earn rewards for their work. We make it safe to share blogs, videos and photos with friends, family and the Makewaves community. For Teachers (Publishers) Transform your curriculum with Makewaves badges. We give you the control you need to deliver and manage learning online. Track progress, give feedback and support students of all levels. Easily capture learning across formal and informal settings via web, mobile or tablets. Partners (we call you Mission Makers) Organisations can create Missions at a national scale and enable teachers to engage with your topic and issue awards on your behalf. For Parents/Carers Parents/carers can be assured their child is part of a secure moderated community. They can follow their child’s learning journey, provide encouragement and receive updates from the school to their mobile.

Name of Service: Open Badge Academy

Website of the Service (if still active):

Description of Service (or any information about why it is no longer active…if you have that information):

Today, learning happens everywhere. Yet, we still struggle to capture valuable learning that takes place outside of formal settings. We need a new way to help people capture and communicate all of their talents and use them to transition into new opportunities. Open Badge Academy is a complete solution that makes recognising lifelong learning simple. Organisations create academies to launch open badges Use badges to recognise learning, validate skills and build capabilities Track and demonstrate the impact of your programme Learners use badges to build a richer picture of themselves Evidence badges on the move via mobile Share your profile to stand out from the crowd Professionals verify skills using endorsements Experts, educators and peers provide evidence based endorsements of badges Connect with the people who matter to you.

Name of Service: Accredible

Website of the Service (if still active):

Description of Service (or any information about why it is no longer active…if you have that information):

This platform allows the creation and management of Open Badges and Digital Certificates. Issuers can create, manage and deliver credentials to recipients via email and view detailed reports on recipient engagement, views, shares and website referrals.

Name of Service: CanCred Passport

Website of the Service (if still active):

Description of Service (or any information about why it is no longer active…if you have that information):

CanCred Passport is a free, easy to use home in the cloud for Open Badge eCredentials that you earn for yourself. CanCred Passport is the default destination for badges issued by CanCred Factory, but will store and display all Open Badges that comply with the Mozilla standard. Passport is completely open in both directions: users can also export their badges to Mozilla Backpack and display them on social media such as LinkedIn. Another key feature of CanCred Passport is that users can publish their badges on an unlimited number of Pages, complete with additional text, files such as resumes and even embedded video. This means each Page can become its own goal-driven mini-ePortfolio, powered by the authentic evidence of Open Badges.

Name of Service: Open Badge Passport

Website of the Service (if still active):

Description of Service (or any information about why it is no longer active…if you have that information):

Open Badge Passport is a free, easy to use service, where you can receive and store your Open Badges safely and share them with whomever you like and wherever you like. Your free Passport account is a secure place for you to: • Accept and upload Open Badges from any service that supports the Mozilla Open Badge standard • Store and manage your badges for future use • Display your badges on Pages, or “micro-portfolios” with other files and multimedia content • Share your badges on social media such as LinkedIn, Twitter and Facebook