Quantification is Not the Key to Academic Excellence, but THIS Is

I just finished giving an opening keynote on mission-minded approaches to assessment in schools. This was to an audience of educators and administrators in Christian schools, so my charge was to invite them to use a distinct (perhaps unique) lens for thinking about the role of assessment as it relates to their mission. I was the philosophical introduction to a two-day event that would be otherwise applied. My session seemed to go okay, but following a presentation like this, I usually find myself flooded with new thoughts, questions, analogies and illustrations. This time was no different.

In this case, I find myself reflecting on the state of assessment and evaluation in many learning organizations, whether we are talking about assessment and academic performance or evaluation and planning on the organizational level. It brings me back to a study several years back. Years ago when I conducted a study of highly innovative schools. I concluded with a list of ten traits that were consistent among the leaders that I interviewed. My results were not intended to offer any truly generalizable set of traits that lead toward being a leader of an innovative school I was content providing a rich description of the sample that I studied with the hope that there might be inspiration or some potentially transferable insights.

One trait consistent among those interviewed was that these leaders were “addicted to effectiveness data” but I’m beginning to think that I need to adjust that wording. “Data” leads too many to think that I am talking about quantifiable data, but that was actually less common among many of the leaders in these schools. Most of the schools that I examined were charter schools with smaller enrollment numbers and this is certainly an important factor, but the leaders of these schools were not necessarily as interested in data as they were in feedback on how they were doing. In other words, they were addicted to finding out how they were doing and how to make the school even better….a commitment to continuous improvement. This is an important distinction because leading a high-impact and innovative learning organization doesn’t require being a statistician or quantifying everything. It is more about being interested in how well you are doing, facing the “facts”, and doing something about them.

In fact, the leaders of the most innovative schools or learning organizations that I’ve examined over the years seemed just as inclined toward rich stories and narratives; in-depth feedback through conversations, observations and qualitative survey questions. Similarly, if we look at amazing and inspirational educators around the world, we will find many of them are not addicted to numeric benchmarks for students as much as they are interested in mentoring students, helping them to grasp and apply increasingly complex skills and/or concepts as they progress toward excellence. We see this in the classroom, among private tutors, with great athletic coaches, as well as teachers and tutors of those in the performing arts. It is their deep sense of excellence and what progress toward excellence looks like that empowers them to help others achieve great things. How true is this for leaders in our learning organizations as well?

As such, it doesn’t take the quantification of everything to make for a high-impact learning organization or community. It does usually take people (learners, teachers, sometimes both) who have a goal or vision, work toward that goal or vision, crave and use feedback, and adjust accordingly. Sometimes numbers can help with this, but they are rarely essential. In fact, insisting upon the superiority or necessity of quantitative measures is often more about embracing a certain positivistic philosophical stance on education than it is about excellence, growth, or achievement.

There is Not One Future of Higher Education: Toward a More Nuanced Conversation

Will MOOCs disrupt higher education? What about online learning or competency-based education? Or, what about alternate credentials like the open badge movement? The more I engage in such questions, the more important it is for me to add adequate detail to better frame the conversation. Higher education is a broad term. It includes community colleges, technical colleges, trade schools, research intensive schools, liberal arts schools, faith-based institutions, for-profit institutions, schools that focus on serving non-traditional or post-traditional adults, etc. It also includes seminaries, graduate schools, distance learning schools, alternative colleges, and dozens of other types of institutions. Then there are many higher education institutions that include several of these under the same name.

Programmatic Distinctions

After looking at these distinctions, we also need to look at the different programs, professions and disciplines. The impact of online learning is different for a performing arts program than a history program. The potential benefits of alternate credentials will have different levels of perceived value for English majors and those in information technology. Similarly, the impact of MOOCs is unlikely to have the same influence on those pursuing college for the social experience as much (or more) than the academics.

Student Goals and Motivations

This is described from another perspective in the 2014 Differentiated University Pantheon Group report by Haven Ladd, Seth Reynolds, and Jeffreny Selingo. They surveyed 3200 American prospective or current college students. Instead of focusing upon traditional demographic data, they examined the reasons for a student’s interest in college. They were able to describe six distinct profiles of students: aspiring academics, coming of age, career starter, career accelerator, industry switcher, and academic wanderer. Each of these represent different motivations, goals and aspirations; leading to varied values about what constitutes their ideal higher education experience. Looking at current and prospective students from this perspective offers a clearer understanding why something like a MOOC, competency-based program, digital badge or online course might have higher or lower value to a given student.

Government, Community, and Business

While the desires and profiles of learners have an enormous impact on the future of higher education, there is also the influence of external stakeholders: government, communities, business. If we look more closely at these influences, we recognize that they do not share a single value in higher education either. Government influences might have a bias toward economic development. Business might be primarily interested in the development of a workforce that meets their varied needs. Community might have a heavy interest in the way that a higher education institution impacts the quality of life. These are too general, but they illustrate the fact that a single future model of higher education is no more likely than it was in the past. There is a reason why we have so many different types of higher education institutions today.

New Education Options

As we look to external influences on higher education, we must also look at the rapid growth of a new education industry. We look at CodeAcademy, General Assembly, Khan Academy, Udemy, new corporate training programs, and the overall increased access to free and open learning experiences online. This goes back to the different profiles of prospective learners, but the development of this new education industry gives each of us more options that ever before. They may not be quick to disrupt medical schools, but they have already established alternate routes into some of the top high-demand jobs of the next decade, jobs like software developers and system administrators.

Financial Models

There are also important financial factors. That which disrupts an expensive but non-exclusive college depending heavily upon tuition will be different from what disrupts a partly state-funded public community college, or an élite school with a massive endowment. There are schools with multiple sources of revenue and others that are almost entirely tuition-dependent. Some schools will struggle to keep their doors open without federal financial aid. Others have already opted out so they have the freedom to pursue different models of education. Such factors, combined with the others listed above, will decide the time it takes for an innovation to impact a school, and whether the school finds it necessary to respond with any urgency. And this is largely focused upon the state of funding in American higher education. If we look at if from a global perspective, we also see models where most or all of the entire enterprise is government funded. Such distinctions are too important to miss when we are looking at the consequence of educational innovations.

The Need for Nuance

None of this is to suggest that higher education as a whole will not be influenced by educational innovations. There is a long and clear history of innovation’s impact on education. At the same time, I suspect that our conversations about the future of higher education will benefit from a  more nuanced word choice. I have been as guilty as many other media outlets in making broad and general comments about the future of higher education in light of emerging innovations. While my comments are often coming from an analysis of a specific type of higher education, I have not always been clear about that fact. The same is true for many articles that we read at Inside Higher Education, The Chronicle, and elsewhere. Such articles make for interesting conversation, but without adding depth and nuance, they fail to give us tools for truly thinking about how to prepare for the future.