Remember the classic scene from Dead Poet’s Society where the new teacher, John Keating, instructs the students to rip pages out of the poetry text? He offers the students a different view of poetry, one that invites people to feel instead of analyze. There is something to be said for appreciating the forest and not just chopping it down into discrete pieces that are more easily quantified. At the same time, we are in an increasingly data-driven world, and I don’t expect to see it disappear from the field of education anytime soon. So, what are we to do?
In the Walking Dead of Higher Education, a software vendor wrote an article about the need for more satisfactory ways of approaching assessment in higher education. He contends that higher education administrators are having trouble proving the value of a college degree because they don’t have effective ways to measure what students are learning. This is not a new notion. In fact, it is a mantra in our increasingly data-driven world. We see the drive for more data-driven decision-making in everything from political campaigns to marketing plans, church growth strategies to public health initiatives, crime prevention efforts to assessment of organizational effectiveness. This is the expected progression of living in an increasingly digital and connected world. As I’ve mentioned elsewhere, the first wave of the digital era was about increased access to information through the Internet. The second wave was the discovery that it is the connections, communities and social interactions that have some of the most transformational possibilities in the digital age. Now, in this third wave, we see the convergence of the first two waves (These are not entirely distinct. There are social and analytic elements already in the first wave). Now we have increased access to more information than we ever imagined, but every one of our actions (social and otherwise) is becoming part of that information, data points to mine and analyze for meaning, and used to achieve business, educational or other goals.
When I first understood that Google was mainly an advertising company, I had a great illustration of this new age. I took people to Google, conducted a random search and then we looked at the results. What do you see? Most of us now get the general idea. The first results to show up are paid ads. What is Google selling? They are selling you, based upon data about you, data suggesting that there is a possible match between who you are, what you want or might want, and what others are selling. The consumer becomes the product being sold via a business-to-business interaction between Google and another company. This is very old news, but this concept, evident in many other online business strategies since the 1990s, illustrates the idea that our behavior becomes valuable data.
Now schools and education vendors are increasingly involved in similar practices. Education companies are coming to Universities and P-12 schools, offering ways to collect, track, analyze, and extract meaningful insights from student data, not just student performance on tests and key assignments, but thousands of other potential data points. Some are coming to schools that already have a massive collection of dust-covered student data, and companies are showing them the stories that these data tell, and how these stories are important for the work of the school. Others are starting at the beginning, offering new ways to collect data that have never been collected before, let alone analyzed.
I’ve written about adaptive software before, even providing a simple visual to illustrate the feedback loops involved with such software. Adaptive software is an example of how these data are being used on the micro level to create learning experiences where student performance and progress is persistently tracked and the software adapts accordingly (or, apart from adaptive software, where the teachers or students are prompted to respond). The testing culture in American K-12 education illustrates some of these data on the macro level. There is no question that big data sets will be used in ways that will amaze, trouble and baffle many of us over the upcoming decade. This is certain to happen in the education sector as much as anywhere else.
Yet, it is important for us to remember that the technologies of modern statistics and data analytics have values. The more we seek to use them, the more they drive us to look for that which they are best at measuring. This means that standardized tests are likely to be used before portfolio assessment practices, personalized adaptive learning software before messier student-centered self-directed learning practices. Until we gain the ability to collect and analyze broader types and ranges of learner activity, these technologies have a way of telling us what to value as much as measuring that which we value. For example, suppose I wanted to understand how much my children love me. Would I be satisfied with a dashboard that indicates the number of times they spoke those words in the last month, the number of acts of kindness toward me, or something else that is easily quantifiable? I’m reminded of something I referenced by G.K. Chesterton back in a 2012 article about letter grades:
G.K. Chesterton, in The Everlasting Man, is talking about a completely different subject, but he discusses how many people seek to quantify things that are not inclined toward quantification. They are pictures, not diagrams. Consider your favorite piece of art, song, or sunset. Would you agree that a careful quantitative analysis and report on any of those would give an accurate picture of why it is your favorite? Would you be satisfied with a quantitative analysis in place of a picture or the real thing? Suppose you went to an art museum and discovered that every painting and sculpture was replaced with a chart or diagram that represents the same concepts or ideas that were previously illustrated in the work of art. I contend that letter grades too often do a similar thing.
This is an important caution as we move forward with learning analytics, big data, and strategies to identify quantitative measures for student learning. The measures are often (usually…almost always…maybe always) just approximations. The data will tell a story, but there are other important stories of learning to be told that are not read in the numbers, not in any set of easy measures that we develop. I am not arguing that we resist the move toward analytics in education, only that we better understand it, what it is, what it is not, how it can influence and shape our mission, vision, values and goals. The data are not just measures of our stand alone goals. Key Performance Indicators, for example, are hardly ever just measures of things we value. Over time, they influence what we value, sometimes in subtle and hardly noticeable ways, but other times (and over time) the influence can be substantial. Is it time to share one of my favorite Postman concepts again? With any technology (including big data, statistics, and learning analytics) it is not just about learning how to use the technology, but taking time to understand how it uses us.
The author of the article that I mentioned before (Walking Dead of Higher Education), for example, seems to propose that we provide quantitative answers to questions I’m not sure can be answered by quantitative measures alone, not unless we change the questions to make them more easily quantifiable. If we really want to get into what students are learning and how the higher education experience is impacting students, it will be a longitudinal mixed methods study on every learner. However, that is too messy, time-consuming, expensive and unrealistic…at least for some (but this might change). What usually happens is that we reduce the academic standards to something less (even as we do it in the name of something like academic rigor), but something less that is easier to measure, communicate and understand by parents, consumers, the public and politicians. The technology of modern statistics is not neutral. It values that which its current tools can measure and tends to minimize the rest. Without thoughtful deliberation, the charge to become more data-driven in education is a proposal to change the core values of schools to fit a new set of values embraced by a specific set of stakeholders. Only time will tell how this push for elevating the quantifiable will play out in higher education and P-12 schools.
As much as this article may seem to indicate otherwise, I am an advocate (or rather a Luddvocate) for learning analytics and big data in education. I am not, however, a champion for doing it with our eyes closed. It is time for us to count the cost as best as we are able, and to allow our institutional mission, vision, values and goals to maintain a strong voice in this growing world of data.
By the way, if I put my futurist hat on for a moment, I caution some of the loudest advocates for data-driven education and the dominance of quantitative measures in schools. Big data are here to stay, and it will push itself into many other areas, not just education. Politicians are using data to drive campaign decisions. Wait until data start to flip, when citizens start to use data to analyze the behaviors of politicians. Just as information was democratized with the first phase of the digital revolution, wait for some excitement when analytics becomes equally democratized (look for a full article on this topic in the near future).