The Promise, Peril, and Possibility of Data, Analytics, and AI in Higher Education (3 of 7): Student Learning

This is the third installment of a seven-part exploration of data, analytics, and AI in higher education. In the first article, I set the groundwork for the series. I introduced the framework from which I intended to write, identifying what I consider to be six distinct aspects of data and AI’s influence in higher education. It simply follows the course of a learner’s relationship with a learning community. As such, the prior article looked at the pre-enrollment relationship, considering how data and AI are changing and will continue to change the way in which learners find, select, and build a relationship with a learning community (whether it be a formal college/University, informal learning, open learning resources or communities, people and experiences that foster learner, or a combination of these). Now, in this third article, I focus my attention upon how data, analytics, and AI will continue to influence the actual learning experience.

I have not intention of providing an exhaustive exploration of the topic. Of course, that would require countless volumes. Instead, what I offer at this point are a couple main observation about the challenges and opportunities that come when we venture into questions about how data, analytics, and/or AI might influence the learning experience.

The Adaptive Learning Revolution Will Really Be a Revolution

For those who still largely conceptualize the bones of the college learning experience as a collection of classes that culminate in the issuing of a credential, the world of data and analytics gives us even more opportunity to reconsider that construct. 

“Learning analytics” is a phrase that we often use in reference to all the data that we can collect and analyze to determine the extent to which students are learning, or the extent to which they exhibit behaviors that someone deems important for student success. Related to this, “adaptive learning” is a system that uses these data points to adjust the learning or learner experience with the hope of some improved outcome. On the K-12 level, we see rudimentary examples with software like Aleks Math or Dreambox. The software includes a hierarchy of levels of mastery. It introduces learners to content, challenges, and lessons; and the learner interacts with this content. All along, the learner’s performance is being assessed, and the software is adjusting accordingly. Each person is taken on a slightly (or sometimes significantly) different learning journey based upon prior knowledge and performance while using the software.

Most of the examples that are prominent today are early experiments with adaptive learning, but the truly interesting stuff is still on its way. While some critique such software as too mechanistic or representative of a simple and behaviorist approach to learning and mastery, that will change. It is already beginning to change. I commend the ingenuity and creativity of early developers of adaptive learning software for math and language acquisition, for example, but most of the work at this point lacks depth. It has yet to dip into the incredible pool of research and insight that now exists about the nature of learning. The companies and learning communities that demonstrate the forethought and wisdom to soak their adaptive learning software into this larger body of knowledge may well end up being some of the most powerful influencers of learning in the 21st and 22nd century.

You might recall the buzz when Thomas Frey made the prediction that the largest Internet company of 2030 will be an online school. I’ll build on his idea to say that it will be a organization that taps into the combined power of AI and the best of learning science research. It will boast of performance increases and learning outcomes that make some dominant teaching practices look like the tools of prehistoric cavemen.

In this new and emerging learning context, we are not talking about a collection of classes that lead to a credential. We are looking at constantly monitored and documented ebbs and flows in learning. We are considering measurements of mastery with regard to discrete skills and knowledge, the nuanced changes in learner motivation, mindset, traits, and disposition as well. As the research develops, we will also see far more complex calculations that look at patterns of thinking and behavior that are predictive of success or failure in various life, work, and real world contexts. Not only that, these new contexts will make massive strides in providing greater insight about knowledge transfer challenges, the extent to which someone’s learning in a classroom or on a computer adequately transfers to various real-world or novel contexts.

Similarly, measurement of learning will not just be about a moment in time. These emerging technologies will make it possible to monitor what is retained, lost, refined, or re-purposed over years or decades. As such, the monitoring of learning for someone like a surgeon will not stop when the MD is earned (if the construct of an MD persists into the 22nd century). Monitoring of learning and performance will continue and be tracked throughout one’s career.

There is much caution when it comes to such measurements. They are often crude and disregard important nuances and factors. Measurements become values-laden clubs that beat people and communities into submission. Over time, they become so accepted and commonplace that many think you are a social deviant to question their propriety.

Education and learning is rarely just about outcomes. As I so often write, education is always values-laden, which is the source of most great debates in the past and present when it comes to the what, why, and how of education in different contexts. Education isn’t just about the future as well. This AI future that I am describing is most likely inevitable, but we are still wise to not forget the significance of the hidden curriculum. We are wise to consider how these new technology-driven learning contexts shape how we think and our sense of humanity. We are wise to discuss and consider the values-laden nature of each new iteration. The most progressive transhumanists among us might dismiss these warnings as nostalgic hogwash. I do not. I hope that many who read this do not consider it hogwash, especially those who will help bring about the AI in education revolution.

Instructor Resistance 

In such a future, holding on to sentimentality will not be a successful resistance. We must think deeply about what it means to use, live, and teach in a world of big data and artificial intelligence. What do teachers do best? Or, what do we want to be provided by other humans instead of from technology, even if empirical data might suggest that the technology achieves the same or a better result?

Clayton Christiansen’s theory of disruptive innovation suggests that innovations often take root by serving a population largely overlooked, under-served, or even disregarded by others; and they do so with what might initially be an inferior product. Over time, the product improves and gradually captures a larger market until it has the potential to disrupt or displace what was happening before. This is already occurrring when it comes to data-enhanced instruction and learning analytics software. These are supplements to traditional teachers. They are also being used as replacements. As the outcomes of these substitutes or replacements increase, a growing number of people will choose to skip the traditional teacher and classroom. We can lament the loss. We can grieve for what this means about the deeply human and personal aspect of education. It will still happen.

Research from the psychology of attention is being purposed to track facial expressions, heart rate, and more. These too will eventually be part of the data set used in adaptive learning software. Smart watches and health trackers are getting more sophisticated every year. All of these lead to data points that will allow for increasingly nuanced and sophisticated learning analytic data sets. Are there problems and concerns with all of this? Absolutely. Yet this is also an indication of what is possible with AI and learning analytics. These systems will eventually capture psychological nuances and social cues that a single teacher cannot possibly notice and take into account when teaching a group of 50 in a traditional classroom. 

This doesn’t mean that people will become obsolete in learning of the future. However, it does challenge us to reconsider roles. In a recent interview with Stephen Downes, he noted that many of these musings about AI are much further in the future than some suspect, and that the most promising present possibilities reside with open networks of learning, new technology-enhanced connections and communities among people. It is amid these connections and among these communities that we can see some of the greatest strides in the more immediate future. We can see support for his position in the rapid adoption of social media, open learning communities, and other such networking over the past couple decades. Perhaps it is within these connections and communities that we can make greater sense of the roles best played by people and the extent to which AI and adaptive learning might be integrated. A blend of these two is what we already see emerging, and I see no evidence of that slowing. What happens in the long-term is yet to be seen. 

The Future of Learning

I have no doubt that AI and adaptive learning will bring about some of the more significant changes to how people learn over the upcoming decades. Tbe algorithms will develop and evolve to draw upon increasingly complex data sets. Incredibly consequential errors will be made along the way. Tension will grow and persist regarding the role of AI alongside the role of human interaction in learning. This is the future of learning in the rest of the 21st century.

The Death of Testing and the Rise of Learning Analytics

I know that it is sad news for some, but more than a few of us have assessed the situation, and the prognosis is not good for our friend (or perhaps the arch enemy to others of us), the test. We might be witnessing the death of testing. Tests are not going away tomorrow or even next year, but their value will fade over the upcoming years until, finally, tests are, once and for all, a thing of the past. At least that is one possible future.

Tests are largely a 20th century educational technology that had no small impact on learning organizations around the world, not to mention teachers and students. They’ve increased anxiety, kept people up all night (often with the assistance of caffeine), and consumed large chunks of people’s formative years.

They’ve also made people lots of money. There are the companies that help create and administer high-stakes tests. There are the-the companies that created those bubble tests and the machines that grade them. There are the test proctoring companies along with the many others that have created high-tech ways to prevent and/or detect cheating on tests. There are the test preparation companies. There are even researchers who’ve done well as consultants, helping people to design robust, valid and reliable tests. Testing is a multi-billion dollar industry.

death of testingGiven this fact, why am I pointing to the death of the test? It is because of the explosion of big data, learning analytics, adaptive learning technology, developments around integrated assessments in games and simulations and much more. These technologies are making and will continue to make it possible to constantly monitor learner progress. Assessment will be embedded in the learning experiences. When you know how a student is making progress and exactly where that student is in terms of reaching a given goal, why do you need a test at the end? The student doesn’t even need to know that it is happening, and the data can be incredibly rich, giving insights and details often not afforded by traditional tests.

Such embedded assessment is the exception today, but not for long. That is why many testing companies and services are moving quickly into the broader assessment space. They realize that their survival depends upon their capacity to integrate in seamless ways with content, learning activities and experiences, simulations and learning environments. This is also why I have been urging educational publishing companies to start investing in feedback and assessment technologies. This is going to critical for their long-term success.

At the same time, I’m not convinced that all testing will die. Some learning communities will continue to use them even if they are technically unnecessary. Tests still play a cultural role in some learning contexts. My son is in martial arts and the “testing day” is an important and valued benchmark in community. Yes, there are plenty of other ways to assess, but the test is part of the experience in this community. The same is true in other learning contexts. Testing is not always used because it is the best way to measure learning. In these situations, testing will likely remain a valued part of the community. In some ways, however, this helps to make my point. Traditional testing is most certainly not the best or most effective means of measuring learning today. As the alternatives expand and the tools and resources for these alternatives become more readily available, tests will start the slow but certain journey to the educational technology cemetery, finding a lot alongside the slide rule and the overhead projector.

5 Strategies for a Balanced Approach to Big Data in Education

We are in the decade a big data. During this second decade in the 21st century, many are grappling with the challenges and opportunity of massive data and the emergence of tools to mine and analyze these data. Within education, this is not new. It started long before No Child Left Behind, with the 20th century growth of modern educational psychology and measurement movements. From that era we saw IQ and aptitude testing, standardized and multiple choice tests, the Bell Curve, and countless efforts in quantifying almost anything about students: achievement, retention, reading proficiency, performance by demographic data, etc. While some of these ideas have a much longer history (China used proficiency exams for civil service already in 2200 B.C.), these certainly gained a new level of attention and importance over the last 150-175 years. Consider how things have changed, as explained by David McAruthur in his 1983 report, Educational Testing and Measurement: A Brief History.

In the mid-1800s, Horace Mann launched the use of written exams in the United States. Based on that, promotion to the next grade was based on performance on these exams. Prior to that, it was oral exams and personal recommendation of the teacher. Testing was not central aspects of American education before this.

Already by the end of the 19th century, because these tests and the perceived negative impact by some, we saw the birth of a new concept, “teaching to the test.” In places like Chicago, there was even a ban on using tests for grade promotion, arguing that the teacher’s recommendation was the better option. The concern was that we would lose much of the “magic” in teaching and learning environments if we used a reductionist approach like just focusing upon students performing well on the tests. Nonetheless, even today there is an entire industry around test preparation and equipping people to perform as best as they can on tests ranging from the SAT to the GRE, LSAT and MCAT.

At this point in history, with more teaching and learning happening partly or fully through technology-enhanced means, we have even more student data to track and analyze. Every action on a device can be captured and reviewed. Similarly, external agencies are requiring the tracking of data about students: data ranging from demographics to attendance, vaccination records, and academic progress.

The advocates for big data point to many affordances. We can identify people at risk before it is too late, sometimes even proactively. We can use data to drive improvements in one or more eras. We can use data to more quickly identify and address problems. We can use data sets to personalize learning, conduct research on best and promising practices, measure progress, and to prevent students from slipping between the cracks (any number of cracks: socially, academically…).

Critics bring plenty of concerns to the conversation as well. Large data sets might inform policy, but while those policies help many, there are always losers with some policies as well. For example, perhaps predictive analytics allow learning organizations to track who is likely to succeed in an upper level math course. As such, they use this to track students on pathways that are more likely to work out for the students. That might exclude a student who is passionate about a STEM field and is willing to work hard enough to overcome the risks and alters that discouraged such a path. Then there are concerns about data privacy, misinterpretation of data, and losing sight of the people…the faces behind the numbers. Empathy and personal connection can be easily disregarded as important part of informing policy. Numbers matter, but so do the people represented in those numbers. There is an important difference between knowing that 80% of a given population is performing below grade level on reading and knowing the stories, challenges, and lived experiences of the people in that 80%.

How do we pursue the benefits of big data while also avoiding some of the limitations or negative elements? There is no easy answer to such a question, but I offer the following ten suggestions.

  1. Persistently challenge the assumption that quantitative data are more important. Get adept at arguing for the benefits of qualitative and quantitative measures. There are plenty of stories and examples from we can pull to make our point.
  2. Learn about the stories of big data success and invest just as much time in learning about big data disasters. Specific cases and examples can help important practice. Push for much higher levels of big data fluency. If we are going to be increasingly data-driven, then we need people who have higher levels of quantitative fluency. Without that, we either relegate important thought and work to a new quantitative tehcnocracy or we risk making flawed, even dangerous, conclusions by misreading the data. Anyone arguing for increased use of data must also be ready to put in the hard work of becoming more literate and fluent.
  3. Beware of the drive to value that which is easier to measure. This starts by persistently bringing the group back to mission, vision, values and goals. If we do not do this, it is easy enough for missions and goals to change just because some goals are more neatly and easily measured than others. Big data is not just about numbers. You can have big quantitative and qualitative data. Be a firm voice in starting with mission. We want to be mission-driven, data-informed, not the other way around.
  4. Consider an equal treatment approach to data usage. If teachers insist on using big data to analyze students, then shouldn’t big data be used to inform policies for teachers as well? What about the same thing for administrators and board members? While this will never be perfect, pushing for an equal treatment approach is likely to nurture empathy and more balanced consideration by decision-makers. For example, consider how many educators insist on the value of frequent tests, quizzes and grading practices that they would vehemently oppose if the same practices were applied to them. Take this an apply it to the state agencies, federal agencies, and politicians as well. Any politicians committed to arguing for big data on a state or federal level in education should be just as open and welcoming to the use of an careful data-driven analysis of their success, record and behaviors in office.
  5. Champion for the most highest possible ethical standards when it comes to data. Sometimes it is so tempting to use data, even for noble purposes, but we have to pass for security reasons or to protect various parties. We must hold the highest possible standard in this regard, even when personal loss is involved.

Big data in education will continue to have affordances and limitations, but these five strategies are at least a good start in promoting a more balanced approach.


10 Higher Education Trends to Watch in 2015 & Beyond

Thanks to the University of Wisconsin Madison Department of Educational Policy Studies and the Wisconsin Center for the Advancement of Postsecondary Education (WISCAPE), I enjoyed sharing a draft paper and informal comments yesterday on When For-Profit and Non-Profit Meet: Monopolists, Entrepreneurs and Academics in Higher Education. I might be able to share a polished version of that paper at some point, but driving back from the event, I realized that I have failed to share my predictions post for this current year, something that I’ve done each year since 2012. So, five months into the year, here goes. I’ve decided to focus this list mainly on higher education trends and innovations, although some of them have parallels in the K-12 sector. These do not necessarily reflect the paper or comments shared in the mentioned presentation, but there is certainly some overlap.

Which trends should we watch in the second half of 2015 and beyond? This represents one of the more common types of questions people ask me. Which trends in education are most noteworthy? Which ones will persist and grow? Which ones will fade and wither? I try to add an important disclaimer when I step into a futurist role, even the near future, because education is one of those deeply political and regulated sectors, adding plenty of uncertainties in any claims. Nonetheless, here is my short list of ten. None of them are new, but I expect them to gain increased attention into the second half of 2015 and well beyond this year. Or, in some cases, look for interesting pivots or adjustments in these areas.

Customized / Personalized Programming

Some  higher education institutions are not positioned to respond to this growing request. As such, this is a promising opportunity for the more agile colleges and universities, continuing education units, as well as a range of education companies interested in providing training, courses, or educational opportunities. What I’m referring to here is the idea of a company or organization partnering with a University or an education company to create and offer custom training, courses, degrees, or programming. It might be a partnership to create a custom leadership training program for those who show promise in a given company. It might be a business school that offers a special cohort MBA program for a given employer, agreeing to integrate case studies and other elements that are directly related to that company. It might be a school district partnering with a college to design professional development (for credit or more) that helps teachers pursue specific district goals for improving student learning. There are so many possibilities. While this is not new, this approach is gaining more attention. A growing number of Universities are showing the interest and willingness to pursue these partnership. At the same time, we see existing education companies ramping up their capacity for these services as well as startups that specialize in such a model. For the latter, it isn’t the type of model that gains extensive interest from investors because such personalization sometimes prevents the scaling that leads to the payoffs they are seeking. As such, this leaves ample opportunity for willing colleges and Universities along with boutique education businesses.

Educational Partnerships

I’ve already written about the Starbucks / ASU partnership, but this is just the beginning. Expect to see several other high-profile announcements of similar partnerships over the next couple of years. Also scan the growing size of offices in Universities dedicated to building external partnerships. For the colleges, this helps them save marketing dollars, and sometimes allows them to pass that savings on the the students. For the employers, they have an employee perk to keep good talent, and raise up the next group of leaders by investing in their education. The PR for both sides doesn’t hurt either.

Big Data & Business Analytics in Education

Advancement wants it. Admission wants it. Marketing wants it. Professional advising staff wants it. More higher education leaders are interested in dashboards that give them a snapshot of the University status regarding key performance indicators. As blended and online learning grows, there are also more data points about student and faculty behavior that are recorded and can be mined. This world of informatics and analytics is growing quickly, and it is a massive money maker for software providers. Consider how some healthcare systems are paying a quarter of a billion dollars or more for implementations of new informatics systems. While not typical at that price point, Universities have already started investing hundreds of thousands (sometimes millions) to implement data warehouses and analytic software. I don’t expect it to gain as much traction among faculty in the classroom this year, expect where there is experimentation with adaptive learning and the like, but that will potentially come within the next 2-4 years.

Alternative Education Meets Higher Education

I’m probably a little early on this one, but I still expect to see signs of it over the next 12-24 months. What I’m referring to is a form of what we’ve seen happening with independent schools, charter schools and magnet schools on the K-12 level. We see project-based schools, classical schools, self-directed learning academics, place-based learning schools, leadership academies, etc. I expect to see an equivalent emerge in the higher education space. Look for more colleges and Universities offering niche routes (sort of like the already existent honors colleges at some schools, but focused on niche approaches like project-based learning, service-learning, even self-directed learning. In addition, while this is not nearly as easy of a development, I expect to see the announcement of at least a few new higher education institutions over the next 1-2 years that have interesting niches and approaches, schools like Minerva or more long-standing schools like Antioch College, Bennington College, Goddard College and Prescott College.

Virtual Reality in Education 

This one will probably gain more traction on the k-12 level in 2015, but look for it to make a few headlines in higher education in 2015 and 2016 as well, especially given that some of the software is starting to catch up with the hardware in this industry. Events like the Silicon Valley Virtual Reality Conference will give us a glimpse into near future uses in higher education.

The Rise of Higher Education Beyond Regionally Accredited Entities

As we continue to see the Department of Education and regional accrediting bodies trying to make sense of the developments and innovations in higher education, it puts innovation Universities at a disadvantage, especially the smaller to mid-sized schools. As a result, we will see foundations giving even more interest to education companies that are not bound by things like the federal financial aid system or regional accreditation. Expect more announcements from new and existing companies that provide courses, training, credentials, and “degrees” of their own; but not part of the standard regionally accredited higher education system. Many of these will earn their credibility through close connections, conversations and sometimes formal partnership with employers and professional organizations that oversee credentials for a given trade or field of work.

MOOC Credentials

We’ve heard a little bit about it in the past, and while I’m still not one to jump on the claims that MOOCs will make traditional higher education obsolete, this year and next will be the time when we see the growth of credentials (certificates, badges, etc.) and even traditional college credit being associated with learning demonstrated through MOOCs. Coursera and EdX are already engaged with this to some extent, but expect other players and for credentials from these sources to be refined, expanded, and gaining more traction. It should probably go without stating that there will be tension and push back on this one.

Competency-based Education

The Competency-based Education Network accepted its second group of 30 schools into the network in 2015 (I work at one of the new member schools). Beyond that network, I’m also directly aware of countless schools that have moved from interest to formal exploration, experimentation, and even implementation of competency-based programming. The scope of this trends impact will surprise many in higher education.

Self-Directed Learning

The most attention to this topic will continue to come beyond the walls of formal higher education. However, interest in self-directed learning is a natural progression of the digital revolution, driven by a combination of increased access to online content, resources, communities, etc; as well as education companies targeting “learners” with apps, services, and resources that allow them to reach formal and informal learning goals.

Self-Blended Learning

This is essentially self-directed learning finding its way in the formal learning environment. As more resources for learning emerge online and college students become further informed about them; we will continue to see creative and unexpected student-led “blends” used to help them find success in school and to achieve personal goals that are not being adequately supported by the formal college experience.

There are plenty of other trends that are likely to grow and expand, but I’m confident that these ten are here to stay. Expect to see them in more headlines, to learn about new products and services focused upon them, and for them to become more common aspects of higher education discourse.