Challenge Based Hiring: Exploring the Promise of Blending Hiring and Lifelong Learning

Recently, inspired by a group of colleagues to submitted and entry, I decided to offer a submission to the Re-imagining the Higher Education Ecosystem challenge put out by the US Department of Education Office of Educational Technology. Reading through the call for participation, I was excited by how this challenge set right at the interaction of so much of my work, writing, and research over the past number of years. There was workforce development, learn agency, learner-centered educational ideals, a sympathy for alternative credentials and alternative learning pathways, as well as a vision for increasing access and opportunity. So, browsing some of the other proposals and drawing upon some of my most recent musings, I put together the following draft of a proposal for what I call Challenge-Based Hiring. It isn’t revolutionary. It isn’t even brand new. Yet, the more that I began to pull together disparate ideas into this promising experiment, the more excited I got about the possibilities. As such, I’ve included a rough draft of the proposal below, with a few sections (like the timeline) removed. I welcome your thoughts. By the way, the challenge allows for others to join teams, so if this captures your interest and you would like to join in potentially making this idea a reality, consider becoming a partner.

The Challenge Based Hiring and Learning Platform

Elevator Pitch (it can be the same than the “Summary” section)

What if we could turn job postings into authentic learning challenges that increase access and opportunity, give rich and authentic learning experiences that lead to current or future employment, provide opportunity for anyone to show their skills (or develop them along the way) and readiness, and improve the match between employer and prospective employee? The Challenge Based Hiring Platform is designed to do just that.

Describe the Education Ecosystem of 2030

The education ecosystem of 2030 will be more open, blurring the lines of learning across context, but also blurring the lines between activities that are currently separated from one another. One such line is the process that companies use to search for and hire new talent, and the world of learning and preparation for such jobs. As such, the Challenge Based Hiring Platform is an experiment in blending these two worlds in a way that has promise to benefit both employers and individuals seeking work or just ongoing, authentic learning experiences that can build competence, confidence, and create new opportunities.

Challenge Based Hiring is intended to be an alternative to current job boards, and the standard process of job postings, applications/resumes, interviews, and then companies struggling to find the right match. Rather than focusing upon past credentials, diplomas, or degrees; challenge-based hiring is focused upon whether people can demonstrate, in the present, that they have the knowledge, skill, and dispositions necessary to do a job well. Or, if someone does not yet have the skill, participation in one or more challenges allows the person to develop new skills and document them in a sharable and discover-able online profile.

First, this platform will help employers take a job description for a vacancy and turn it into one or more authentic, tasks-based challenges/competitions. Employers agree to offer some sort of reward or prize for finalists, a small but reasonable cash prize that recognizes the time and effort devotes by one or more people, or perhaps a recognition of accomplishment or endorsement of work. Challenges can be designed where there is one winner, a select number of winners, or a large number “winners”; and awards are created and distributed accordingly. Those who take on and complete challenges also get to create profile that includes past experiences, education, credentials from across contexts, etc. However, challenges are designed so that employers/challenge creators are not able to view participant profiles until after they judge/select/identify winners. After winners are assigned (again, this can be one or more), profile data is released to the challenge organizer / employer, and an introduction is made for the possible next steps of employment.

Each challenge is designed and aligned with a core set of skills that the employer deems essential or non-negotiable to complete a current and specific job/vacancy at the company. As such, those who participate in challenges, regardless of whether they are hired, are engaging in challenge-based learning experiences that deepen their knowledge and skill, and further equip them for skills that are indeed vetted and valued in an actual workplace environment. Participants are encouraged to take on challenges that extend beyond their current abilities.

In the future, higher education partners might contribute challenges that align with common, non-negotiable skills for jobs posted in past challenges, allowing for the addition of a school-based credentials or recognition. However, this is not essential to the model or the early pilot.

As part of a challenge, participants are provided with guidelines, suggested resources to guide their work on the challenge, and sometimes learning resources, courses, and training modules provided by third parties (curated to align with the challenge).

Upon completion of a challenge, feedback is provided to finalists (and others when deemed possible and reasonable). In addition, all participants are provided with further resources, links to online courses, and other learning opportunities that can deepen their expertise in the area of the challenge. Use of these resources can be documented and added to a person’s ever-growing profile.

The participant profile will be designed in such way that participants can include information about their performance and learning from past challenges and associated courses and training. As such, those on the platform are building an increasingly substantive portfolio of lifelong learning. This increases their ability to communicate their knowledge and skills to employers. It also gives richer information that employers can use to connect with them. In future iterations of the platform, there is the possibility of using simple algorithms to recommend certain challenges to people based upon their background, experiences, and interests described in their profile.

A secondary but significant benefit of this platform is that is teaches employers to think about hiring in a new way, paying greater attention to competencies and proven skills, and less to formal but indirect signals of competence like degrees and other credentials. As such, we are focusing our early efforts on jobs with varying level of skills, but those that do not have a legal requirement for specific credentials (like some in healthcare or other professions that require licenses). With that said, future iterations could entail participation in a series of challenges that lead toward some licences and credentials in high demand fields, or at least provide early progress toward that. Successful completion of challenges could also eventually be considered as alternative evidence of learning, even used as evidence for prior learning credit that is offered by higher education institutions as part of a degree program. This allows the platform to serve and function within the current and dominant formal education ecosystem while also preparing for a more open and cross-organization ecosystem likely to develop as we look to education in 2030.

Describe a pilot or “scalable beta” that will move us toward this vision of the future.

For the initial pilot, we will recruit 3-5 companies in a highly populated area. Each will agree to work with us to design at least one challenge that is tied to the job description of a current or future vacancy. We will work with these companies to carefully design high interest, high value challenges and curated resources that could serve as learning tools for these challenges. Then we will release the challenges to the public, targeting people living within a reasonable commute of these places of employment, working with community organizations, education institutions, and government agencies to ensure that we reach a wide array of potential participants.

We anticipate that this approach will reach and motivate certain individuals and not others, and the pilot will provide us with greater insight on how to reach and engage an increasingly diverse population in the community. While future iterations can go national or beyond, we want to start local so that we can refine the process, gain actionable insight, and increase our chance of participants in the pilot obtaining valued job skills and some getting gainful employment. As such, we anticipate pursuing a series of challenges and using each one to deepen our understanding of what is working, what is not, and how to improve.

Who are the “users” or beneficiaries and how will their experience in the future of learning and working be impacted by your pilot?

Users are both employers, particularly employers who have difficulty filling vacancies for jobs, but who offer a solid, living wage, good benefits, and opportunity for employee growth and increased opportunity over time. Users are also people in the community who are already working, in school, or who are seeking new employment in the present and future. While the challenges are designed to connect people with employers, they are equally designed to deepen and promote lifelong learning that increases competence, confidence, agency, access, and new opportunities. As such, users might be job seekers, those seeking ongoing learning and professional development, and/or both.

How will your project be inclusive of a diverse population of students and their needs?

Working with community, government, and education partners will be an important part of this project. We will need to experiment with them so that we can determine the most effective ways to encourage participation from those who want to, but might lack the confidence. As such, given the necessary resources, we plan to include embedded guides who monitor participant involvement in challenges, and explore a myriad of creative, playful, and specific ways to encourage persistence in the challenges. Some computer-generated game design features in future iterations may assist with this as well. How will success of your project impact the learning ecosystem of the future and how will you measure this? Based upon insights from the pilot, the intent is to partner with more employers, working with them to design further challenges on the platform. A great success would be a growing number of employers partnering with us to embrace or at least experiment with challenge based hiring, early wins by successful matches between employers and job seekers, growing numbers of participants engaging in and completing challenges (and expanding their profiles), and either the exponential growth of this platform, or a number of other organizations creating comparable platforms. In the latter case, it would be highly desirable for platforms to build a consortium that allows the interoperability with regard to learner profiles.

Diploma Mills Redefined: There are Far More of Them Than You Might Think

What is a diploma mill? Scan the web and you will find a range of answers. Some limit their idea of a diploma mill to illegitimate schools. They are scams that claim to be legitimate organizations, but only to take your money, sending you a diplomas for little to no work. Others think of diploma mills as actual colleges or universities that offer degrees, but these schools might not be accredited, and/or they offer degrees with a watered down academic experience. Still others use the phrase “diploma mill” to describe a college or university that is more interested in getting a student’s money than in the quality of the education. They are essentially selling diplomas. All of these represent working definitions for a diploma mill today, but I’d like to offer an expanded definition.

A diploma mill is any college, university, or organization that has the earning and issuing of diplomas as its core value and/or function as described by employees at the organization and/or the students.

Notice that many definitions of a diploma mill speak to the idea of organizations that issue diplomas for a price, but without the academic rigor or the evidence of actual student learning. That is indeed a diploma mill, but I’d like to suggest that this is only one type. There are also diploma mills that make things quite hard for students. They add lots of rules and academic hoops. They might have challenging tests. They require papers and projects. They might even offer good and valuable learning experiences. Yet, in the end, they or the students that they serve see them as mainly just steps toward getting that piece of paper at the end. That is a diploma mill. A mill is a place where machines grind out grain into flour. A diploma mill is a school that grinds out people into graduates with pieces of paper. The focus is upon how many pieces of paper they issue. The process is not as important as the end goal of getting the piece of paper.

You know that you are in a diploma mill if you were to get half way through a degree program, leave, and believe that it was therefore a complete waste of your time and money. The focus was on getting a diploma, and you didn’t get it, so you basically look at it like going to the store, paying for a product, but leaving the store without the product.

For non- diploma mills, the product, if we want to stay with that metaphor, is not the diploma. It is the mentoring, the learning, and the journey of personal transformation. Do you consider that idealistic? If so, then I’d like to suggest that such a reaction is further evidence about how rampant diploma mills are in higher education today.

Scan online advertisements and you will read grand promises of earning your diploma in increasingly shorter periods of time. There is a drive to condense and crunch things into smaller chunks of time, removing time for reflection and deeper learning. There are ads that place a prominent image of a student crossing the stage with the diploma in hand, since that is the goal in a diploma mill.

The paper is not without value. Companies value it when posting jobs. Certain pieces of paper from colleges are required to even apply to some jobs. The paper adds some level of prestige. It serves as a signal of achievement and progress in a person’s life. Family members and recipients often beam with pride.

None of this is inherently bad, but I’d like to suggest that it is separate from real, deep, important, substantive learning and education. If you go back far enough in the history of Harvard, you will find a time when students who graduated didn’t even get diplomas. If they wanted one, they had to hire someone to create it and they go the President’s office, paying the president to sign it. That was in addition to and/or separate from the real focus of Harvard, which was to provide a rich and valuable education.

In the flurry of higher education innovation today, and the praise of massive enrollments, we are wise to not lose sight of the dangers associated with becoming a diploma mill. They risk devaluing the essence of great higher education. In addition to that, the more that colleges see themselves as mainly selling verification and diplomas, the more that they set themselves up for a future and impending disruption. Things might go well for them, even exceptionally well, in the short to mid-term, but that will eventually change. The learning communities that truly contribute to something important in life and the world are the ones that likely issue diplomas at the end, but that is not the main thing for them. They are true communities of learning, inquiry, and transformation.

The Promise, Peril, and Possibility of Data, Analytics, and AI in Higher Education (2 of 7): A Coming Shift in How People Find and Select a College

In article 1 of this 7-part series on data science and AI in higher education, I introduced a simple categorization for thinking about five ways in which data is used and shaping the present and future of higher education: college search and connection data, student retention and success data, student learning/mastery/progress data, organizational health data, and data that enables graduates to connect with other people and organizations. In this second article in the series, I am focusing upon that first category, college search and connection data. Some of my regular readers may be less interested in this category, while this is right in the sweet spot for others. For those who cringe at language about recruiting, admission, and marketing; I contend that what I write here has quite a bit of relevance for the nature of the learning communities in higher education as well, so I invite you to consider this article with that perspective in mind. Regardless, be assured that the next articles will focus upon the learner experience in higher education.

How do colleges find and recruit students?

There is not a single answer to this question. When it comes to how colleges find and recruit students, the answer depends upon the type of student (prospective traditional undergraduates, adults seeking a first degree, adults returning to complete a degree, adults seeking to return for a second degree or part-time graduate student, people seeking full-time graduate study, etc.). It also depends upon the type of institution. Nonetheless, the modern world of data science plays no small role. People search for almost everything online, and that is certainly true when it comes to exploring opportunities for a college degree. However, the dominant ways in which people use the Internet to search for and reach out to possible schools has more than a few limitations. Consider this one.

The Facebook Peer Advice Story

As part of my research about how people search for colleges, I informally interviewed lots of adults seeking to return for graduate degrees. I also identified some of the online platforms, online resources, and communities targeting people who are looking for degrees. Beyond that, I played the role of a prospective student and tried my luck at using Google as I typed in various questions, taking special note to which featured and paid ads showed up for me and followed me around the web (search re-targeting).

However, one of the more interesting experiences came in some of the education Facebook groups in which I participate. I noticed as people asked others in the group for advice on good online graduate education programs. Then I read through the hundreds of suggestions and comments. It was amazing to me how many people claimed that the online degree program of their choosing was the least expensive, most flexible, highest quality, etc. Granted, some of these insights are going to be subjective, but price is not. Yet, countless people thought they were getting the lowest price. Or, perhaps on the subjective side, they thought that they were getting the best value. Looking at the advice as objectively as possible, I can say with confidence that it was riddled with errors and misleading information. This was more like a debate about favorite sports teams than a careful investigation of schools and analysis of the features/attributes that are most important to a given prospective student.

Such folk systems work for people today. They have no small influence on how traditional students choose colleges as well. A reference from a trusted source, even if it is not well-informed or a best fit, is a major factor in people’s decisions. In fact, in a world of information overload, trusted sources, even wrong ones, are increasingly valued by people.

How do students search, learn about, and select colleges?

They Google. They browse YouTube. They talk to peers, mentors, and trusted people in their lives. They read articles and follow social media about various schools. They ask people in person and digitally. They are unquestionably influenced by media, marketing, and advertising. If they were not, there is no way that so many schools would be spending tens of millions (or in some cases more) annually for digital ad campaigns. Yet, I can say with confidence that these ad campaigns do not improve the quality of match or connection between student and school. A school is not the right fit simply because it was willing to pay more than another school to show up at the top of a Google search. This current system is hiding learning communities that would be a great fit for some people, and they are driving people to the learning communities with the most clever marketers or that prioritize marketing in their budget over other things.

The Dominant Metaphor

Right now the dominant metaphor related to college search and connection, especially in the adult and graduate world of part-time and online learning is a sales metaphor. We advertise to students. Even if schools resist the language, they are selling. Students are shopping. As such, much of what goes with that metaphor informs how people select schools. They shop. They are influenced by some brands more than others. Sales strategies and tactics are used and result in students choosing. There is buyer’s remorse in some cases. Even the attitude and mindset of students once they are enrolled continues to be influenced by this sales metaphor. In addition, schools are often thinking about recruitment as sales and competing with similar “products” and “services.”

The Changing Metaphor

There is a strong competitor to the sales metaphor for college search and connection, and it is growing. It is the dating metaphor.

Note: While some might not appreciate the shopping or dating metaphors because they would prefer something from high culture, or at least not from popular culture, these are the metaphors that I notice animating the language and systems that people are using.

With regard to the dating metaphor, think about the idea of a dating website. People don’t shop for a date. They enter data and the system makes recommendations about others who might be a good match. They might also have the opportunity to search for a “good match” but it is qualitatively different from dating. It is more about seeking out a good connection or relationship and less about buying something. In addition, some of the algorithms behind these dating websites are getting increasingly sophisticated, drawing from years of insights about what types of matches did or did not work well.

The dating metaphor points us to the growing role of algorithms and potentially full artificial intelligence to match and connect people with other people, people with organizations, as well as people with various products and services. Consider MyOptions (formerly, a site that collects up to hundreds of bits of data about a prospective college student, and compares that with the data set they have about a large number of colleges. Based upon everything from weather preference to level of academic challenge, the system makes recommendations for students to consider. While it started as a system only for prospective traditional undergraduates, in my interview with the founder in 2017, she indicated plans to extend it to other populations as well. For better or worse, this is going to be how a growing number of people select education opportunities in the future.

Reliance upon trusted networks and sources will remain a strong (perhaps the strongest) source that informs decisions for quite some time, however. Yet, these algorithmic developments will likely begin to further shape the nature of recommendations within these trusted networks, leaning more and more toward a dating versus a shopping metaphor.

The Change

This will change the nature of recruitment and marketing, especially for the adult, part-time and online student. In fact, there is a good chance that it will disrupt the multi-billion dollar business of higher education advertising in the digital space. Without question, there will be platforms that claim to be a neutral source of matching students and colleges (these already exist), but they are actually getting paid by colleges to rank some over others. Yet, the platforms that truly are neutral and that combine both trusted networks and algorithms will win the day. There will also be (or rather there already are) systems that simply rank colleges on the basis of something like the College Scorecard. Those lack complexity and matching capabilities, however, and will likely be secondary to the more personalized and algorithmic platforms.

This will challenge some schools that spend 20% or more of tuition revenue to recruit students through traditional advertising. As more people turn to these “dating” platforms for finding educational options, the advertising wars of higher education may well begin to wane, and I consider that an incredibly positive shift.

Note that there is no small number of entrepreneurs seeking ways to capitalize upon this shift from the sales to dating metaphor. For example, in a 2017 interview with John Katzmann (founder of the Princeton Review, 2U, and Noodle Companies), he articulated a vision for a an of the education space. Granted, Amazon at first consideration sounds like more of the shopping metaphor, but even in my short conversation with Katzmann, I suspect that what eventually comes about will be much more driven by that dating metaphor. And John represents one of more than a half-dozen significant entrepreneurs with whom I’ve spoken who are working on this problem (the higher education advertising wars) and opportunity (making better connections between students and colleges).

Transparency Matters

There are many important aspects and nuances to what I’ve written here about the metaphor shift, but for the sake of brevity, I offer what I deem one of the most important, especially in the early stages of this algorithmic revolution in college search and connections. Algorithms are created by people and they include the beliefs and biases of people. As such, an algorithm can mislead and do harm as well. It is for that reason that I argue for making these as open and transparent as is reasonable. In addition, when possible, why not give the learner some control over how the algorithms? In other words, instead of having a fixed system that matches students and colleges on the basis of what the platform owner/creator deems most important, why not leave some room for the end user to manipulate the algorithm, playing with prioritizing and re-prioritizing some aspects of the matching to see how that influences the recommendations? Better yet, why not create a system that shows people recommendations/rankings based upon four or five different configurations of the algorithm? This empowers the learners and invites them into being a co-creator of the system as opposed to someone who is just matched and sorted. Agency matters for people and society, and as this is in the early stages, this is the time to think about how we are going to create systems that are more open, humane, and nurturing of agency.

Will Google Eventually Lose Its Dominance in College Search?

Betting against a powerhouse advertising company like Google on almost anything in the digital world might seem like betting against Russell Westbrook in one-on-one against a middle schooler of your choice. Google has the talent, track record, global respect, and long list of wins that make it the obvious choice in almost any head-to-head battle in the marketplace, at least one that is within Google’s wheelhouse. It is this last statement that settles my stomach as I take a risk and make the call about Google’s role in college search.

Right now, it is arguable that Google (and Youtube) is the leading venue through which most people connect to and explore higher education options. They get on their device of choice and start searching by school name, modality, degree type, major, and related terms. They go to Google and compose questions not unlike what you might imagine people discussing with an informed relative or high school counselor in the past.

I’m in the middle of writing a series of articles about the role of big data and AI in the future of higher education (This is not one of those articles. It is related, but must ultimately be described as mild procrastination.). These articles are meant to reflect what has been years of formal and informal research on this subject. As I explained in the first article in that series, I put higher education data into five major categories, with the first related to learner-college search and connections. In my second article, I will describe an important metaphor shift that is occurring with learner-college search and connections. Without giving that away quite yet, I will say that Google’s current approach is unlikely to hold up to what is coming.

Google remains first an advertising company, and higher education institutions are not an insignificant part of the advertising dollars paid to Google. This puts Google in an interesting position. The company has a stake in maintaining the status quo when it comes to how people use Google to search for colleges. They can tweak the system, but if they break it, that hurts financially.

“Disrupt yourself before others do.” That might be a favorite quote turned cliche in Silicon Valley and the startup world, but in the real world, that is not easy to do. In fact, it is incredibly rare, even from some of the most innovative organizations led by executives who know and agree with the wisdom in such a statement.

That is what came to mind for me when I read the recent article in the Chronicle of Higher Education entitled, “Google Wants to Play a Bigger Role in Your College Search. Here’s What You Need to Know.” Essentially, the article explains how Google intends to include data from the College Scorecard (a flawed representation of data that I wrote about several years ago) and the Integrated Post-Secondary Education System Data System. In other words, Google is adding some value and insight from these data sets, but it is ultimately a safe move, and not one that forces too much of an immediate change in Google’s business strategies associated with higher education advertising.

To illustrate why I contend that Google will ultimately lose this “market” unless they take a more aggressive “disrupt yourself” approach, I will pose five questions.

  • If you want to find a date using a digital tool, what would you use or where would you go?
  • If you want to search for reviews and/or buy a book online, where would you go?
  • If you want to search for a new job in your field, where do you search online?
  • If you want to find reviews of movies to watch, where do you search online?
  • If you want to find and purchase a used product online, where do you search?

Google makes money that comes from all five of the areas represented in the questions above, but each of these questions also point us to what some refer to as platforms. More generally, answers to each of these questions draw our attention to instances where a significant percentage of people do not persistently turn to Google for answers. They might start by using Google to investigate various dating sites, but people probably end up going with one or a few of them. From that stage on, they are on the dating site, and not Google, at least not when it comes to finding a date. The same can be said for many people when it comes to buying a book (or other products), finding jobs, getting movie reviews, and posting or searching posts of used products. Google still makes some money as people compete to be the platform of choice, but not necessarily as much as if Google were more central to people’s ongoing search.

College search will go the way of dating sites and job boards, only it will be even more nuanced in its sorting and recommendations. Increasingly sophisticated algorithms blended with robust and vetted qualitative reviews and social connections are well on their way to college search. Options like this abound, but they largely represent first generation prototypes. Wait for the generation to see some disruption. Google might have a few years, maybe even 5-10, but change is coming, and it appears that a central business model that revolves around advertising puts Google at a competitive disadvantage, at least if Google wants to even get in the range of claiming to offer search services that improve the quality of one’s choice about college. So, for now it seems that the decision is to play it safe with data sets that create new winners and losers in the higher education game. In the end, however, it is seeming to me like Google might just end up being one in the losing category.