What is Regular and Substantive?

A January 2016 Inside Higher Education article highlighted the scrutiny that Western Governors University is getting by the US Office of the Inspector General. This is not new, as the investigation has been underway for almost three years. There is much that is interesting and noteworthy about this investigation, but it all revolves around determining if Western Governors University is compliant with the Higher Education Act of 1965 which distinguishes between collaborative and correspondence course by whether there is “regular and substantive interaction” between the faculty and the students.

Whether you get categorized as one or the other matters mostly because if you get labeled a correspondence program, that changes if and how you can participate in the federal financial aid program. That, of course, impacts whether students depending upon that federal program can afford to go to your school. As such, I would like to focus on that one phrase, “regular and substantive interaction.” I will share a list of observations, questions and proposed considerations; paying special attention to how all of this impacts the pursuit of educational innovation in education.

Where is the definition and explanation of what do we mean by “regular and substantive”?

Nowhere in the Higher Education Act of 1965 is there a clear and substantive definition of “regular and substantive.” Think about that. The fate of a University’s financial model depends upon whether they are compliant with a three-word phrase that is never clearly explained. Someone like the US Office of the Inspector General or the US Department of Education can come in, review your programs, and suddenly decide that you are not compliant.

Where is the review of existing models of education?

I’ve studied at close to twenty Universities, almost all of which would be considered quite traditional schools. I can say definitely that I had plenty of courses and experiences in those programs where there was limited, minimal and sometimes no “regular and substantive interaction” with my professors.

Is an non-interactive lecture substantive?

Keep in mind that even the most fundamental definition of “interaction” is that it is reciprocal. That means that an instructor standing up front, lecturing at a class, without a regular and substantive back and forth exchange is not, by definition, interaction. I have not found a single example of any federal agency pursuing a University for allowing professors to engage in non-interactive lecture as a common practice in their classes. If we are going to have such a standard, we need to apply it universally. Yes, if we are going to truly apply this standard across the board, then instructors who do not have truly interactive classes would seem to be threatening the financial aid status of the University.

What is regular?

Is this multiple times a day, daily, weekly, monthly or something else? Again, this comes back to a lack of clarity. Interview University studies around the United States and I can guarantee you that there are plenty of students who went through entire degree programs with less than twenty or thirty personal, one-on-one exchanges with professors. Where is the regularity in that? If you have ever been through the process of writing a dissertation, had an advisor who took forever to get back to you and you struggled your way through that solitary experience, I wonder if you would define that as “regular” interaction with a professor.

What is substantive and how does it impact grading practices?

Again, this has implications for interactions with students through grading and assessment. Is sending a test through a Scantron machine and sending it back to students substantive interaction? What about the instructor who just grades a paper and provides little to no deep feedback on the student work?

What does this mean for faculty autonomy?

If federal agencies are going to threaten fining Universities over whether this is happening and they were to actually apply this standard universally instead of picking on certain models over others, this would have a huge impact of faculty autonomy. It would be goodbye to faculty just doing their own thing. It could, ironically, lead to more centralized standards and control that are more common in institutions like Western Governors University, many for-profit online schools, and many non-profit online programs that have built similar systems. As with all things, there are affordances and limitations to this, but applying this standard across the board would shake things up. Of course, I see little intent or interest in doing that.

Why do traditional practices and programs get a pass?

While I’ve stated this already in some of the other questions, there is something really important in this question. Why is it that we have not seen any audits of existing programs on the basis of this “regular and substantive” expectation? What does this say about how people are using the policy? What is it really about? Does this have anything to do with academic quality or was it just an effort to maintain or protect a status quo, even though the statement itself can be used to challenge the status quo if it were used equitably?

Does this help educational innovations that truly have promise to increase student learning, engagement, graduation and more?

We know that faculty-student interaction can be an incredibly important factor. There are other really important factors as well. My question is simple. Does this current policy really help? How has it resulted in better outcomes?

Is it time to revise the policy?

Given all these points, might it be time to revise this part of the Higher Education Act of 1965? Perhaps we need new wording and a new vision for promoting and empowering a diversity of pathways to high-quality, high-impact, high-return higher education learning experiences. As it stands, I fear that the current policy and practice only stifles innovation while giving less than promising existing practices a pass.

Is There a Testing Crisis in Competency-based Education?

The more I follow the growing competency-based education movement, the more I am interested by certain decisions. Why are we so often limiting ourselves when it comes to assessment? It could be that we have an emerging testing crisis in competency-based education. Allow me to explain.

When the University of Wisconsin announced that they would be offering bachelor degrees based on competency testing instead of seat time and credits earned the old-fashioned way, there were many reactions. Some rejoiced that this was a promising step toward increased access and opportunity, and extending affordability of higher education. Others lamented the loss of the intangibles and deeply human side of instructor-student and student-student interaction. Still others decided to withhold judgment until they could learn more. Then there were others who couldn’t quite grasp what this meant or what it would look like. The UW model is not that different from many existing CBE programs. If you can show that you know it, you get the “credit”, and there are pre-established “assessments” to determine if you know it.

Yet, “assessment” is a loaded term. For some, that can mean projects, portfolios and a myriad of other ways to determine and document learning. For others, they mainly think about traditional tests. As such, if we look at the growing interest in competency-based education, I have a serious concern that some (not necessarily the UW model, I’ve not examined that one enough to know) are failing to be as innovative with their approaches to assessment of learning than they think they are being with their overall shift to a CBE approach. When this happens, all we are really talking about is allowing people to test out or progress by testing well.

Some join the conversation at this point noting that this is a very important topic. That is why we must become experts in test design, knowing that our tests are valid and reliable measures of student learning. Why focus on traditional tests? Even common approaches to assuring reliability and validity of tests do not go so far as to look at transferability. If you perform well on a test, does that mean you are highly likely to perform well in that domain in a real-world (by real-world, I don’t just mean work) context?

I call this the testing crisis because traditional tests (especially the common multiple choice, true/false types) are not the real world. We don’t hire people to take tests. The testing crisis is that we have created this entirely alternative world in educational institutions that do little to amplify the best of a school’s historic or contemporary identity and mission. Many tests are designed in ways that they are abstractions, one or more steps removed from authentic interactions with knowledge or skills.

Yet, they are scalable and that seems to be what people most value about them. You can assess people quickly and with measures of reliability and validity that most others accept. So, they are time-savers for the teacher with a large number of students. They support the ability for less personal and less intimate forms of education, although I acknowledge that some learning experiences can be rich and rewarding without being personal and intimate. As best as I can tell, these types of tests are and always have been efficiency more than anything else.

When it comes to  competency-based education or even traditional contexts measured by seat-time, I continue to plead with people to imagine the possibilities for their assessments. I recognize the time challenges of teachers with large groups of learners, but there are far more possibilities than many initially imagine. You can design a robust, rewarding and effective assessment plan by mixing a myriad of current and emerging practices. You can leverage projects and rubrics, performances and rubrics, papers and rubrics, narrative assessment and feedback, portfolio assessment, self-assessment tools, peer-assessment strategies, scenario and case-based learning with integrated assessments, many approaches to authentic assessment and more.

When it comes to the CBE movement, some argue that traditional and more standardized tests are an important part of the movement itself. These tests will allow them to validate this method and communicate the widespread impact and effectiveness. It gives thems the numbers that they need and want to defend their innovation. Yet, there are others ways to do this. We just need to take the time to more broadly and deeply explore and imagine the possibilities.

Competency-based Education: A Mission and Values-Driven Approach

Competency-based education continues to gain traction as a growing number of Universities are exploring it and we continue to see articles and blog posts build a public and collective conversation about the subject. I’d like to add an element to this conversation. It relates to our conception of what competency-based education looks like or should look like. As it stands, more people are trying out CBE by modeling themselves after a handful of groundbreakers. Benchmarking can be helpful, but rote imitation is no better than mindless memorization. We are wiser to innovate from a place of clarity and purpose; and that means starting with a clear mission, vision, set of values and goals. Instead of just doing what other competency-based education innovators are doing, each learning organization is wiser to start by looking inward before looking outward.

Allow me to give a personal example. When the team at my University decided to explore the possibilities of a competency-based approach to our graduate program in Educational Design and Technology, it did not start with researching SNHU, WGU or any other school. Yes, we were informed about their model and practices, but our focus was not, “How do we build another WGU?” We looks at the needs and goals of the learners that we currently served and aspired to serve. We examined the needs of those who do or might hire those students who go through our program. We considered the distinctives and core values of our University. We also candidly talked about the type of social good that we aspired to achieve through the program. Having worked through those questions, then we started selecting options for how to re-imagine things.

Yes, while we were doing all this formal and informal preparatory work, we were also experimenting with different innovations like digital badges, open education resources and textb00kless classes, different forms of assessment, and the like. Yet, none of these were technologies that we decided to use in advance. We wanted the tools that would enhance your mission, vision, values and goals. That meant imagining something completely different from any other competency-based program in the United States or beyond, as described in this past article.

For us, we wanted something that addressed a number of considerations. As such, the following list represents an important consideration or feature for us followed by how that impacted the design of a competency-based program that is uniquely focused upon project-based learning, mastery learning and personalized learning.

Desired Feature – The curriculum needed to be deeply practical and applied but rooted in great ideas and research.

Design Decision – Every assignment or assessment would have real-world relevance. It could be used or shared with people in a real world context and have or create value in that context. As such, traditional tests were out.

Desired Feature – Students learned by doing, and we wanted that doing to be as authentic and contextual as possible.

Design Decision – Whenever possible, students would learn from student-centered, project-based learning activities that also measured and documented student competence.

Desired Feature – Students should be able to recognize the discrete knowledge, skills and abilities that they were developing throughout the program.

Design Decision – The curriculum would be broken down into small and discrete competency-based digital badges that allowed students to see what they have learned and how they are progressing in the curriculum (learning journey).

Desired Feature – Students could demonstrate progressive competence to employers or others rather than relying upon abstractions like course grades, credits and a diploma.

Design Decision – Students would earn digital badges that include specific criteria that would have meaning in learning organizations. In addition, students would build a massive portfolio of work from which they could pull to demonstrate their knowledge and skill.

Desired Feature – Students would create evidence of their learning that could also be used to address real needs in their world or in different learning contexts and organizations.

Design Decision – While core competencies are universal in the program, there are elective competencies where students could specialize. In addition, students are able to demonstrate competence through projects that they personalize to address needs in one or more personally meaningful real world contexts. 

Desired Feature – There would be no “busy work” or effort extended by the students that was not connected to a real need in the education space.

Design Decision – Abandon the use of traditional papers, tests and assessments that resemble work that would only make sense in a classroom context. As much as possible, all student work and activities would have meaning and value even if it were done outside of a formal graduate program.

Desired Feature – Readings and learning activities would be personalized, allowing individual students to into readings that helped them progress toward program goals but also specialize in fields are areas related to their current or future work. In other words, a person aspiring to be an instructional designer in higher education and another in the same class interested in middle school teaching would have personalized reading lists and learning activities based upon these differences. 

Design Decision – In general, move away from a canonical approach to readings. Make sure students are exposed to core theories, people, and concepts and any seminal works; but apart from that, have a repository of readings from which students can select (and to which instructors/coaches can direct students) along their journey toward competence in a given area.

Desired Feature – Community and feedback is a value but it must not turn into a dictator. 

Desired Feature – Students should receive personalized coaching as they progress through the learning experience.

Design Decision – Apart from the required assessments for competencies and weekly group discussions, the instructor is not able to require standard activities for the entire class. Learning plans should be personalized. 

Desired Feature – Reflective practice needed to be nurtured and emphasized throughout the learning experience in order to develop increasingly effective reflective practitioners.

Desired Feature – Whenever possible, students should not be penalized for needing more practice, more feedback, or having less background knowledge than others in the class.

Design Decision – Apart from “grading” of weekly discussions, everything else will be built upon a mastery learning approach. Students can revise and resubmit as much as necessary to reach competence, granted that it is within the formal time period for the course.

Desired Feature – Scalable is nice but as long as it is financially viable, we will go with the model that best meets these other priorities.

Design Decision – We will not revert to objective tests and assessments just because they are scalable or achieve some sort of abstracted form of reliability and validity. It has to have meaning in the real world and produce a have a high level of confidence that the learning is transferable to real world contexts. 

Notice how our values and convictions led to the design of an entirely different approach to competency-based education than what we see in many schools today. That is because it was mission and value program and not a cookie cutter approach. I’m convinced that the broader CBE community could benefit from a larger dose of this approach. What do you think?

Are Some Learning Pathways Superior to Others? Why this Matters for #CBE

Are some learning pathways superior to others? When it comes to competency-based education (CBE), people point to personalized pathways as an affordance of this approach. Many traditional teaching and learning contexts have prescribed pathways toward reaching a given learning objective, one established, guided and controlled by the teacher. Everyone goes on the same pathway together, led by the teacher. CBE leaves greater room for the possibility that different learners will go on different journeys toward demonstrating competence. A learner with significant prior knowledge might be able to take a shorter pathway to competence than someone new to the field. One learner might opt for more practice exercises, more readings, more one-on-one coaching sessions, or a variety of real-world experiences intended to help one make progress toward competence. One learner might need one or two laps around the track, while another might need twenty before reaching competence.

This is, from my perspective, a core affordance of competency-based education. Once we are able to articulate with clarity what it means to be competent, then we are well-positioned to start thinking about multiple learning pathways to that single destination. Yet, this is far from a universally accepted understanding of education for several reasons. One reason is that some pathways are deemed superior to others even in the absence of solid data to support such claims. A second is because critics are often concerned about trying to reduce learning to a discrete list of competencies. That risks of losing the forest by chopping down all the educational trees and placing them in nicely organized piles. Yes, there is a qualitative difference between a forest and a lumber yard. Critics argue that CBE often ignores such distinctions. Another is that CBE seems to unbundle or strip learning from social interaction, community, and a culture of learning that some value as much as they do any demonstrable outcomes of the experience.

Yet, as I look at the types of assessments and requirements established in a growing number of competency-based education programs in the United States, I often see as many limitations on learning pathways as I do in traditional learning contexts. Schools are prescribing the pathways that people must take to demonstrate competence with a level of detail that makes it resemble the traditional methods. They often do it in the name of scalability and efficiency. For example, if you are in a CBE MBA program, you can’t necessarily demonstrate your competence in financial analyses of businesses in multiple ways. You must do so in the form of an established business simulation. The simulation is the assessment, but it also becomes a significant part of the learning process…the pathway. Students might take varying levels of time to prepare for the simulation. They might leverage slightly different readings in preparation. Yet, many end up on the same general learning pathway.

Not all pathways are equal. This comes back to something that I repeat often, that education is not just about measurable results, data and evidence. It is also about deeply held beliefs, values and philosophies of education. Educators don’t choose one learning pathway for others over another because they have carefully analyzed pools of learning data and decided that this pathway is objectively superior to others. Some might do it that way, but that is in the minority. Most do it because they prefer, value or are attached to certain pathways (and some take offense at being forced to defend and articulate their reasons for a given pathways). They often can’t fully articulate why learners should follow the learning pathway that the instructor has established. They just believe that it is important, maybe even fundamental to the task at hand. “Learners need to learn and experience this in community,” we might argue. “You can’t truly grasp this through an online learning experience,” another might explain. “There is something important but intangible about doing this in small groups or a workshop,” yet another will defend. “Without work through these specific seminal works, the learners would be ill-educated on this topic”, the teacher points out.

Even as we witness the great unbundling of education, there are still many educators who reject the unbundling. Some never thought of their lessons in categories like learning objectives, learning pathways, and learning assessments (formative and summative). Some haven’t even thought of what they do in terms of lessons (especially in higher education). It was all just teaching and learning. It was content-driven, experiential, social, an art shaped by an autonomous artists known as a professor or teacher, a blend of these, or perhaps several other perspectives.

I suspect that this is why the debate around competency-based education remains tense at times, limited at others. The CBE conversation seems to be growing more slowly than some expected. It is a massive disconnect for many educators because it hardly resembles their careers and callings as they have understood them. They find it difficult to imagine losing the many qualitative benefits of what they do now and are perhaps offended (or frightened) by the claim that these competency-based education alternatives offer a comparable or equally valuable education to people. Not all pathways are equal and desirable. Even changing the pace of the pathway for different students is not agreed upon among educators. As such, if we are going to have a rich and valuable conversation about the affordances and limitations of CBE, perhaps we are wise to spend more time examining the role of learning pathways.