Our Approach to Assessment is Holding Us Back in Schools

Back in April 2016, I enjoyed serving on a panel at an event hosted by Educause called Exploring the Next-Generation Digital Learning Environment: Opportunities and Challenges. My panel included three people. One focused on learning analytics. A second concentrated on student advising. I was given the area of learning assessment. The connection between these three areas is that they all rely on learner data. As such, all of this was in the context if what some refer to as the Next Generation Digital Learning Environment.

Talking about learning assessment is a bit tricky because not everyone works from the same definition of assessment. However, consider the following two uses of the word.

Assessment is a process of determining “what is.” Assessment provides faculty members, administrators, trustees, and others with evidence, numerical or otherwise, from which they can develop useful information about their students, institutions, programs, and courses and also about themselves. This information can help them make effectual decisions about student learning and development, professional effectiveness, and program quality. Evaluation uses information based on the credible evidence generated through assessment to make judgments of relative value: the acceptability of the conditions described through assessment. –

The National Academy for Academy Leadership 

Compare that with the following explanation from Duke’s Academic Resource Center.

Assessment focuses on learning, teaching and outcomes. It provides information for improving learning and teaching. Assessment is an interactive process between students and faculty that informs faculty how well their students are learning what they are teaching. The information is used by faculty to make changes in the learning environment, and is shared with students to assist them in improving their learning and study habits. This information is learner-centered, course based, frequently anonymous, and not graded.

Evaluation focuses on grades and may reflect classroom components other than course content and mastery level. These could include discussion, cooperation, attendance, and verbal ability.

While there are varying uses of the term, here we see that some use assessment as a non-evaluate and largely formative process. Its purpose is to help teachers refine and improve the learning experience for students.

Yet, if we look at various formal assessments and their uses throughout history, we see another perspective. In some instances, we see that assessments were largely used to identify, weed out, and qualify. Consider that the multiple choice test that dominates so many assessments in learning organizations, was used to test the intelligence of WWI soldiers. It did not initially get traction but was applied in school settings as the education system in the United States was becoming industrialized, with more of a mass production mindset.

Where assessments might have been a means of weeding out in the past, today our needs are different.  The college degree is like the high school diploma of the past to many. With the doors opening to college and the desire to expand formal education in the broader population, the goal is less about weeding out or restricting. It is now about identifying, supporting, maximizing impact, empowering, and even personalizing learning. This is an important and significant shift in the way we use and think about assessment, even as some still think of it in this older sense.

As I’ve mentioned many times before, I tend to focus upon formative assessment because it is focused upon that which can help learners right away. It is the checkup at the doctor compared to the autopsy. The checkup allows us to suggest changes and prescriptions to improve a patient’s quality of life. When you get to the autopsy, that is no longer a consideration. It is just about getting insight that might help the next person.

When it comes to assessing learners, however, there are many perspectives to consider. There is assessing progress toward a standard or objective. This is about seeing how close or far one is from meeting that standard or reaching that objective. Then there is assessment focused on personal or relative progress. It is comparing how one progressed from where that specific person started. Did that person grow significantly or not? Then there is assessment focused on the nature of the progress, progress compared to some expected rate or type of progress (a typical student would be here by now). Next there is assessment interested in the process of the learning. It is not just about what you have learned but how you have learned, giving teacher and student insight into successful or less than successful strategies.

I contend that our approach to assessment must continue to change. If not, it will prevent us from accomplishing the broadening aims of education in the contemporary world. It will inhibit access and opportunity. It will also fail to help each student maximize personal growth and development.

 

Education is the Dangerous Business of Influencing People

A wise mentor once explained to me that, if you are in education, then you are in the business of influencing people. If you don’t like that, then you should find another field. While I couldn’t deny the truth in what he said, I didn’t like hearing it stated that explicitly. It took me years to figure out why I disliked it so much, but now I’m ready to explain it, even if ever so briefly.

Education is about growth. It is about personal formation and development. Yet that growth can take many forms. You can grow into a downright cruel and evil person. You can grow into a coward who runs away from the slightest challenge, even if it is for the grandest of causes. You can grow into a cold and calculated criminal. You can also grow into a self-controlled, confident, courageous, wise, and compassionate person. In other words, the direction of one’s education matters.

When we claim that someone is educated, what do we mean?

Most people probably use the term “educated” in reference to a person who is literature, informed, knowledgeable, and maybe even having acquired a certain set of credentials, typically including a high school and college diploma. Others usually reserve “educated” for people who are well-read and well-rounded in their knowledge of some valued cannon of knowledge, perhaps the Western classics or a set of words like the Harvard Classics.

Travel the world and ask people in different cultures what they mean by this term and you will probably find a few commonalities but also plenty of differences. You don’t even need to travel far from home to discover differences. Just interview people in a diverse community within the United States. This is because we do not agree upon the aims of education, at least not all of them. We can and do find common ground. In other instances, countless parents and students just don’t ask the question. They submit to the aims affirmed by the educational establishment or, in some places, the political establishment.

The Reason for My Discomfort

This is why I was uncomfortable years ago when that mentor drew my attention to education being a “business of influencing people.” First, I was never completely comfortable referring to education as a business because my impression of business was one that made financial gain a primary purpose. Yet, it was the “influencing people” part that troubled me.

Don’t get me wrong. I try to influence people all the time. I even seek to do it in this blog. I try to challenge myself and others to consider affordances and limitations, and to expand their awareness of the possibilities. I believe that the self-examined life is worth promoting. I value agency over mindless compliance and conformity. I also believe in the existence of classic concepts like truth, beauty, and goodness; and I value them over falsehood, ugliness, indifference, and evil. In that sense, as much as I embrace change and champion educational innovation, I still find an intellectual home in classical antiquity.

My issue is with establishing mandatory educational institutions that seek to push forward their agenda at the expense of other legitimate alternatives, even though we find ourselves disagreeing today over what is a legitimate alternative. We do this by claiming that some educational establishments should be government-funded while others should not. We do it by establishing policies that favor our agendas and muzzle those of others. We find our schools turning into political battlegrounds. We don’t value the pursuit of knowledge wherever it leads, but only wherever our agenda deems it appropriate to lead. We celebrate and seek to secure power and influence in education more than having a candid, substantive, persistent, and transparent exploration of ideas.

When we do this, we are not engaging in mere influence. We are dangerously close to turning schools into places shaped by propaganda and manipulation. We find ourselves using the very same tactics employed by middle school cliques or bullies. We use subtle or direct name calling. We use leading questions that would cause most to feel like fools if they did not follow in our desired direction. We push people to accept and move on instead of giving time for doubts, questions, and exploration. We muzzle dissident voices and evidence that does not support our cause. We can do this in wonderfully subtle and intellectual ways. We can use words and body language that aspire to civilize these efforts. We value ideological ends and wins over empowering people with a growing sense of agency and a robust intellectual toolkit.

This is why I struggled with the idea that education is the business of influencing people. It is because, while I can’t deny the truth of the statement, it must be kept in check, at least in a society of that values freedom. This is not easy. There are countless challenges along the way. It is, nonetheless, worth the effort.

Boredom in School As Preparation for the #Workplace

While riding in the car with my family, I mentioned something to my 9-year-old son about the fun and excitement of school. I don’t remember the entire context, but it didn’t take more than a couple of seconds for my son to reply with what he deemed an important reality check for me.

“Dad, everyone knows that school is boring.”

“Everyone?”, I asked.

“Yes, everyone knows this about school, dad. School is boring for everyone.”

I challenged his assumption. “So you are saying that your Tae Kwon Do class is boring? Your hip hop dance is boring? Your Lego club is boring? How about your explorations and hikes in the woods? Or what about…

“Those are not school”, he explained.

“Why not? Why can’t they be part of your school? Isn’t school just a place where you learn?”

The interesting part is that my son does not go to a traditional school. He is home schooled. He was describing his impression of the traditional school as he experienced it in previous years, as he heard about it from friends, and as he saw it represented on various media. School is something that people do to you. You have limited choice. You do what others say. You work hard. You focus on things that are neither fun nor interesting. That is his impression of school.

I didn’t feel that way about school. I loved school, even the “boring” classes. I didn’t always enjoy the classes and I was not sure why some teachers persisted in being teachers when they seemed to be so unhappy with it. I wished that we could have more rich conversations, do research, conduct experiments, and do lest test preparation (and I had very good school experiences). Nonetheless, school was where could feed my insatiable curiosity. Plus it was a place where I got to be around other people my age.

Yet, this idea of boredom is obviously not limited to my son and a few of his friends. Depending up the study that you review, somewhere between 25% and 75% of students think school is boring. This is true on the college level as well. In a study reported in 2016 in the Journal of Media Education, over 85% of the students surveyed reported checking their phones during class for non-class purposes, and one of the main reasons for doing so was boredom. In fact, 63% of those surveyed reported using their phones to “fight boredom.”

Boredom is not only in school, of course. Just do a quick online search about “boredom in the workplace” and you will find a long list of articles and studies. Plenty of people go through school bored. Then they get a job and are bored. As such, I suppose that, as the cartoon at the beginning of this article indicates, maybe school is preparing students for all of those boring jobs.

Yet, there are debates about the cause of boredom. Some point to the environment. Others point to the person. In this chapter on Burnout, Boredom, and Engagement, the author explains this.

…it is suggested that the experience of boredom at work is due to an internal need for high stimulation; the greater this internal need, the more susceptible one would be to feeling bored (Farmer & Sundberg, 1986). Both perspectives define boredom in terms of its antecedents, so these definitions are circular: employees feel bored because they work in boring (monotonous) jobs or because they are boredom-prone by nature. In order to avoid this circularity, Loukidou, Loan-Clake and Daniels (2009, p. 383) define boredom simply as an ‘unpleasant and deactivated affect’. However, this description is rather narrow and unspecific because it limits boredom to a mere affect and does not refer to the work context. We therefore propose to follow Mikulas and Vodanovich (1993, p.3) and define boredom at work as an unpleasant state of relatively low arousal and dissatisfaction, which is attributed to an inadequately stimulating work situation. For a description of boredom see the story of Geoff – a bored assistant (Work Psychology in Action box).

Schaufeli, W. B., & Salanova, M. A. R. I. S. A. (2014). Burnout, boredom and engagement at the workplace. People at work: An introduction to contemporary work psychology, 293-320.

People care about this because boredom can lead to any number of negative outcomes. People are not engaged. They are less productive. They don’t stick around. It puts a damper on the culture. And let’s not forget about the story of The Little Boy Who Cried Wolf. Yes, maybe the main point is about telling the truth, but why did the little boy lie in the first place? In many versions of the story, he came up with the idea of crying wolf because he was sitting out there alone, with a bunch of sheep, and he was bored. So, boredom conceived and gave birth to lies, and we all know how that story ended.

This is not the first time that I’ve written about boredom, but now I’m extending the reflection from school to work because I can’t help but wonder about the connection between the two. Could it be that schools conditions people to settle for boredom in work. Or, maybe students don’t learn the many secrets to move from boredom to engagement in school, and so they are stuck in what sometimes turns into a lifelong cycle.

Whatever the case, I see no reason why we would need to settle for boredom, especially in the long-term. I don’t get bored. I decided not to get bored over twenty years ago. It isn’t that I don’t find myself in situations that many might consider boring. It is just that I choose not to let the situation determine my mood. Perhaps that is what some students are doing when they pull out their phones for a distraction, but there are more productive ways to get at this too.

At the core, however, I tend to agree with the quote often shared on the web and attributed to Ellen Parr (which, by the way, sparks my curiosity since I can’t seem to find any definitive information about an Ellen Parr…now that is a mystery to be solved). “The cure for boredom is curiosity. There is no cure for curiosity.”

Okay, I confess that, after finding that quote, I just spent thirty minutes searching the Library of Congress, Worldcat, Google, and a library meta search engine to find out something about this Ellen, but I was definitely not bored, just a little distracted.

This reminds me of an experience from a graduate course years ago in interpretive methods in research. The professor invited two others to co-teach the class. So, we had the professor who was an expert in qualitative research. There was a philosopher. Then there was a Baptist minister from the Caribbean. Add the fact that you had me, an elementary school teacher, a young student who identified with Marxism, and a few other curious students; and it made for quite the learning community. Now back to the story… We were discussing phenomenology and at point, the professor made a declaration that he and I were an unusual sort because both of us could probably be content studying toll-booth-ology. It was his way of suggesting that we can be interested in pretty much anything.

Maybe that is just a genetic or largely fixed trait. Maybe it is that some people are curious and others are not. Curiosity is my go to way of living well beyond boredom and it works for me. Yet, I’m pretty sure that there are a dozen other ways too. It is just a matter of finding it, embracing it, and living it. Yet, I don’t think we are wise to just let boredom take over. It is too destructive of a force…in school, in work, and in life.

Yet, not everybody agrees with this. Or rather, some warn about the dangers of striving to avoid boredom at all costs, filling our minds with whatever entertainment might save us from that moment of boredom. In the Nicholas Carr’s The Shadows: What the Internet is Doing to Our Brains, he warns that media can be like a drug that we use to numb ourselves from pain, boredom, or any other unpleasantryof the mind. Bertrand Russell argued for the virtue of boredom in his book, The Conquest of Happiness. He wrote:

We are less bored than our ancestors were, but we are more afraid of boredom. We have come to know, or rather to believe, that boredom is not part of the natural lot of man, but can be avoided by a sufficiently vigorous pursuit of excitement.

Many point out that the effort to avoid boredom can lead us into bad habits, it can dull our minds, and it can prevent us from achieving many rich and rewarding goals. Some goals and efforts in life are not easy. We sometimes have to learn how to persist through the monotonous to reach the glorious, and that is an important life lesson as well.

Some things that are less interesting are nonetheless important and worthy of our effort. Some tasks are tedious but persisting though them can be worth it. What I don’t accept is that we should settle for boring, that we should lift it up as a virtue. Boredom itself is not admirable so why settle for it? Instead, why not learn how to see the meaning and value in sometimes less than interesting tasks, therefore removing the boredom? Meaning, like curiosity, defeats boredom with ease.

Then there is the relational angle. Sometimes we engage in activities that some might consider boring when they are alone. Do it with a group and it takes on new meaning. Sometimes things spoken are more interesting because of who says them and our connection with them. As such, if we build rich and positive relationships with the people around us, boredom also has a way of losing its hold on us. That is because we are relational beings, and the experience of being in community with other people enriches us in countless ways. So, now we have a third means of escaping boredom.

There are plenty of other paths and I’ve already spelled out at least four ways through and beyond boredom. We can be curious. We can see the daily in terms of how it helps us achieve personally meaningful goals. We can see and find meaning in what we are doing and/or why we are doing it. We can cultivate positive relationships and a sense of community with the people in our school or workplace. So, how about if we don’t settle for boring in school or in work? How about if we instead set our eyes on curiosity, meaning, personal goals, community, truth, beauty, and goodness?

The One-Question Proficiency Exam & Proficiency Versus Growth

In the 2017 committee hearing for Betsy Devos as a candidate for the next Secretary of Education, Senator Al Franken of Minnesota asked Devos about her position on proficiency versus growth in education. Regardless of your views on Devos or Franken, this was one of the more interesting questions at the hearing, even if it was not a great forum to deeply explore such a question. I realize that such hearings are not simply part of a vetting process. Committee members are also using it as platform. Yet, putting that aside for a moment, I’d like to focus on Franken’s question.

I contend that it is a good and interesting question to start a discussion. At the same time, it can also be used to create a dangerous and overly simplistic dichotomy. This cannot and should not (I don’t use that “should” word too often in my writing) be an either/or debate. The answer is yes to both. Proficiency measures are useful in education in some contexts and for some purposes. Measures of individual and group growth are also useful.

In fact, Franken’s one-question proficiency exam for Devos is a good illustration of one limitation of the proficiency approach in education. Our assessment questions are too often lacking context. We set a standard and proceed to evaluate people on the basis of that standard. At times we look for yes or no, black or white and while that is sometimes present, it often does not tell the entire story. We risk tricking ourselves into thinking that a person’s answer to a small number of questions, questions that can indeed be better understood and answered with more context, should be the main basis upon which we judge and evaluate people in school and elsewhere. You answer the questions according to my standard, in my format, and with my timing, or you don’t make the cut.  You lose points for asking questions or seeking clarification. I’m talking about what too often happens in our schools, not necessarily what happened in the hearing.

How would you answer the question? Do you have leanings toward one side or the other? Or, do you find yourself somewhere in the middle?

Just in case you are not aware of this education discourse, allow me to briefly explain. Imagine that you are learning to ride a bicycle and I have the task of documenting what you learn. I can create a list of standards for riding a bicycle and then I can evaluate your performance according to that standards. As such I rate you on some sort of proficiency scale. It might be broken down into a list of standards so that I can give an even more granular assessment of your proficiency according to those standards. Or, I can measure you on the basis of progress. One person might have prior experience on a bike while another may have never seen the bike before. It isn’t just about whether you can ride a bike yet. It is also about how much you’ve learned and grown since you started. Are you making progress or are you stuck? The growth approach is less focused on meeting this universal set of standards. It is more interested in individual progress along the way.

When looking at schools, we can have a set standard for proficiency in subjects like math and reading, and we can lament the fact that 70% of the students in a school are not proficient as determined by some objective assessment.  Yet, what if 100% of the students in that school started at 0% proficiency and progressed from there to nearly proficient? That looks like a failure on the proficiency scale, but it looks much more positive if we can see how much they had grown from the beginning of the year. Do we want them to be highly proficient in reading? Yes, we do, but we are wise to recognize that they did indeed make important progress in their learning and, given adequate time and support, may far exceed some external measure of proficiency (even if the standard are not explicitly used or stated in the school).

Of course, both proficiency and growth measures have their place in education. There are times when we want to know who can ride a bike and who can’t. We also want to know how people are growing and developing along the way. I want my doctor to have passed a rigorous set of proficiency exams before doing surgery on me. At the same time, we know that many doctors, after getting out of medical school, don’t necessarily grow from there. After a decade or two, some plateau or even decline in their knowledge and skill. We want them to grow, not just according to some set standard. We want them getting better every year that they practice medicine.

Oftentimes we establish narrow measures of proficiency for students. We not only set standards that students are supposed to meet, but we determine the age or grade at which they should meet them (Devos mentioned this in her response to Franken). Yet, that can sometimes be a deterrent to student growth and development. Students don’t learn at the same level. Some students might hit the standard and then check out, thinking the task is complete instead of striving to learn something new or best themselves on the next learning challenge. Still other students struggle to hit the standard and are focused almost exclusively on the idea that they failed or missed the mark. This can be deflating and de-motivating for some who see the standard as so far out of reach that it isn’t worth the effort. This might prevent some from recognizing that they are growing and making important progress. With time and effort they can far exceed any standard.

There are times when it is best to remove the standards and simply challenge people to become increasingly better, to solve a problem, face a challenge, immerse themselves in a task or project, play, experiment, or explore. Even without carefully defined standards and measures according to those standards, many people learn to read and speak, a couple of rather important life skills.

Yet, feedback is a powerful and useful too for learning whether we are in a learning context that is standards-based, personal growth and progress oriented, or a blend of the two. In fact, if I had to answer Franken’s question for a hearing, that is probably how I’d approach it. I’ve seen learning contexts that focus upon standards that serve students well. I’ve seen learning contexts that focus upon student growth and serve students well. I’ve seen others that seek a blend and balance of the two that do well by the students. Yet, in each of these contexts contexts they have a shared commitment to some form of valuable feedback.

When it comes to education policy on a state or national level, should the focus be upon measuring each student’s growth and progress over time, or should it be focused upon students meeting explicit and stated standards for proficiency? That may have been where Franken was going with his question. After all, we know that education policy can drive schools to focus on one or the other depending upon what the policy expects a school to measure and the “motivators” attached to those measures. We also know that the emphasis has significant implications for the school culture, the student experience, and the mindset that develops in many students and teachers. I understand why some are drawn to clear-cut standards and measures of proficiency based upon those standards and there is a time for that, but most of us also know that a rich and transformational education is more than being labeled proficient.

A hearing for Secretary of Education is not an optimal context for a rich and robust exchange on such a topic. Given the time to tease out important nuances to that type of question, I suspect that many of us would find ourselves appreciating Franken’s perspective as well as that of Devos. These are not easy issues, especially when a small change in policy can create a seismic shift in school focus.