Why Some Educational Technology Efforts Decrease Student Learning

“There is plenty of research to indicate that educational technology does not improve student learning.” “Many studies show a ‘no significant different’ result for educational technology.” Studies indicate that educational technology can moderately or even significantly increase student learning.”  Depending upon the person, place and time, it is possible for you to read or hear one or all the above statements. Which one is correct?  All of them.

Educational technology, as we all know, is not a single practice or entity.  There are hundreds of thousands of different educational technologies. Even with a single educational technology, there are countless factors that influence the impact (or lack thereof) of integrating that technology in a learning environment.

To say that educational technology is effective or ineffective is a bit like claiming that tools are effective or ineffective at fixing things around the house.  Which tools?  Which project? How skilled is the person using the tool?  How is the too being used? Or, it might be like saying that books are effective or ineffective at communicating content.  Which book?  Which content? To whom and under what circumstances? These and more questions are important aspects of considering the potential impact of a given integrating technology initiative.  It is too general to simply state that SmartBoards do or do not improve learning, that 1:1 programs increase differentiated instruction, that iPads are the key to bridging the achievement gap, that the flipped classroom with result in students functioning at higher levels of Bloom’s Taxonomy, or that online learning matches or exceeds student performance in face-to-face classes.  It is possible that each of these statements may be true in a given case, but it is equally possible that they could be false.

With this in mind, how do we engage in integrating technology efforts that actually improve student learning?  Here are eight tips, focused upon answering why some efforts have little or no impact, or that they decrease student learning.  Take care to heed the insights from this list and you are much more likely to use educational technology in a way that enhances and improves student learning.

1. The Novelty Effect – This is a well-documented phenomenon.  Adding a new or novel technology to an environment might have an initial positive impact upon student learning, but the benefit quickly fades as the technology becomes more ordinary to the learners.  Our brains attend to novelty, so adding something new is a way of gaining the attention of the learners, hence increasing the chance of learning.  However, as the technology ceases to be experienced as novel, it also fails to keep the attention any better than classroom activities without the technology.  As a result, the justification of technology initiatives needs to be based upon something more than simply that it is something new, especially given the consequential cost in time and money that goes into many educational technology efforts.

2. Bad Instructional Design – How we design a learning experience that impacts what students learn, which students learn, and how much they learn.  There are many models and approaches to instructional design, but I argue that any good design needs to ask, answer, and take into account six basic questions (borrowed from multiple ID models as well as DuFour & Eaker’s work on Professional Learning Communities).

What do I need to know about the learners?  “Know your learners” is fundamental to any good message, speech or lesson.  A well-designed lesson for one audience might be terrible for another audience.  Age, demographic, prior knowledge, attitudes and beliefs…all of these and more can give helpful insights into designing a high-impact lesson.

What do I want them to learn?  Without a target, it would be hard to measure whether students learned what you wanted them to learn.

How do I make sure that students are interested and attending to the learning? Without attention, we don’t learn.

How will I know when they learned it?  This is where you decide upon an assessment or means of measuring whether they met the stated learning objective.

How will I help them learn it? This is where we select the best methods, strategies, resources, and activities that will aid students in meeting the stated goal(s).

What will I do when/if some students are not learning it? As a teacher, it is easy to feel like all is well when only half of the students are “getting it”, but that means that half or not. What is the plan for that other half?

In my years of consulting with schools and teachers, I almost always find that the problem with student learning resides with a failure to ask, answer or apply insights from one of these questions.  The best technology in the world will not improve student learning unless it we dress it up in instructional design attire that is appropriate for the occasion.

3. Cognitive Load Problems – This likely fits with #2, but given that I often see it as the specific problem in classes, I chose to list it as a separate item.  The working memory in the human brain can only handle so much at a time. Overload it and you risk decreasing learning, confidence and attention.  That is a recipe for decreased student learning.  Consider a math teacher who adds PowerPoint to the class.  I have seen classes where students are directed to frequently switch attention between a textbook, notebook, mobile device, the projected PowerPoint as well as worked problems on a traditional white board.  This requires students to determine what to look at and when.  Especially when it is a complex task, this switch tasking will leave students with less mental energy to focus upon the real learning.  So, as we add educational technology, it is important to take into account the impact of potential switch tasking.

In the cognitive load literature, we often see the distinction between extraneous, intrinsic and germane load. Extraneous load comes from how you design the lesson and/or how the content is packaged.  For some tips on addresses it effectively, I have seen no better source than Ruth Clark’s work on the subject. With regard to intrinsic load, that is the load that comes with a given task.  Some tasks are more complex and so they naturally take up more of a load in the working memory.  The more complex the task (and the larger the intrinsic load), the more important it is to manage the extraneous load.  Germane load is another item that we can address. This refers to the load required to process the information, to make sense of it, and to create new “schemas” in our brains.  Effective teachers can help to manage both germane and extraneous load.  For more information about that, consider help from this recent and excellent article on the subject.

4. Make it About the Technology & Not the Learning Objectives – Educational technology can be like a shiny new toy, and I find instances where that takes the place of a focused learning experience with a clear goal or objective in mind.  I recall one instance where a calculus teacher excitedly showed me how he integrated this new web site into his class.  The site was rich with activities for teaching math…but they were all focused upon basic math, nothing that remotely resembled the goals of a calculus class.  When I mentioned that to him, his dismissed it, noting that they were having a great time.  That seems like a prime case of making the technology usage the goal and not staying focused upon the purpose of the class, unit, or lesson.

This is why I express some concern about a popular graphic that is spreading around the web, the pedagogy wheel. This is a graphic that seeks to line up apps with different parts of Bloom’s Taxonomy (and the SAMR model, something that I will describe in a moment). The problem is that most of the apps listed in the chart can be used in many ways, addressing almost any level in Bloom’s Taxonomy.  To me, this chart is too simple, and it risks leading teachers to thinking that simply using a given app results in students functioning at a given level.  That disregards the critical “design” considerations that I mentioned before. It is not just what you use, but how you use it.

5. Replacing Higher Level Learning Activities with Lower Level Learning Activities.  Think of Bloom’s Taxonomy. The lower levels deal with tasks like remembering and understanding.  The higher level tasks involve applying, evaluating, and creating.  In many elementary and secondary 1:1 programs, I see teachers excitedly using new subject-specific apps with students that were little more than drill and practice exercises (or the equivalent of digital worksheets). In many cases, these replaced previous low-tech activities that involved students applying their knowledge or even learning through creation.  In other words, by using the app, they significantly decreased the challenge of the learning activity, spending more time at the lower levels of Bloom’s Taxonomy.

6. Replacing Lower Level Learning Activities with Higher Level Learning Activities – I also see the opposite problem from #5. In these instances, the educator uses the technology as a tool for project-based learning, learning through creation, or some other task that requires functioning at the higher levels of Bloom’s Taxonomy.  In doing so, the teacher removes time that she previously spent helping students gain foundational facts and understandings about the topic.  This leaves some students struggling and confused with the project, resulting in poorer overall performance for a given populations of the students.  Project-based learning and learning through creation are both wonderfully promising activities, but there is need to plan and prepare for ensuring that students gain the necessary prerequisite knowledge to thrive in such a learning environment. This can be done in a number of creative ways, and it is sometimes as simple as directing students to resources that help them with the “basics” when and if they need them.

7. Using Technology as a Simple Substitute for Previous Technologies or Resources – Dr. Rueben Puentadura conducted research on technology use in classrooms and produced a helpful model for thinking about the different ways in which technology is used, giving keen insight into how given usages do or do not impact student learning. You can review much of his work here. His model breaks up usage scenarios into four categories: substitution, augmentation, modification, and redefinition. The substitution level is simply using one technology as a substitute for another. This might be like a teacher who gets a SmartBoard and uses it just like they did a white board.  Augmentation is when they use the SmartBoard mainly as a high-tech white board, but they start to dabble with some enhancements not otherwise available with the whiteboard (like the fact that you have nearly unlimited white board space on a SmartBoard).  Other than as a result of the novelty effect, we would not expect significant improvements in student learning with substitution or augmentation.  However, when we get to modification and redefinition, that is where we start to leverage the technologies to do things that were not easily done or even possible without the technology.  It is at those levels (granted carefully and intentional planning) that one just might get at enhancements and improvements in the learning outcomes and experience. There are many videos and articles about this topic, but the resources on Dr. Puentadura’s web page are a great place to start.

8. Not Addressing Motivation – Careful consideration of all the above items might still leave us with little to no effect with a technology integration. In that case, I will draw out yet another aspect of good design, planning for student motivation.  This might even be an “ethos” or culture problem in a given class or school, where student’s just do not come to class with much interest in learning, regardless of the lesson, method, or technology.  For those instances, I offer two considerations/models for starting to tackle this formidable foe.

The first is something that I often mention on the blog, the PERMA model (popularized by Martin ). While this is not specifically about motivation, it is about cultivating well-being. A culture of well-being in one where motivation is likely to be less of a challenge.  Here is a resource for getting you started on the PERMA model applied to education.

Keller’s ARCS Model. What leads to increased motivation?  According to Keller, it is increased attention, relevance, confidence and satisfaction.  A learning environment rich with student’s experiencing these four items is one where motivation is likely high, leaving students much more prepared to learn.  Here is a quick primer on the ARCS model.

In the end, the simple reality is that some technology integrations work and others do not.  There is an experimental aspect to teaching and learning.  It is both art and science.  Nonetheless, I contend that taking into account these eight elements will greatly increase the chance of using technology in a way that actually improves or enhances student learning.

Posted in 1-to-1, blog, education, educational design, educational technology

About Bernard Bull

Dr. Bernard Bull is an author, host of the MoonshotEdu Show, professor of education, AVP of Academics, and Chief Innovation officer. Some of his books include Missional Moonshots: Insights and Inspiration for Educational Innovation, What Really Matters: Ten Critical Issues in Contemporary Education, The Pedagogy of Faith (editor), and Adventures in Self-Directed Learning. He is passionate about futures in education, educational innovation, alternative education, and nurturing agency and curiosity.