- How many students have failed their last two math quizzes?
- Which students have missed three or more days of school in the last month.
- What is our 4 or 5 year graduation rate? How about our first year retention rate?
- Which students are most at risk for dropping out?
- What percentage of students are first generation college students?
- What factors most lead to student engagement and improved learning?
- How much class time is “on task” for each student? What is the average cost to recruit a student for a given program?
Ask any question about student learning, motivation, or engagement. Then find data to help answer that question. Now what? What do you do with data? How will it inform your decisions? This is what people refer to as data-driven decision-making, and it can be wonderfully valuable. However, it can’t drive decisions, not by itself. Decisions are not data-driven. They are driven by mission, vision, values and goals. We want purpose-driven organizations, not data-driven ones.
Without clarifying one’s goals and values, the data are of little value. Or, perhaps even worse, the data lead us to function with a set of values or goals that we do not want. I’ve seen many organizations embrace the data-driven movement by purchasing new software and tools to collect and analyze data, but they did not first figure out how data will help them achieve their goals and live out our values. I’ve seen organizations that value a flat and decentralized culture be drawn into a centralized and largely authoritarian structure…because the systems were easier to use that way or it was less expensive. I’ve seen organizations that value the individual and personal touch abandon those emphases when data analysis tools were purchased. I’ve also seen organizations spend large sums on analytic software, only for it to be largely unused. These may not be all bad, but it is wise to recognize how data will influence an organization.
An important part of any organizational plan to collect, analyze and use data sets is to establish some ground rules, working principles, and key performance indicators. These should reflect the organization’s values and mission. Yet, it is easy to set up some key performance indicators over others simply because they are easier to measure, that is how another organization did it, they are values and demanded by external stakeholders, or because a small but influential core wants it. As such, data analysis can lead us away from our mission, vision, values and goals as much as it can help is achieve or remain faithful to them. The data that we see and analyze has a way of establishing institutional priorities. The data not collected or analyzed ceases to have a voice amid such outspoken data sets.
In addition to this, data analysis is not neutral. The methods and technologies associated with it are values-laden. They typically amplify values like efficiency and effectiveness. Few people will disagree that both of those have a role in learning organizations, but not at the expense of other core values. As such, I contend that, alongside key performance indicators, it is wise to establishing core value indicators when implementing a data analytics plan. What indicators let us know that our values are visible, strong, and amplified?
In the end, behind any decision is a mission, vision, set of values, and list of goals. Not data. Start with the goals and values. Then ask how data can serve those values and goals…not lead them.