Who's got the data on data-driven decision-making?

It seems educators, reformers, and advocates everywhere are committed to the idea of “data-driven decision-making.” Presumably, this term and its popularity are outgrowths of increased visibility and accountability in public education along with the rapid growth of the role of data in other parts of our lives. 

And, let’s be honest: it sounds good. It makes us feel secure. It sounds really smart. And, if done well it probably could be transformative. 

In order for data-driven decision-making to have much meaning, however, we need to maintain a critical eye and keep asking questions (let’s at least try to keep it from being pure jargon anyway):

DATA … what’s data and what’s not data?

The brands that survey us and track our online and buying patterns never really ask if the data they see can be verified by a nationally recognized higher education institution. It doesn’t matter. It’s data, and they use it for what it is.

In education, we need to get the idea of data out of the clouds (only data wonks understand it), out of institutional paradigms (data-driven and evidence-based are not synonymous), and demystify it a bit (every interaction with another human is full of data points). Data isn’t just delivered to us from the researchers or the “data people” at central office. We don’t need a published report or a study to have and use data.

When we talk to a student and ask what she is interested in, what her concerns are, how she is feeling: that’s data. We just need to ask.

If you ask 30 students at your school if they have been bullied in the last month and 15 say yes, you have data that suggests a bullying problem – University X doesn’t need to confirm it. 

DATA … WITH PEOPLE …

Is the data merely accessible or is it actually consumable?

In its current state, data in education feels too complex, distant, and obtuse. I am a reasonably educated guy, and when I look at some of the data and spreadsheets that actually get shared from time to time with students and parents and even with teachers more frequently, my eyes go crossed. And, when I think about indicators like school climate that might actually be helpful in real-time, there’s no good data being captured. Because it makes sense in a database – or to someone building a database – doesn’t mean data makes sense in the hands of those who are supposed to interpret and use it. 

Because we can report on it annually as a school system or community and say “yep, we track that” or dig it out occasionally for a grant means little to its usefulness in our daily work. In fact, data that is six clicks deep in a database or learning management system should probably not even be considered available to most users. It’s not consumable anyway.

Data-driven decisions start with our ability to process data in terms we understand and in the context of decisions and actions we actually control.

DATA…WITH PEOPLE…WITH DECISIONS TO MAKE

Who are the deciders and what do they get to decide?

As you might guess from my previous writing and work, I wonder, in particular, where our students are in this conversation. In my experience, students generally don’t have much say on important issues in their schools (I am being gentle here). So, obviously, school data isn’t something we discuss with them. Instead, we treat students merely as data providers not data users. Meanwhile, they collect and analyze data everyday and in every interaction and use it to make important personal and relational decisions.

But, there is a challenge here for teachers and other staff too. Most teachers I talk with view data as more of a tool of external accountability than professional process and continuous improvement. And, often the data they are accountable for reflect variables they have little-to-no control over, particularly in the short term. So basically, 1.) they have access to data that doesn’t relate to their actual realm of decision-making; and/or, 2.) they are trying to make decisions (and someone wants the data that supports it) and they don’t have it.

But, if data-driven decision-making is critical to the improvement of schools and development of communities, shouldn’t it be critical (available, consumable, and relevant to decision-making) across all stakeholders? If the range of consumable data, data usage, and the related (or unrelated) decision-making processes are narrow, unclear, or inconsistent, then we can be pretty sure our data-driven outcomes will be as well.

Like anything else, the data on data-driven decision-making will likely reflect the quality of implementation not the idea itself.

---