Skip to main content

What does it mean to be driven by data?

Dale-Blyth.jpg From evidence-based practice to data-driven decision making, the role of data in driving everything forward is becoming omnipresent. As a recovering quantitative sociologist this excites me. As a person devoted to building the field and making a difference in the lives of youth it raises both opportunities and concerns.

Like driving a car, youth work is a navigational sport filled with hundreds of decisions on a moment-by-moment basis. Whether it is the development of the field of youth work or the development of a young person, we process thousands of bits of data to make decisions.

But the data driving these decisions, like many others, are a real mix -- some conscious and quantitative and some unconscious; some rational and some emotional. We drive differently when we are angry than when we are happy.

If youth work is to become a data-driven field, we had better make sure we know what that means and take a strong role in shaping the data available and how they are used.

In driving the field of youth work there are decisions at many levels. Decisions at the policy level about what we fund and support, how and for whom. Decisions on the system level about what quality looks like and who is qualified to practice. Decisions at the program level about what we offer and how it's designed. Decisions at the offering or activity level as a youth worker plans and executes part of a program. And then there are the decisions by each youth, which shapes the experience for themselves and for others.

As several new books point out, from David Brooks' The Social Animal to Incognito: The secrets of the Brain by David Eagleman -- we are learning that more and more of the data driving our decisions are youth-road.jpgcollected and processed unconsciously -- not in some simple rational, conscious and largely cognitive ways.

Over the last few weeks I have had the pleasure of thinking about how we collect and use data on young people's learning, especially but not solely about non-formal learning in out-of-school-time opportunities. Data that can help us to drive decisions on what we do and how we do it with respect to the learning and development of young people.

Learning is about both the journey (the levels of quality in a program, a young person's engagement, and opportunities for youth to contribute) as well as how the journey helps youth get to critical destinations or outcomes.

What mix of data do you think should drive our field and the practice of youth work?

Dale Blyth, associate dean and director

Print Friendly and PDF


  1. Dale,
    I like that you bring up the fact that humans are not rational in their processing of information. I think this should be considered when we collect and present data to our audiences and decision makers. What kind of implications do you think that has for our field?
    In past weeks discussions, Jen Skuza has brought up the idea of youth driving learning and the data collected in our system. I would agree with her that involving youth in these decisions is critical to collect data that has direct relevance to our target audience.
    I worry that a tension will grow between collecting accountability or impact data and program improvement data and that improvement data will not be seen as important for programs. I believe that we need a healthy mix of both.
    Thanks for starting discussion around this important topic.

  2. Thanks for your comments Sam. Great points. It is at the core of what I was trying to get at in the blog. We see data in too narrow a set of terms and in too silo-ed ways.
    I do believe our field could become divided by those who use data to improve practice and those who use data to judge it in terms of impact - from both inside and outside the field That would be unfortunate and is, I believe both unnecessary and unhealthy for the field.
    The future of the field lies in finding ways to better collect, analyze and use data in both ways and for both ends -- but to do so in ways that are aligned and connected in both efficient and effective ways. We will not get better impacts if we do not improve and sustain strong practice. We may not even have practice if we do not show it makes a difference -- and not only for academic outcomes.
    But beyond even these two ways of using data we need data we also need data on whether we are reaching youth with the types of learning opportunities beyond the classroom that are critical to their learning, growth and ability to contribute. Data on access and how communities, not just youth and programs, are assuring youth are engaged is needed.
    In all three cases, data can be a critical tool assuring we are making progress along the journey as well as reaching important destinations.
    The implications of all of this as well as the non-cognitive or unconscious elements in decision making for how we present data to our audiences is an intriguing question.
    In some ways it reminds me of brain growth in adoelscencce. It is not just, or even primarily, that brains are developing a new ability to get to the right answer. Rather it is that the efficiency, effectiveness, and very nature of the processes are changing. If we are going to optimize the use of data in decision making in our field we have to help people understand data on both the outcomes and the process are important.
    One implication is making sure we do not polarize or accept a polarization of what making a difference means in terms of just narrowly defined outcomes. We need to present data on whether and how youth are participating, to what extent they are engaged, why participation and engagement are important, and how they are linked to valued societal outcomes such as learning academically as well as socio-emotionally. We have to stop saying we know youth enjoyed it but not whether it made a difference. Setting enjoyment as an important, but not sufficient, part of whether the program will have an opportunity to make a difference is important.
    Similarly, we have to not go so far to the other side and say we only and always have to have youth drive the data (not that that is what either you or Jennifer is saying). Data that is important for decisions needed to be seen from the perspective of the decision maker but informed by an understanding of the dynamics involved in the context. A decision maker may not want to know about non-formal learning's role in the achievement gap but if it is part of the context needed to understand and change it, we have an obligation to help inform the decision maker about the context using the best data we can.
    What do others think about Sam's question?

  3. This was a great post and discussion thread. I echo Sam's question and I agree with Dale's response. After having read The Social Animal (I recommend this book to anyone out there) it became even more apparent to me that data must be "informed by an understanding of the dynamics involved in the context" (Dale).
    I remember going to a workshop a few years back in which the two researchers presented to teachers about data interpretation and how even those of us interpreting the data must be aware of our own preconceived ideas, biases, unconscious processes that guide our decision making and meaning making when it comes to data. So this is a different question than Sam's and maybe a little off topic but since Dale you brought up The Social Animal and Incognito I can't help by wonder how do we keep our unconscious processes at the forefront of data-driven decision making? Some food for thought.
    Thanks again for a great topic.

  4. Now this is an interesting conversation! I have a couple reactions to your very thoughtful commentary.
    Personally I resist the term "data driven." It elevates "data" to the force, the major factor, the driver. I prefer to view decision-making as "informed by data" -- as well as by political and financial realities, mission considerations, funder priorities, common sense, experiences with best practice, stakeholder interests, etc. Those working in our field know full well that there are many considerations that drive decision-making, but today "data" is perceived to have a force and power beyond all else. I say put the thoughtful and relevant data on the table with all the other considerations -- as a source of information -- and then do the tough work of deciding and prioritizing. Dale acknowledged this when he used words like "optimizing the use of data" and "data as a tool" (not the only tool or driver).
    Without definition, people too often see "data" as numbers and mysteriously complex analysis. Data is just information that's been gathered in a deliberate way and analyzed or processed for use. it comes in lots of forms -- numbers, words, ideas, opinions, stories. The process for gathering must have integrity and the analysis rigorous. What's important is to have a variety of relevant information on the table when making decisions.
    Josey's notion of keeping the/our unconscious processes at the forefront when gathering, analyzing or using data is the most important idea for me. Researchers or evaluators frame the issues by the words they choose, the questions they ask, and the people from whom they seek answers. We see it in political polls all the time. I often say to myself, " Well, if you ask it that way, I'd say yes, but it you ask it this way, I disagree!"
    Final thought today: Any data put on the decision-making table needs to pass the test of "Why is this data on the table?" and "What important question does this data answer?" Data should stimulate us to ask more questions like "How is it relevant to our organization?" and "What does it fail to consider that's important to us?"
    Framing data as a litmus test -- as the driver -- doesn't help much. Framing data as indispensable, solid, useful information makes sense. Data's an important thing, but not the only thing! Thought - provoking! Thanks.

  5. There are so many compelling points and questions in this commentary. Great blog!
    Here are some of my thoughts on data:
    So often, the word “data” is narrowly understood as being “quantitative” – so only including numbers and involving statistical analysis. Like Joyce Walker mentioned in her post, data can take on many forms -- numbers, words, ideas, stories, themes and so forth. Along with that narrow view, in many cases, greater value is placed on “quantitative data” – such as when people say “do you have any hard data?” - they are typically referring to results with statistical significance. If someone comes forth with qualitative data, sadly it is often accompanied with an apology or the question “do you have any numbers to back that up?”
    Qualitative and quantitative data each play an important role in “knowing”. Scientific rigor can be applied to the collection and analysis of both – as long as they are understood within their own epistemological context. So, I think it is important to make a shift toward valuing both qualitative and quantitative data. It opens up our possibilities of knowing more about life and informing decisions on what we do and how we do it in regard to learning and development of young people.

  6. Great conversation, but you assume the people making decisions even value data. If they don't trust science or research, they will ignore it and mistrust it, regardless of the integrity or rigorousness of the research. They will use their ideology to make decisions. The greatest data is of no use when it is ignored.

  7. How does youth development affect the economy

  8. Power Point PresentationsNovember 27, 2011 at 9:18 PM

    For me the issue is no longer about whether data will inform or drive decisions in our field. That is a given. The real issue instead is the extent to which our field will learn about and from different forms of data and use that data effectively to build a stronger more impactful field moving forward. If data is going to become a driver then we as practitioners, program leaders, system builders, and funders better get some more driver's education when it comnes to data.