By Samantha Grant
My oldest child is in a classroom that gives students points for good deeds done throughout the day. My guess is that the teacher is trying to encourage social and emotional learning (SEL) skills. Every day I check in to see how my child did, and every day I think about how this would have gone for me. I was the kid in class who could never stop talking, so my daily points for "works quietly" and "on task" would have been abysmal. I wanted to talk- and was willing to talk about learning or my new shoes or what we were going to do at recess- the topic didn't matter. It did matter that learning for me was a social activity.
Now I’m an evaluator, which means I not only get to talk to lots of people but I also get to think about how we measure big concepts like SEL skills. As a measurement person, you might assume that I want to measure everything. Not true! In fact, I’m not sure that measuring SEL skills is the right way to go.
There is a debate right now about measuring SEL in school. (Check out Angela Duckworth’s work because she is leading the field on this topic.) Youth workers are talking about it too, and I can see why programs would want to measure SEL skills. These skills are often well aligned with the mission and goals of positive youth development, and most of us would agree that a good youth worker thinks intentionally about SEL skills.
Where I think programs go wrong is that we want to turn SEL measurement into accountability measurement. Should we be judging the value of our programs based on how well youth are building SEL skills? Personally, I think this measurement takes us away from the heart of youth work and turns us into hamsters running on the accountability wheel.
The danger of this accountability measurement is that we start to teach SEL skills in a way that encourages youth to learn them simply to pass the next test. If your program is set on measuring SEL skills for accountability, how do you decide which one to focus on? Here’s an example of how I think focusing on SEL skills in isolation can be problematic in youth development.
Take the famous marshmallow experiment:
This test assesses the ability of a child to delay gratification. Research suggests that delaying gratification is good for us and is connected to educational attainment and positive life outcomes. If your program was measuring the ability to delay gratification, you would want youth to not eat the marshmallow.
Or maybe your program cares more about Joy. Some would argue that treasuring every day and experiencing joy is an important SEL skill. If that was the case, youth should be popping those marshmallows into their mouths and asking for more!
You might also care about friendship skills and compassion. Youth might choose to share their marshmallow with a friend. Do they pass your test?
So which one are you measuring? This example is just one argument for why it’s tricky to measure SEL skills. Read more about consideration for practitioners if you are planning to measure SEL skills for accountability.
Guess what? My super smart colleagues at the Center for Youth Development recently created a tool that’s worth exploring: SEL in Practice: A Toolkit of Practical Strategies and Resources. The toolkit has a section on evaluation that includes some amazing activities that can help you think deeper about the evaluation of your SEL program elements. Many of the activities focus on how programs can evaluate SEL skills as a process measure rather than an accountability measure.
Check out the toolkit, and let me know: do you measure SEL skills in your program? Have you learned something valuable in your measurement that everyone should know about?
And just because I’m curious, would you have eaten the marshmallow?
You are welcome to comment on this blog post. We encourage civil discourse, including spirited disagreement. We will delete comments that contain profanity, pornography or hate speech--any remarks that attack or demean people because of their sex, race, ethnic group, etc.--as well as spam.
My oldest child is in a classroom that gives students points for good deeds done throughout the day. My guess is that the teacher is trying to encourage social and emotional learning (SEL) skills. Every day I check in to see how my child did, and every day I think about how this would have gone for me. I was the kid in class who could never stop talking, so my daily points for "works quietly" and "on task" would have been abysmal. I wanted to talk- and was willing to talk about learning or my new shoes or what we were going to do at recess- the topic didn't matter. It did matter that learning for me was a social activity.
Now I’m an evaluator, which means I not only get to talk to lots of people but I also get to think about how we measure big concepts like SEL skills. As a measurement person, you might assume that I want to measure everything. Not true! In fact, I’m not sure that measuring SEL skills is the right way to go.
There is a debate right now about measuring SEL in school. (Check out Angela Duckworth’s work because she is leading the field on this topic.) Youth workers are talking about it too, and I can see why programs would want to measure SEL skills. These skills are often well aligned with the mission and goals of positive youth development, and most of us would agree that a good youth worker thinks intentionally about SEL skills.
Where I think programs go wrong is that we want to turn SEL measurement into accountability measurement. Should we be judging the value of our programs based on how well youth are building SEL skills? Personally, I think this measurement takes us away from the heart of youth work and turns us into hamsters running on the accountability wheel.
The danger of this accountability measurement is that we start to teach SEL skills in a way that encourages youth to learn them simply to pass the next test. If your program is set on measuring SEL skills for accountability, how do you decide which one to focus on? Here’s an example of how I think focusing on SEL skills in isolation can be problematic in youth development.
Take the famous marshmallow experiment:
This test assesses the ability of a child to delay gratification. Research suggests that delaying gratification is good for us and is connected to educational attainment and positive life outcomes. If your program was measuring the ability to delay gratification, you would want youth to not eat the marshmallow.
Or maybe your program cares more about Joy. Some would argue that treasuring every day and experiencing joy is an important SEL skill. If that was the case, youth should be popping those marshmallows into their mouths and asking for more!
You might also care about friendship skills and compassion. Youth might choose to share their marshmallow with a friend. Do they pass your test?
So which one are you measuring? This example is just one argument for why it’s tricky to measure SEL skills. Read more about consideration for practitioners if you are planning to measure SEL skills for accountability.
Guess what? My super smart colleagues at the Center for Youth Development recently created a tool that’s worth exploring: SEL in Practice: A Toolkit of Practical Strategies and Resources. The toolkit has a section on evaluation that includes some amazing activities that can help you think deeper about the evaluation of your SEL program elements. Many of the activities focus on how programs can evaluate SEL skills as a process measure rather than an accountability measure.
Check out the toolkit, and let me know: do you measure SEL skills in your program? Have you learned something valuable in your measurement that everyone should know about?
And just because I’m curious, would you have eaten the marshmallow?
-- Samantha Grant, evaluation director
You are welcome to comment on this blog post. We encourage civil discourse, including spirited disagreement. We will delete comments that contain profanity, pornography or hate speech--any remarks that attack or demean people because of their sex, race, ethnic group, etc.--as well as spam.
Thank you for writing (and sharing!) this article Sam. I think the evaluation's perspective is critical to think about as we continue as an organization continue to invest in SEL. It just proves further that we have to be strategic in evaluations and be aware of the consequences of either poor or unnecessary evaluation measures. Thank you for your evaporator's perspective!
ReplyDeleteThanks for reading! The more time I spend in evaluation, the more that I see that intentionality is just as important in evaluation as it is in youth work practice. We get the best results when we are thoughtful about our practice.
DeleteVery interesting points to consider.. thank you for your article. I'm hungry and definitely would've eaten the marshmallow!
ReplyDeleteThanks for reading! Here you're proving that context matters in SEL measurement too- if I was hungry, I would have eaten the marshmallow too!
DeleteGreat questions about measurement & SEL - thoughtful piece worth reflection as we try to improve SEL & social emotional competencies that support success. I find I partially agree and partially disagree with your views.
ReplyDeleteI agree that using measures of SEL for high stakes accountability is simply wrongheaded and neither justifiable in terms of the measures we have available nor the outcomes we seek. Making youth pass a SEL "test" in some prescribed way and then teaching to the "test" is not helpful -- especially not in SEL. So I would agree SEL measurement should not be used in that way,
But I do not agree that means we do not measure it or that we should fear it will be used in those ways. The problem lies in the way data is used not in the data. Using your marshmallow test example, if the only thing you are trying to teach is delayed gratification and a child can not pass until they can last 15 minutes, that is really not learning nor is it teaching – it is obeying which is a very different thing. But doing the test itself may create useful discussions of exactly the types you noted and could help youth see the differences in what happens when you do delay and when you don't. These uses can help youth learn about delaying gratification. It is data used to improve the learning not judge the student or the teacher.
I also disagree that a lot of people working on SEL in out of school time programs are trying to either push or respond to accountability – especially high stakes accountability. Angela Duckworth’s arguments are more about using them for accountability in schools – and there I broadly agree. Accountability tends to be to someone for something. I do not see most programs doing SEL in ways that they have to be accountable for success to others – even their boards – for measurable increases in a narrow set of measured SEL skills. We are very far from that currently being the case.
A positive use of SEL measurement is in evaluation to see whether what we are doing is “working” – that is are we getting or improving outcomes for youth. This type of evaluation of outcomes to inform what we might do to improve is healthy & measuring students SEL skills might well play a constructive role in that process. Again, it is how the data is used, rather than whether we should measure, that is the important question. If the data is not going to be used or is used inappropriately then yes, I too would resist measurement. But those do not reflect is happening in SEL. What I see are sincere efforts to learn about whether our programs are affecting SEL skills and whether the ways we are doing our program can learn to do it better. The data is used for learning and improvement, not judgment, blame, and taking away resources. I see data being used to give students better feedback about their skills & causing different discussions. I see staff looking at data from the Holistic Student Assessment and wonder how they might better reach the group of youth they have.
In the end, I think youth programs need data to help them see what they are doing in new light, get additional grist for the program, and generate ideas about how to improve.
Thank goodness our field is not living in an environment that demands the kind of narrow, wrongheaded, and high stakes accountability you rightly fear. Unfortunately, sometimes the fear of that makes us far too hesitant to measure things like SEL and use data well to improve what we do. It is that unnecessary, fear driven discouragement of measurement that scares me.
Final, I wish to be clear that measurement is not the first or even primary thing we need to do to get intentional about SEL – but it can be useful and help guide other more critical efforts around adult professional development and intentional practice.
Hi Dale. Thanks for your thoughtful comments and bringing this post up at the beginning of your speech today. I love hearing different perspectives in this debate. I'm also relieved to hear that you don't see youth programs moving toward a system of using SEL for accountability. I agree with your idea that "it is how the data is used." We don't need one more system of data collection if it isn't going to lead to better programs for youth or improved development for young people.
DeleteI think some of the best youth workers that I know are already thinking hard about these ideas and are finding soft measurements of the skills they hope to foster in young people. I really appreciate the tools that were put together in the SEL toolkit because they keep evaluation grounded in use. As you noted in your speech, "Growth doesn't just happen." We need to be mindful and intentional about it.
Thanks.