The Challenge

Instead of living in the day-to-day,  YouthPower365 at the Vail Valley Foundation wanted to be ahead of the curve in educational programming, and focus on where it wanted to be in years to come.

The Result

YouthPower365 at the Vail Valley Foundation was able to create a clearer concept of their PwrHrs programming, which helped their partners implement the program with more fidelity. Additionally, the team was able to revamp how they collected and used data, so that they were able to make real-time adjustments to their program that would help improve their student outcomes.

Strengthening Programs with Data

“As advanced as we thought we were, we realized we were nowhere near where we could potentially be,” said Peter Barclay, Program Director for the PwrHrs program at the Vail Valley Foundation.

PwrHrs was already a strong program – at the time, the organization had 14 staff, engages hundreds of volunteers and teachers, and serves roughly 5,700 students each year. They came to ResultsLab looking for support in program design and evaluation, two areas where their team was looking to build their skills.

Peter and the PwrHrs team provide supportive educational and enrichment programming to children and youth from kindergarten through high school in and around Vail, Colorado.  ResultsLab worked closely with this team, as part of a capacity-building grant provided through the Tony Grampsas Youth Services Program.

Creating Clearer Concepts and Alignment

The initial phase of our work together was focused on coaching Peter and his team on developing a Theory of Change, identifying their long-term outcomes, and aligning their current activities to those changes. “That was really eye-opening to decide where we should be focusing our energies and where we get the biggest bang for our buck,” Peter mentioned.

Next came coaching and support on the specific elements of PwrHrs, teasing out what, where, when, and how staff and other program facilitators work with students. This helped PwrHrs more clearly articulate what it needs from its partner sites and program facilitators. “Program fidelity was another big ah-ha for us,” Peter says. “Whereas we used to give our schools a lot of autonomy to decide what’s best for their programs, we realized there are some non-negotiables that we need to hold true to.” Now, staff can clearly tell potential partners what the program is and what to expect if they want to be part of PwrHrs.

Having a clearer concept of the program also allowed Peter and his team to revamp their training approach. Because teachers facilitate much if the PwrHrs programming, the organization historically assumed that these teachers had all the relevant skills they need. But through ResultsLab’s coaching process, they realized, “because we’re an out-of-school program, we have different expectations, different guidelines, different ways which we deliver, different program beliefs, and a different culture … We need them to be certified as a PwrHrs instructor.”

That year PwrHrs trained 67 summer staff on how to run summer camps that exemplify the PwrHrs culture and experience. This training allowed their summer instructors to marry what they know about child and youth development with what PwrHrs has learned, making for an effective program to serve children and youth in the best way possible.

Building Processes for Real-Time Data and Decision-Making

Finally, ResultLab helped PwrHrs revamp how they evaluate their work so they can adjust their approach with students in real-time. For example, much of PwrHrs programming during the school year involved providing targeted reading and math tutoring in a group setting, where students are placed in groups with others at a similar proficiency level.  Students used to stay in the same group for 7 – 12 weeks because the tests used to determine proficiency weren’t practical for week-to-week assessment.   “Now we’re trying to work more closely with the school district to find out their checkpoints and data points,” Peter says, “so we can regroup our students more often and get them exactly what they need. If a student were 60% proficient and now they’re 80%, we don’t want to keep them in the 60% percent group. We want to get them in the 80% group and push them to 100%.”

Capacity Building and Long-term Solutions

ResultsLab’s work with Peter and PwrHrs illustrates something we think is important: helping nonprofit staff build the skills to evaluate and redesign their programs themselves. We realize this results in more effort, but also in more long-term payoffs. Peter noted, “The process was scheduled, and it kept us on task. It took a lot of man-hours … But it’s going to be work that we’re going to use not only in the PwrHrs program but in the whole organization.”