ECVO’s evaluation learning journey started right around the time we first offered the Eval Lab course in 2018. A partnership between the Evaluation Capacity Network (ECN) at the University of Alberta, the Government of Alberta (GoA), and ECVO, Eval Lab was designed to help organizations embed evaluative data into strategic decision-making. While our mission is to build capacity throughout the sector, we acknowledge that we have a lot of learning to do ourselves. This is especially true in the realm of evaluation. Like many organizations, our evaluation efforts were fueled by funding requirements with no real mechanism to embed our learnings into our work. Working with ECN and the Government of Alberta on the first Eval Lab course truly kickstarted our journey.
At the same time that Eval Lab 1.0 and then Eval Lab 2.0 (the first and second iteration of the same course) were being offered, we had an internal moment of realization – we needed to solidify our own evaluative processes if we were going to continue to offer this course! With little time or knowledge to undertake such a project, we contracted In Situ Change Strategies to help us develop an evaluation framework.
What they did instead was change our world (not to be dramatic – but seriously).
The experts at In Situ suggested that a more appropriate and valuable undertaking would be to develop a strategic learning framework that not only incorporates all the components of an evaluation framework, but also goes a step further by mobilizing the data for strategic decision-making. The goal of this framework was to streamline all of our evaluation processes to be manageable, both financially and given limited capacity (i.e. not having a dedicated evaluation specialist on staff), while building a mechanism for learnings, no matter how small, to improve strategic decision-making.
Of course, the first step in this process was to provide an overview of strategic learning; what is it, why does it matter, how is it different from evaluation, and who is involved in strategic learning? Once those questions had been answered, we could dive into creating our strategic learning framework. Our framework includes four main features:
It took several months of working with the consultants to flesh out the four components of our framework and reach a point where we felt ready to start implementing the framework into our everyday work, but in November 2020, we felt ready.
Now, I will be the first to admit – this has not been an easy or smooth journey so far. Truthfully, we probably weren’t as ready as I thought we were. Embedding new processes and creating a culture of learning while navigating a pandemic, supporting a struggling sector, and welcoming new staff has been a challenge.
Nevertheless, I’m happy to share some of my personal reflections on the process in the hopes that it might help other organizations find their way towards more effective strategic learning.
1. Timing Matters.
It’s probably best to avoid implementing a new Strategic Learning Framework in the middle of staff turnover or a holiday or an off-season. This may sound ridiculous, but hear me out – our framework was drafted in November 2020, at which time we sent it to staff to review individually before convening for a final group review and discussion. The intention here was to implementing the framework in our work in January 2021. Of course, the holiday break ruined any momentum that we had built.
We reconvened in January and planned to re-introduce the framework in February 2021, but soon thereafter we had multiple new team members join the organization, which means we had to introduce the framework — yet again. Before we knew it, summer was upon us and we were up against vacations and extended absences. I soon realized that nobody had started implementing the framework in their work, and we hadn’t had a single strategic discussion about our learnings yet! So, in short, timing really does matter.
2. It’s a Journey, Not a Destination.
Evaluation, like any learning, is a journey – and often not a straightforward one. We learned to understand and embrace that any frameworks or plans related to evaluation and learning were going to be living documents, and that’s okay! Expect changes. It’s all part of the messy process of becoming more reflective, proactive, and adaptive.
3. Teamwork Makes the Dream Work.
It’s crucial to dedicate some time to getting your team on board. Hold staff capacity building sessions. Provide ample space for questions. Don’t assume that everyone has formal knowledge or training in evaluation. Walk people through examples of how they might implement the framework, provide space (and suggestions) for reflective practice. Most importantly, make your framework as clear and understandable as possible. Keep in mind that non-profit professionals are incredibly busy, so don’t make it hard for them to get on board.
4. Use the Buddy System.
Find a buddy and stick with them! It can be very difficult and frustrating attempting to implement a framework like this into your organization, especially when a culture of learning wasn’t there before. In hindsight, I wish ECVO had gone through the Eval Lab course as a student so that we could have walked through the process with other organizations from the very beginning. Luckily, I was able to connect with an evaluation specialist at another local non-profit going through a similar process of implementing a strategic learning framework in their organization. The opportunities to share frustrations and insights, brainstorm solutions, and stay accountable to one another proved invaluable.
5. Patience is Indeed a Virtue.
Always remember – these things take time. We are now approaching a full year since our framework was first introduced to staff. We had our first strategic learning discussion last month. It was a great discussion, which led to several decision points being added to the agenda for our upcoming strategic planning initiative. We also learned what worked and what didn’t work while facilitating this conversation and we have already noted these learnings so we can adapt things for our next discussion.
While going through this process and preparing for the third iteration of Eval Lab, we recognized that many organizations don’t have the resources to hire consultants to collaboratively build a strategic learning framework. Which leads us to today — after a year’s worth of learning, we have adapted the third iteration of Eval Lab to include theory and practice on strategic learning, as an accessible way to start thinking about strategic learning.
We have kept all the best parts of the past Eval Lab offerings: a flexible co-learning environment, opportunities to test your learning with the class and with your organization, instructor office hours, and great evaluative content. We’ve also added content on strategy and learning, a COVID-friendly blended learning model, and additional resources and frameworks to support continued learning long past the course end date.
We are thrilled to be offering a new-and-improved Eval Lab: Strategic Learning for Improved Decision-Making course starting in January 2022. We encourage you to check the listing for more details, or reach out to emma@wordpress-778227-2652451.cloudwaysapps.com if we’ve piqued your interest!
