7 ways to assess capacity-building
Drawing a straight line from capacity-building to policy change is likely impossible. But we can still bring structure and rigor to a holistic assessment process.
A short overview of seven ways to assess capacity-building in English and French
How we approach capacity-building
We approach capacity-building as a form of technical assistance. We aim to tightly align the workshops and scholarships we provide with specific regulatory or policy goals identified by government partners. This helps build institutional momentum and the conditions for regulatory change.
Read more about what we’ve learned about capacity-building →
How we assess capacity-building
Drawing a straight line from capacity-building to policy change is likely impossible. But that doesn’t mean that we cannot assess capacity-building activities across a range of criteria.
Below, we share seven dimensions that reflect the complexity of capacity-building interventions. Within each dimension, you’ll find:
A key question
Sample assessment indicators
Sample evidence
Taken together, we believe these seven dimensions can provide a more holistic analysis of our capacity-building activities and also enable us to assess the work of our partners. We hope they are useful to you as well.
+ 1. Relevance
Key question
Is the capacity building intervention relevant in terms of the project’s overall objectives?
Assessment indicators
- Capacity-building is in line with the diagnostics and the National Financial Inclusion Strategy
- There is evidence to show that capacity-building was conducted in response to a national request
- There is evidence to show the capacity-building content is up-to-date and updated regularly
Evidence
- Letter of request for capacity-building signed by national authorities
- Project plan
- Diagnostic document
- National Financial Inclusion Strategy
- Course syllabus
+ 2. Coherence
Key question
Is there a coherent approach with respect to the capacity-building interventions of our development partners?
Assessment indicators
- There is evidence in the project plan of awareness of other development partners's work in the field of capacity-building
- There is evidence in the project plan of complementarity of this capacity-building exercise with our teams's previous and current work
Evidence
- Project plans
- Emails of enquiry with development partners
+ 3. Effectiveness
Key question
How does capacity-building achieve the objectives included in the project’s results framework?
Assessment indicators
- Intended outputs have been produced through the capacity-building initiative
- Capacity and commitment were endorsed by local stakeholders (ownership)
- There is evidence of positive impacts of the capacity-building initiatives on achieving the project objectives
- Capacity-building initiative was well-perceived and well-attended by participants
Evidence
- Final project report (completion)
- Mission reports
- Feedback forms
- Target audience participation (e.g., women)
- Participation ratings (if scores are available)
- Number of scholarship programs completed
+ 4. Efficiency
Key question
What is our evaluation of the sequencing, timeliness, and cost efficiency of the capacity-building intervention?
Assessment indicators
- There is evidence that the deadlines were met
- There is evidence that staff and consultants were correctly assigned the tasks for the capacity-building activity
- There is evidence that the capacity-building activity was well sequenced in the project (e.g., course participants were selected carefully before being registered)
- There is evidence that the costs of the capacity-building activity were on target and proportionate to the needs ofthe institution
Evidence
- Final project report timeline (completion) compared to the initial project document
- Memo on the selection process of course participants
- Retro planning
- Analysis of budgeted expenditures in relation to the attendance of course participants
+ 5. Sustainability
Key question
How does the capacity-building intervention strengthen institutions and knowledge in a sustainable fashion?
Assessment indicators
- There is evidence that the capacty-building outputs are sustainable (e.g., capacity-building outputs like training materials were used beyond the end of the training
- The capacity-building activity has produced local requests for further capacity-building exercises
- There is evidence of follow-up work with other development partners
- There is evidence that sustainability issues were considered at design stage and were addressed (project doc and final completion report)
Evidence
- Workshop participants's feedback forms
- Number of views online (if a blog was written)
- Type of follow-up actions undertaken by local authorities (e.g., internal dissemination exercises following a training)
- Letters of thanks from local authorities
- Other development partners's plans
+ 6. Behavior
Key question
Has the capacity-building intervention influenced individual participants's behaviour in a direct or indirect way?
Assessment indicators
- There is evidence of what and how the participants learned through the capacity-building intervention
- There is evidence of changes in the attitudes and beliefs of participants, staff members, or community members regarding the problem or issue being addressed
Evidence
- Observation of training process
- Participant interviews and surveys about self-reported learning (including pre- and post-tests and/or comparison group studies)
+ 7. Impact
Key question
What are the direct and indirect impacts of the capacity-building exercise?
Assessment indicators
- Contribution of the capacity-building to financial inclusion or digital financial inclusion
- There is evidence of the contribution of the capacity-building initiative to regulatory change
- Note: Regulatory changes can range from an amendment in a regulation to a whole new law; however, this is a very complicated process that involves many stakeholders. While capacity-building can be useful, it cannot be deemed to be the only factor impacting regulatory or institutional changes.
Evidence
- Intermediary impacts could be in the form of institutional or regulatory changes decided following the capacity-building
- Indirect impacts in the form of established multiparty dialogue channels
- Long-term impact is difficult or almost impossible to assess
Examples of our assessment approach in action
Increasing female participation in scholarship opportunities by using sex-disaggregated data and taking a new recruitment approach
We collect sex-disaggregated data on our scholarship recipients. But we were unsatisfied with the percentage of female participants we had recruited between 2019-2021 (34%).
In response, we launched women-only application calls for Cambridge University and Women's World Banking executive education programs in 2022 in addition to working closely with partner institutions to make sure they were nominating female participants.
As a result in our efforts in 2022, the percentage of female participants increased to 45% from 2019-2022. In 2022 alone, the percentage of female participants was 60%.
Improving alignment between scholarships and regulatory priorities so that training outputs accelerate the policy change process
Like many development partners, much of our capacity-building support moved online with the COVID-19 pandemic. During the early stages of the pandemic, we would typically provide scholarships to individual participants or within individual departments. The scholarships sought to address knowledge or skills gaps but did not necessarily have a specific policy opportunity in focus.
Once we adjusted to the new paradigm, we saw room for improvement.
In 2021, we piloted a new high-intensity, cross-institution model in Egypt in collaboration with The Central Bank of Egypt (CBE) and The Tufts University Fletcher School Leadership Program for Financial Inclusion (FLPFI). For two months, 29 participants from 14 institutions worked together to identify and develop policy solutions in promotion of Egypt's National Financial Inclusion Strategy.
We learned that this approach, while more complex and requiring more internal resources from our team, was viable.
We have since expanded and applied this model in Ethiopia to include civil society and the private sector (in addition to the public sector) as part of a larger strategy of convening stakeholders across sectors to influence the policymaking process.
Validating new approaches by building participant surveys into learning agendas
In 2022, we signed an agreement with a government partner to provide capacity-building to 30 staff members and five senior managers.
We knew from previous engagements that senior leaders have limited time to devote to coursework and that their needs are different from those of staff. We also knew that staff are more likely to complete coursework if they have strong institutional support.
In response, we developed a two-track learning agenda.
With senior leaders, we provided tailored technical sessions with a shorter time commitment
For staff that would eventually be responsible for policy implementation, we provided scholarships for courses with our training partners
Our hope was that increasing the relevance of the trainings would lead to greater participant engagement, which would then help create the conditions for a smoother policymaking process.
Our preliminary analysis of baseline and midpoint survey results suggests:
Increased confidence on technical subjects related to DFS
Positive feedback on the learning experience provide by UNCDF and training partners
We will share the high-level survey results with senior leaders as part of the collective assessment of our work together.
Based on results so far, we anticipate applying (and refining) this two-track model in future engagements.
More on capacity-building
Authors
Carlo Guicherit
Marie Sudreau
Alexis Ditkowsky