Technical Note Contingency Evaluation Fund (CEF)
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Decentralized evaluation for evidence-based decision making
WFP Office of Evaluation
Decentralized Evaluation Quality Assurance System (DEQAS)
Technical Note
Contingency Evaluation Fund (CEF)
Background
1. Under the Evaluation Policy (2016-2021), WFP commits to assigning 0.8 percent of its total
contributions income annually to addressing the needs of its entire evaluation function by 2021
at the latest1.
2. The Corporate Evaluation Strategy (2016-2021) set out a triple funding model for the
evaluation function on the road to sustainable financing arrangements. This triple funding
model, endorsed by the Assistant Executive Director - Resource Management Department and
the full Executive Management Group in April 2016, includes:
i. PSA for Centralized Evaluations (CEs), the augmented role of the Office of Evaluation
(OEV) setting standards, oversight and guidance for the entire function; and the six
Regional Evaluation Officers;
ii. Project sources (Direct Support Cost (DCS), Trust Funds and, in future under the
Financial Framework Review (FFR), ‘implementation costs’) for the conduct and
management of project-related Decentralized Evaluations (DE)2;
iii. Multilateral funding for OEV-managed services to support the DE function.
3. The Corporate Evaluation Strategy also announces the creation of a Contingency
Evaluation Fund (CEF) with two components and WFP’s Management Plan 2017-2019
includes a 2017 allocation of US$ 2 million for the CEF. The CEF components are:
i. Support to planned and budgeted DEs where Country Offices (CO) face genuine
resource constraints at the time the evaluation is needed (US$ 1.5 million
replenishable from PSA); and
ii. Part of OEV-managed services to support decentralized evaluations, such as
capacity development for DE and an outsourced quality support mechanism (US$
500,000 annual; multilateral funds).
4. This Technical Note describes the process for operation of Item 3 (i).
Contingency Evaluation Fund Purpose and Nature
5. Purpose: to enable the conduct of decentralized evaluations in under-funded situations and to
enable Regional Directors (RDs) to effectively support COs in their evaluation planning and
budgeting.
1
See para 43 of the Evaluation Policy. In 2015, the actual expenditure was 0.19% of total contributions income, but this included only
the centralized evaluation function. Figures for decentralized evaluation were not available.
2 Project sources (and ‘Implementation Costs’) will also contribute to the costs of project-related centralized evaluations, such as of
corporate emergency responses. Under the Country Strategic Plan (CSP) Policy (approved by the Executive Board, Nov.2016) and the
Financial Framework Review, ‘portfolio budgets’ will also include provision for OEV-managed Country Portfolio Evaluations (CPE), to
be conducted towards the end of the life of the CSP – usually Year 4.
1
6. Size and Nature: US$ 1.5 million. Disbursements will be in the form of grants to eligible
applications, approved by the Evaluation Function Steering Group (EFSG)3. When 80% of the
initial allocation has been disbursed, the EFSG will request the Strategic Resources Allocation
Committee (SRAC) for replenishment on the basis of evidence of use for conduct of DEs.
Criteria
7. A key feature of the CEF is that it supports and reinforces desired behaviour change. It is
incentive-based. Under the Evaluation Policy, evaluation must be planned and budgeted
adequately in project documents at the time of project design. The CEF will only be available
for those DEs which complied with this. It will not be used to fill gaps due to planning and/or
budget omissions.
8. The criteria shown below are designed to support that intent and to be straightforward,
transparent and unambiguous, so that requests can be approved rapidly by the EFSG. They
have been developed in consultation with key stakeholders, building on relevant experience
from the Operations Evaluation Series and DEs planned to date.
A) Eligibility criteria:
i. Planning/ budgeting: The DE was actually planned and budgeted in the project
document and/or in the Country Office’s Annual Performance Plan (APP) (and, in
future, Country Strategic Plans and CO Management Plan - COMP)4.
ii. Evaluability: Though under-funded, the operation/activity to be evaluated has been
implemented to an extent that makes evaluation possible. The threshold will vary from
case to case, but would normally be around 30% (of project value).
iii. Utility: The DE is timed in such a way to ensure that the evidence generated feeds into
management decisions on future programmes or policies.
B) Criteria for assessing priorities and size of grant:
i. Reasonable cost: the DE budget should be ‘reasonable’ given the scope, purpose and
proposed team for the DE concerned. Costs will vary according to the scope, methodology
and contracting mode, but an average cost of US$130,000 will be used as an indicative
starting point for estimation of reasonableness for a typical DE; US$250,000 for Impact
Evaluations5.
ii. Relevant to decision making: the DE should be of a scope, purpose and nature
appropriate to feed evidence into programming decisions and in line with DE types listed
in the Evaluation Policy.
iii. Size of the COs (as defined in WFP Management Plan – see Annex II)6: large and very
large COs are expected to have fewer genuine resource constraints than other COs. Small
and very small COs are expected to have the greatest and most frequent resource
constraints. Therefore, an allocation from the CEF is likely to be larger and more
frequently given in favour of small or very small COs.
3
See EFSG membership in Annex III.
4
Since Trust Fund projects are typically funded up front, the budget for evaluation should be available, provided it was included in the
project design. Therefore, very few Trust Funded projects are expected to face genuine resource constraints and be eligible for CEF.
5 This is based on the historical experience of the series of 65 Operations Evaluations (2013-2016) and of DEs planned in 2016 under
the Evaluation Policy - in the absence of reliable figures on the WFP’s decentralized evaluations prior to 2016.
6 This criterion draws on learning from the series of centralized Operations Evaluations, now concluding.
2iv. Need and affordability: DE budget as a proportion of the actual resource-based DSC
available to the CO for the year in question. This will be based on information of the
resource-based budget funded, programmed and spent.
v. Promoting fulfilment of evaluation coverage norms: higher priority is given to
Country Offices where the last evaluation was more than 3 years before the request.
vi. The CO is required to contribute at least 30% to the costs.
Anticipated Demand
9. As at November 2016, globally, 2 DEs have been completed under the new Evaluation Policy,
20 are on-going (to be completed during 2017) and 16 more are planned to start in 2017. Two
have already informed OEV that they will rely on CEF funding to proceed.
10. Since the CO is required to contribute at least 30% to the costs, using the guide prices (above)
of US$130,000 for ‘regular’ DE and US$250,000 for decentralized impact evaluation, the
maximum application per evaluation to the CEF will be US$91,000 (‘regular’ DE) or
US$175,000 (impact DE). Thus, with maximum funding approval for each application, the
CEF would be able to support 8 impact DEs or 16 ‘regular’ DEs, before replenishment is
required. Of DE’s currently planned for 2017, about 40% are impact evaluations.
Fund Management and Process
11. The EFSG is responsible for the management and decisions on allocation of this component of
the CEF on the basis of applications received from RDs7.
12. Allocation decisions will be made by the EFSG, taking into account the advice from review of
applications by each applicant’s Regional Bureau (RB), and then OEV and Budget and
Programming Division (RMB) against the criteria above. In addition, Project Budget and
Programming Service (RMBP) will provide to each CEF decision making meeting of the EFSG
a global overview of the status of DSC funding available in resource-based budgets so that the
degree of need is transparent.
13. Annex I shows a summary flow-chart of the CEF process from submission of application
through to disbursement. Annex II includes the CEF Application Form and Annex III the
proposed Assessment Form to be used for reviewing applications. Table 1 (below) summarizes
the key actors and their roles.
Table 1. Summary of Fund Management Roles
EFSG Fund management & allocation decisions
OEV Secretariat to EFSG and the CEF; assessment of applications
RB REO with RD approval Screening and submission of applications to the CEF
Secretariat
Project Budget and Provision of data on eligibility, prioritization and grant size
Programming Service (RMBP) Role in assessment of applications
Disbursement
Organizational Budgeting
Service (RMBB) Fund establishment and replenishment
7 Corporate Evaluation Strategy 2016-2021 (para 94).
314. Frequency of disbursement decisions: This will depend on actual demand, but is
recommended to be once every 2 months. This will enable timely start and completion of DE’s
which are dependent on CEF support in order to ensure that the evidence generated can feed
into relevant decisions on future programmes (utility).
15. All complete and eligible applications ready for EFSG consideration will be uploaded to the
Teamwork Space (TWS) EFSG Library on the CEF by OEV after clearance by the Director of
Evaluation. The corresponding assessment and record of approvals will also be kept in that
Library.
16. Monitoring and Reporting on use of the CEF: will be coordinated by OEV as the CEF
Secretariat, based on data from WINGS, extracted and collated by RMBP. Monitoring data will
be incorporated in the dashboard of Key Performance Indicators (KPIs) on evaluation function
reporting, for use by the EFSG. Annual reporting on the first CEF allocation will include
reflection on the additional rations enabled by this allocation from the Programme Support and
Administration (PSA)8.
Progress review
17. The criteria, process and template forms may be adjusted after the first round of CEF
allocations, as necessary to ensure smooth functioning of the CEF. The CEF will be included in
the ‘proof of concept’ review of the DE system scheduled for Quarter 3 2017.
8
Required by RMBB because funded from the additional USD 35 million for centralized services supporting COs as per Table IV.1:
Main areas of increase in the PSA Budget of the Management Plan approved at EB.2/2017.
4Annex I: Contingency Evaluation Fund Process
*The Regional Monitoring and Evaluation Advisors will provide support until the Regional Evaluation
Officers are on board and operational at the beginning of 2017.
5Annex II: CEF Application Form
CONTINGENCY EVALUATION FUND (CEF) Application N. [Insert RB,
APPLICATION FORM Year/Progressive
application No]
RBX/YEAR/0X
To be filled in by RB
Submission Date
1. GENERAL INFO ON THE APPLICANT
[To be filled in by CO]
RB ☐RBB ☐RBC ☐ RBD ☐ RBJ ☐ RBD ☐ RBN
RB Contact Person Name: Functional Role:
CO
CO Contact Person
Evaluation Committee Chair Name: Functional Role:
CEF Contact Point Name: Functional Role:
Time period within the evaluation
Evaluation Title scope
(Please attach TORs) Joint ☐ Regional ☐
if so, specify which partners: if so, specify which countries:
Planned start date9: Planned completion date10:
Expected Evaluation duration Latest possible completion
Latest possible date11:
date12:
Project/Activity
Type of Evaluation Project/Activity No
Title
☐Operation(s)
☐Impact
Type of Evaluation,
Project/Activity Title and No if ☐Activity
relevant ☐Pilot
☐Theme
☐Transfer modality
☐Other
Expected duration of intervention
Starting date: Completion date:
to be evaluated
Total budget of intervention to be evaluated US$
Implementation of intervention to be evaluated (US$ funded value
%
versus total intervention cost)
9 An evaluation starts when Inception Mission commences.
10 Completion occurs when the final evaluation report is approved.
11 In terms of timeliness and utility to feed into the decision making process.
12 In terms of timeliness and utility to feed into the decision making process.
6US$:
Total planned budget for evaluation in original plan (CO work plan: % of DSC:
original Project Budget, CP Budget, COMs, APP) (Please attach
evidence of plan & budget for the under-funded DE) % of total operation costs:
% of total activity costs:
Total evaluation cost funded by CO US$: Source:
Total requested from CEF US$: % of total evaluation cost:
2. DETAILED INFORMATION ON THE DECENTRALIZED EVALUATION
[To be filled in by CO]
2.1 Indicate the planned date and Explain the nature of key programming decisions into which the DE
is supposed to feed evidence
Key programming
Nature Planned date
decision
☐Design of new interventions
☐Revision of current interventions
☐Implementation of current interventions
☐Closure of current interventions
☐Scale-up of pilot / innovation / prototype
☐High-risk interventions13
☐Before third repeat of an intervention of similar
type and scope
☐Other (please specify)
2.2 Explain the reasons for the shortfalls in CO funding compared to planned budget for evaluation in
original plan
2.3 Indicate the year of last evaluation of the activity concerned and specify whether the evaluation
was DE or CE
Year Title of evaluation CE/DE
13
Based on corporate enterprise risk management.
73. ADDITIONAL INFORMATION [To be filled in by CO]
3.1 Indicate the date of last evaluation (s) in the CO and specify whether the evaluation was DE,
Operation Evaluation, Country Portfolio Evaluation over the last two years 14
Year Title of evaluation CE/DE
3.2 Evaluation co-funder(s) – if any
Name Contact details % Contributions in US$
3.3 Other remarks (at CO’s discretion)
4. RB ASSESSMENT OF APPLICATION AND PRIORITARIZATION
[To be filled in by RB/REO15]
4.1 Score each question on a scale of 0 (Disagree), 1 (Agree), 2 (Strongly Disagree)
Key Question RB assessment
1. Is the evaluation timely to feed into the decision making process? (yes/no)
If yes, which decision(s) and date of decision.
2. Are there other CEs contributing to the above decision-making process? (yes/no)
If yes: which?
3. Are there DEs contributing to the above decision-making process? (yes/no)
If yes, which?
4. Was the latest evaluation undertaken by the CO conducted more than 3 years
before the submission of the CEF request? Yes/no
5. Is the DE cost reasonable for scope, purpose & contracting mode 16? Yes/no
(If no, application should be re-formulated by CO with RB advice)
TOTAL
4.2 Regional Director Prioritization
High ☐ Medium ☐ Low ☐ [Please tick the box]
Rationale [Open Text, taking account of regional priorities and/or national context]
14
Not necessarily of the activity being evaluated now. Reviews, baseline studies should not be included.
15 Regional Monitoring and Evaluation Advisors (RMEAs)s until Regional Evaluation Officers (REOs) will be on board in each RB.
16 Refer to Indicative Evaluation budgets by operation size in page 15 of DEQAS (average cost: US$ 126,255. For Impact Evaluation the
cost is between US$ 200,000 and US$ 400,000 with the mode US$ 250,000.
8Annex III: Assessment Form for a CEF Request
ASSESSMENT FORM FOR A CONTINGENCY EVALUATION FUND REQUEST
RB:
Application No #
Decision E-consultation No & Date: Meeting No & Date:
(EFSG)
This form is to be used in conjunction with the separate ‘CEF Application Form’. It aims to provide:
1. A framework for systematic and objective assessment of each application for
funding from the CEF, based on the agreed CEF criteria.
2. An advice to the EFSG for its decision on whether and at what level CEF funding be
approved for the application concerned.
3. The basis for feedback to applicants.
9A. RELEVANCE OF THE DE: ASSESSED BY OEV17
[To be filled in by OEV]
Advice Name: Function: Date:
(OEV)
Key Questions Advice (OEV) Source of Remarks
Information
0 if = Disagree TOR
1. Is the evaluation timely to feed into
1 if = Partially agree Other (please
the decision making process?18
2 if = Agree specify):
0 if ≥ 3 CEs TOR
2. Are there other CEs contributing to
1 if = 1 or 2 CEs Other (please
the above decision-making process?
2 if = 0 CEs specify):
0 if ≥ 3 DEs TOR
3. Are there DEs contributing to the
1 if = 1 or 2 DEs Other (please
above decision-making process?
2 if = 0 DEs specify):
0 if = Yes within 3 TOR
4. Was the latest evaluation undertaken years Other (please
by the CO conducted more than 3 years 1 if = No, more than 3 specify):
before the submission of the CEF years
request? 2 if = 0 DEs in the
country
0 if = Disagree TOR
5. Is the DE cost reasonable for scope,
1 if = Partially agree Other (please
purpose & contracting mode19?
2 if = Agree specify):
Sum of the score
Total
17
Only those countries with a minimum of 6 are eligible.
18
Score has to be minimum 1 or 2.
19
Refer to Indicative Evaluation budgets by operation size in page 15 of DEQAS (average cost: US$ 126,255. For Impact Evaluation the cost is between US$ 200,000 and US$ 400,000
with the mode US$ 250,000.
10B. SIZE OF CO: RMBB ASSESSMENT OF APPLICATION
[To be filled in by RMBB]
Advice Name: Function: Date:
(RMBB)
Key Question Advice (RMBB) Source of Remarks
Information
1. CO size classification (as per annual 0 if Large & Very
WFP Management Plan) large
1 if Medium
2 if Small & very
small
C. APPLICANT NEED: RMBP ASSESSMENT OF APPLICATION
[To be filled in by RMBP]
Advice Name: Function: Date:
(RMBP)
Key Question Advice (RMBP) Source of Remarks
Information
1. What is the proportion of the proposed DE
budget to the DSC available for the year to the
applying CO (expressed as %)?
2. What percentage of the applicant CO’s total
resource-based budget for the relevant year is
funded?
3. What percentage of the applicant CO’s total
resource-based budget for the relevant year is
programmed?
4. What percentage of the applicant CO’s total
resource-based budget for the relevant year is
spent?
Conclusion on level of need for CEF funds High/Medium/Low/None:
11SECRETARIAT RECOMMENDATION
TO EFSG
YES/NO
1. Recommended for funding
US$
2. If recommended, advised size of CEF
contribution
3. Any special conditions for funding (Specify here)
EFSG DECISION
YES/NO
1. Approves this DE for funding
US$
2. If approved, size of CEF contribution
3. Any special conditions for funding (Specify here)
12Annex III: EFSG membership
Composition of the EFSG Membership
Chair: Chief of Staff
Secretary: Director of Evaluation
Membership:
Regional Directors
Director, Policy and Programme
Director, Budget and Programming
Director, Government Partnerships
Director, Performance Management and Monitoring
Director, Human Resources
Chief Information Officer
Director, Innovation and Change Management
13Annex IV: List of Acronyms
APP Annual Performance Plan
CE Centralized Evaluation
CEF Contingency Evaluation Fund
CO Country Office
COMP Country Office Management Plan
CPE Country Portfolio Evaluation
CSP Country Strategic Plan
DE Decentralized Evaluations
DEQAS Decentralized Evaluation Quality Assurance System
DSC Direct Support Cost
EFSG Evaluation Function Steering Group
FFR Financial Framework Review
KPI Key Performance Indicator
PSA Programme Support and Administration
RB Regional Bureau
RD Regional Director
RMEA Regional Monitoring and Evaluation Advisor
REO Regional Evaluation Officer
RMB Budget and Programming Division
RMBB Organizational Budgeting Service
RMBP Project Budget and Programming Service
SRAC Strategic Resources Allocation Committee
TWS Teamwork Space
14You can also read
