United Nations Environment Programme

Evaluation Function

The UNEP Evaluation Office is located in the UNEP Secretariat in Nairobi. The UNEP evaluation policy, published in 2009, is guided by four key principles: learning, accountability, independence and ethics. The UNEG Professional Peer Review of the Evaluation Function in UNEP, conducted in 2012, found the evaluation policy to be fully consistent with the UNEG Norms and Standards for Evaluation in the UN System. The policy states that "UNEP has adopted an approach to evaluation where the emphasis is on UNEP's achievement at a programme level". Evaluation in UNEP serves two organizational objectives: i) enabling management to improve programmatic planning, implementation results, monitoring and reporting, and ii) providing substantive accountability to UNEP's Governing Council, donors and the general public. The Evaluation Office is responsible for conducting evaluations of sub-programmes and projects, as well as thematic evaluation and management of studies within the organization. Evaluation types include: project level evaluations, quality of project supervision reviews, sub-programme evaluations, impact evaluations, management studies, meta-evaluation, synthesis and special studies.

In 2014, the UNEP Evaluation Office had five professional staff positions, and three support staff. The annual evaluation budget to fund higher level evaluations totals approximately USD 220,000 per annum excluding staff costs. Total annual expenditure on project level evaluations through project budgets overseen by the Evaluation Office is approximately USD 1,200,000. There are no decentralised evaluation arrangements.

The core focus area / priority for the UNEP Evaluation Office is to conduct higher level strategic evaluations whilst managing a large number of project-level evaluations. Evaluation processes frequently make use of external consultants with intensive oversight and quality assurance processes provided by UNEP evaluation staff. Project level evaluations feed into higher level evaluations.  Evaluation staff are also involved in corporate level training on results based management, and providing evaluation findings to improve programme / project design, planning and management.

Promoting a culture of evaluation in-house

The Evaluation Office conducts evaluations in a transparent and consultative manner, giving opportunities to evaluation stakeholders to provide substantive inputs and comments to evaluation TORs, the inception report and the main evaluation report, and putting much emphasis on the learning objective of the evaluation. Even though evaluations are expected to provide a solid basis for accountability on performance, they are conducted as a participatory learning exercise rather than an external inspection. This helps making in-house stakeholders consider the evaluation as a useful exercise rather than a threat, and promotes collaboration and involvement of stakeholders in the evaluation process. UNEP Evaluation Office has actively engaged in the development of in-house guidance for project planning, management and evaluation. The Office has made substantive contributions to UNEP's Programme Manual and associated staff training modules that promote evaluation as part of a broader results-driven and accountability-focused culture. Theory of Change approaches have been successfully championed by the Evaluation Office as a means to develop strong project designs that have high evaluability.

Snapshot

Evaluation Policy:

• UNEP Evaluation Policy, 2009

Priorities/Competitive advantage:

• Methods and processes

Human Resources:

• Head of Evaluation: M
• Evaluators : Total 6; F=2 and M=2 + 2 Vacant
• Support staff : Total 3 (F)
• Decentralized evaluation staff: None

Evaluations produced per year by the UNEP Evaluation Office

• In 2013, three corporate/thematic/strategic evaluation and more than 30 project level evaluations were conducted.

Key resource:

• UNEP Evaluation Office website.

Independence

The UNEP Evaluation Office is located within the UNEP Executive Office and reports to the Executive Director. The Director of Evaluation has the authority to sign off on/distribute evaluation reports to the Governing Body and/or Executive Head without prior clearance from other parties within or outside the Organisation and has a good measure of control over evaluation expenditure. The Governing Council approves the operational budget of the Evaluation Office as proposed by UNEP management as part of the overall budget of the organisation in its review of UNEP's Programme of Work (PoW). Budgets for project level evaluation are a requirement for project approval. The Evaluation Office is responsible for authorizing project expenditure against project level budget lines.

The 2009 evaluation policy stipulates that evaluation is to have both "organizational and behavioral independence". Accordingly, the Evaluation Office: develops its work programme in consultation with senior management; selects and prioritizes evaluation subjects, with due consideration given to those suggested by senior management / Governments; develops Terms of Reference (ToR) for evaluations, selects and recruits evaluators and manages the resources allocated for evaluations within the organisation without interference; conducts evaluations without interference from senior management; follows-up and reports on management responses and the implementation of evaluation recommendations; submits uncensored reports to senior management and relevant stakeholders without fear of recrimination or dismissal for such; and publicly discloses evaluation findings.

The evaluation function was subject to a UNEG peer review in 2012 and was found to be independent and well established. The peer review also highlighted that evaluation has been growing in importance through the reform process initiated in 2006 and with the increased focus on managing for results.However, the limited core budget and the lack of fungible financial resources for evaluation limits the intensity of effort possible at the strategic level.

Evaluation consultants sign a statement of potential conflict of interest and there are rules and mechanisms, allowing them to report discreetly on cases of wrongdoing.

Agenda Setting & Evaluation Planning

UNEP prepares a biennial evaluation work programme for evaluation which forms a part of UNEP's biennial PoW and Budget. The work programme addresses the timing of evaluations with a view to inform decision-making, priority areas most in need of evaluation, and specification of necessary resources for evaluation (partly). The evaluation work plan is reviewed by the Senior Management Team (SMT) and approved by the Executive Director. The evaluation policy stipulates that the Evaluation Office "shall be free to select evaluation subjects taking into account inputs from UNEP's SMT". The work programme is submitted for review and/or approval to the UNEP Governing Council and Executive Director. Annual workplans are also developed and are updated on an ongoing basis. Annual work planning includes a formal priority setting process that assesses the relative importance of evaluations in terms of their strategic relevance, value for accountability, utility for learning and overall feasibility.

Stakeholder involvement and promoting national evaluation capacity development

Stakeholders are systematically consulted in the planning/design, conduct of and follow-up to evaluations. Peer reviews or reference groups composed of external experts are used in the evaluation process for higher level evaluations. The evaluation teams include professionals from the countries or regions concerned.

Quality Assurance

The quality of evaluation reports are systematically controlled by a set of quality rules that cover, evidence-based evaluation report, findings and recommendations, conciseness of executive summary, methodology and limitations, coverage of TORs, evidence of results, quality of recommendations and how the evaluation dealt with differences in opinion. Quality assurance processes apply at inception, draft report and final report stages. Formal ratings for the quality of evaluation reports at the various stages of an evaluation allows the Office to monitor the value-added by its internal review processes.

Use of Evaluation

An explicit response to the evaluation is required from the responsible officer of the entity evaluated. Responses are in the form of Management Response Matrix and Management Response letters which are reported to the Executive Director.

Evaluation results are disseminated widely within and outside the organisation. Lessons are systematically extracted and communicated through meetings with senior management, meetings with operations management and biennial evaluation reports. The evaluation policy stipulates that all evaluation reports are to be made public and a repository of evaluation studies is available on the UNEP website. Evaluation recommendation compliance reports are prepared on a six-monthly basis.

Joint Evaluation

The evaluation policy sets out that UNEP will "collaborate with other UN system organisations and external partners to continuously refine methods for evaluation, set standards and guidelines that reflect international best practice and promote their application within the organisation". The policy also provides for periodic review of the evaluation function of UNEP by an independent external review team, with the decision to review to be made by the Executive Director.