United Nations Office of Internal Oversight Services

Evaluation Function

The Inspection and Evaluation Division (IED) within the OIOS is co-located with the two other Secretariat oversight divisions; the Internal Audit Division and the Investigations Division. IED undertakes inspections and evaluations to help assure Secretariat programmes' accountability for attaining their mandates, while in the process fostering institutional learning, and improvement, through reflection by programmes and Member States on performance and results.

IED conducts evidence-based evaluations of the UN Secretariat's programmes with three main aims:

(i)  To help programmes deliver better results;

(ii)  To enable the UN as a whole to learn from their experiences

(iii)  To provide public accountability.

And the focus of IED's work is on assessing Secretariat programmes for their relevance, efficiency, effectiveness, and impact.

The inspections and evaluations conducted by IED are considered "external" to the Secretariat programmes, though they are considered "internal" to the Secretariat. For example, an IED evaluation of UNEP would be considered an external evaluation of UNEP, but to the General Assembly, it is considered an "internal" Secretariat evaluation of a Secretariat programme. Evaluations conducted by the evaluation units of Secretariat programmes are considered "self-evaluation" insofar as these programme evaluation units typically report to the programme managers, and the evaluations are managed by the programmes themselves.

The Secretariat evaluation policy, which serves as guidance for OIOS' evaluation work, has been in place since 2000 - these are the UN Regulations and Rules Governing Programme Planning, the Programme Aspects of the Budget, the Monitoring of Implementation and the Methods of Evaluation (PPBME) set forth in the Secretary-General's Bulletin - ST/SGB/2016/6. Internally, in order to support accountability and learning for its own work, OIOS established in 2008 the OIOS evaluation policy to guide self-evaluation work for all three oversight divisions.

IED has a total of 27 staff posts all of whom are based in New York. The Division has close ties with, and is mandated to provide methodological guidance for, the work of all the evaluation functions of the UN Secretariat programmes.

IED is mandated to cover the work programme of the entire UN Secretariat:

-  peace and security (DPKO, DFS, DPA, ODA, DSS, PBSO, UNODC, OOSA)

-  sustainable development (DESA, UNEP, UN-Habitat, UNCTAD, ITC, ECA, ECLAC, ESCWA, ECE, ESCAP, OHRLLS, OSAA)

-  human rights and humanitarian work (OHCHR, UNHCR, UNRWA, OCHA)

-  UN Secretariat management and support services (DGACM, DM, UNOG, UNOV, UNON, DPI, EOSG, OLA)

-  UN Women

Promoting a culture of evaluation in-house

Providing methodological guidance to strengthen the rigor and use of evaluation within the United Nations Secretariat entities is a key function of IED. As such, IED supports colleagues from all Secretariat evaluation units through the provision of guidance documents and ad hoc evaluation advice. Given scarce resources to dedicate to much needed outreach work, IED utilizes UNEG as a forum through which to communicate and network with Secretariat evaluation colleagues, to help promote the sense of community amongst UN evaluators, and foster the culture of evaluation.

Snapshot

Evaluation Policy:

• ST/SGB/2016/6 - The UN Regulations and Rules Governing Programme Planning, the Programme Aspects of the Budget, the Monitoring of Implementation and the Methods of Evaluation (PPBME)

• UN OIOS (Internal) Evaluation Policy, 2008

Current Priorities:

• Improved Risk Assessment Methodology to identify most pressing and strategically relevant subjects/topics for evaluation

• Improved Quality Assurance System to enhance the quality of IED evaluations

• Improved Communications and dissemination of evaluation results to improve use and impact of evaluation work.

• Developing a strategy to support capacity building for evaluation in the UN Secretariat, in consultation and partnership with Secretariat evaluation functions.

Human Resources (reference date: July 2014):

• Directors: Total 1; M=1

• Evaluators : Total 21 ; F=14 and M=5 ; 2 vacant

• Support staff : Total 4 : F=2 and M=2

• Decentralized evaluation staff : None

Annual evaluation expenditure (2013), excluding staff costs:

• $4,580,450

Evaluations produced in 2014:

• 11 evaluations

Key resource: web link/key document here

• OIOS Evaluation website;

• ST/SGB/2000/8 - PPBME (http://www.un.org/Docs/journal/asp/ws.asp?m=ST/SGB/2016/6)

• UNOIOS (Internal) Evaluation Policy, 2008;

• IED Evaluation Manual (http://www.un.org/Depts/oios/pages/ied_manual_v1_6.pdf)

Independence

IED is located within the Office of Internal Oversight Services (OIOS) of the United Nations Secretariat. The IED Director reports to the Under-Secretary-General for Internal Oversight Services. The USG for OIOS, as head of all three oversight functions, is considered the principal officer (i.e. head) for evaluation (as well as audit and investigations) for the UN Secretariat. The USG, as head of evaluation, controls the evaluation budget and has full independence and discretion in issuing evaluation reports.

Agenda Setting & Evaluation Planning

For independent evaluations, IED prepares a biannual work plan that is submitted to the USG of OIOS for approval. This work programme addresses the timeliness of evaluations, the priority areas in need of evaluation, and the resources necessary for evaluation. Once approved, IED informs the General Assembly main subsidiary organ responsible for evaluation, the Committee for Programme and Coordination (CPC) about its work plan through its biennial report on evaluation (for example, see A/68/70; pp  25-27.). This is a General Assembly report that is publicly available.

For discretionary (self) evaluations managed by programme managers, guidance on evaluation planning is provided in "Managing for Results: A guide to using evaluations in the United Nations Secretariat, 2005".

Quality Assurance

Two key quality assurance mechanisms are in place for evaluations conducted by IED: A common set of guidance materials to be used throughout the evaluation process and a peer-review process for completed reports. IED has published an evaluation manual which contains guidance on what constitutes high-quality evaluative evidence and data. This guidance is based on the UNEG Norms and Standards. The current quality assurance system uses a combination of peer, supervisory and management reviews focused at four stages of the evaluation cycle; the evaluation design stage, focused on the quality of the TOR, the data collection stage, focused on the quality of the data collection instruments, the drafting stage, focused on the quality of the draft report, and lastly the final report stage, focused on overall quality of the final product. IED has also begun establishing advisory expert panels to review its evaluations. Through 2013, IED reviewed its Quality Assurance process and has updated its Inspection and Evaluation Manual.

Use of Evaluation

Management responses to evaluation studies are systematically prepared by the head of the entity that is being evaluated. This is done through a Management Response Letter (MRL) and Action Plan. The Action Plan describes how recommendations are followed-up on, who is responsible for the follow-up, and the follow-up status.

On behalf of the Secretary-General, both the Department of Management of the UN Secretariat and OIOS systematically monitor the implementation of evaluation recommendations. Monitoring reports are regularly sent to the Governing Body of the organization. All of IED's reports to the General Assembly are publicly available on the UN website. The rest of IED reports are issued as "reports to the manager", and are both kept on file in the Division, as well as made available to all UN Secretariat staff through the UN Secretariat intranet (ISeek). Lessons from evaluations are also partially extracted. These lessons are then communicated through workshops.

All IED reports to the General Assembly are reviewed by the relevant inter-governmental bodies. These are, typically, the Committee for Programme and Coordination and the Fifth Committee (i.e. the Administrative and Budgetary Committee). Depending on the subject matter, IED reports have also been submitted to other inter-governmental bodies, such as - the ECOSOC, the Special Committee for Peacekeeping, the Committee on Information and the Committee on Conferences.

IED reports typically make recommendations that are considered by the relevant inter-governmental bodies, who can choose to endorse or reject, or as desired, through their deliberations on the information provided by the evaluation, formulate their own recommendations, which upon resolution by the General Assembly, become mandates which the subject programmes will have to implement. For example, past IED reports have supported decisions to restructure programmes (DPKO and DFS) as well as increase resources (DPA).

Providing methodological guidance to strengthen the rigor and use of evaluation within the United Nations Secretariat entities is a key function of IED. As such, IED supports colleagues from all Secretariat evaluation units through the provision of guidance documents and ad hoc evaluation advice. Given scarce resources to dedicate to much needed outreach work, IED utilizes UNEG as a forum through which to communicate and network with Secretariat evaluation colleagues, to help promote the sense of community amongst UN evaluators, and foster the culture of evaluation.

Joint Evaluation

OIOS was a founding member of the United Nations Evaluation Group, and was involved in the preparation and endorsement of the landmark norms and standards for evaluation in the UN system. It continues to be deeply involved in enhancing and harmonizing evaluation practices across the UN system. The IED Director currently serves as the UNEG Chair.