Organisation for the Prohibition of Chemical Weapons

Evaluation Function

The OPCW evaluation policy was established in 2012 but remains unpublished. The evaluation capacity of OPCW is part of the internal oversight mechanism of the Organisation and is managed by the Office of Internal Oversight (OIO). The Director of OIO is therefore, de facto, the head of the evaluation function. The evaluation policy states that the OIO will "assist the Director-General in the management of OPCW's resources through internal audit, inspection evaluation, investigation and monitoring in order to enhance the efficiency, and economy of the operations of the OPCW". The policy further states that evaluation will be used to review the efficiency and effectiveness of the implementation of programmes and mandates of OPCW as well as "cover individual staff performance in instances where such performance has a major impact on the efficiency and effectiveness of a programme".

In 2011, the OIO had a Senior Evaluation Officer as full time staff member for the evaluation function. An assistant provided administrative support both to the Internal Auditor and the Senior Evaluation Officer. A central expenditure for evaluation in 2011 was USD 122,025. OPCW do not conduct any decentralized evaluations. 

The core task of the evaluation function is to lead an evaluation team. Other tasks include developing evaluation capacities in OPCW.

Promoting a culture of evaluation in-house

In line with the evaluation policy, OIO develops and updates tools, guidelines and methodologies for evaluation as required to support effective implementation of the policy. The standard operating procedure provides useful guidelines. 

See: OPCW Evaluation Policy (2012)

Snapshot

Priorities/Competitive advantage:

• To provide support to the Director-General in promoting efficiency, effectiveness and relevance of programmes and management systems and procedures.

Human Resources:

• Director: M
• Evaluators: Total 1 (M) (not including the Director)
• Support staff: Total 1 (F)
• Decentralized evaluation staff: None

Evaluations produced per year by the central unit and by decentralized units

• OIO published more than thirty Evaluation Reports until 2018, classified as either corporate/thematic/strategic or operations evaluations.

Key resource:

• OPCW website (https://www.opcw.org)

Independence

The Director, OIO reports directly to the Director-General, who is the executive head of the organization. Whilst the Director, OIO has the authority to sign off on reports to the Director-General, only the Director-General has the authority to then sign off on and distribute evaluation reports to the Governing Body. The Director, OIO has control over evaluation expenditure. 

Evaluators sign a statement of potential conflict of interest and there are rules and mechanisms, which allow evaluators to report discreetly on cases of wrongdoing. 

Agenda Setting & Evaluation Planning

The OIO prepares an annual work programme, which addresses timing of evaluations to inform decision-making organs, priority areas most in need of evaluation and allocation of necessary resources for evaluation. The OIO invites "senior officials to suggest/propose policy, programme, activity or thematic issues for evaluation", which are considered during the finalization of its annual work programme. Evaluation topics are selected based on a risk-based approach which takes into account various risks assessed by their likeliness and potential impact. The Director, OIO reviews and approves the objective and scope of each evaluation assignment and the methodologies to be applied.

The annual work programme of the OIO is submitted for review and approval to the Director-General.

Stakeholder involvement and promoting national evaluation capacity development

Stakeholders are systematically consulted in the planning/design, conduct of and follow-up to evaluations. Peer reviews or reference groups composed of external experts are not used in the evaluation process. Evaluation teams do not include professionals from the countries or regions concerned.

Quality Assurance

The evaluation policy adopted in 2012 states that an internal quality assurance system "shall identify processes and products of key importance to the quality of its work and define the rules for evaluation reports". A standard operating procedure on the management of audits, evaluations and advisory services in the OIO has been issued establishing the procedures to be followed. The quality of evaluation reports is systematically controlled by a set of quality rules that cover evidence-based evaluation report, findings and recommendations, conciseness of the executive summary (overall assessment), methodology and limitations, coverage of terms of reference, evidence of results and quality of recommendations. 

Use of Evaluation

The evaluation policy stipulates that evaluation reports are to be submitted in two stages, i.e. draft and final, whereby draft reports are issued to the concerned managers for comments. The head of the entity evaluated issues an explicit response to the evaluation recommendations in the form of a Management Response Matrix which is used for the follow-up review of the implementation whose status is reported to the Director-General and to the Management Board on a quarterly basis. Evaluation results are widely disseminated within the Organisation and lessons are systematically extracted and communicated in annual evaluation reports. The Director of the OIO is responsible for preparing a summary report on internal oversight activities, including evaluation, for each year. Evaluation studies are not made available on the Internet.

Joint Evaluation

The OIO has conducted alone its evaluations and has not participated in joint evaluations outside the OPCW.