News Center

By Dr. Juha I. Uitto, Director, Independent Evaluation Office, Global Environment Facility

Evaluating Environment in International Development, edited by Juha Uitto, was published in 2014 by Routledge. As stated by Nidhi Khattri (World Bank, Independent Evaluation Group): “The book is a must read for anyone concerned about development and its effects on environment.”

This blog was developed based on a presentation made by Juha at a recent UNEG webinar.

Global environmental challenges in the post-2015 era

The whole world talks about sustainable development. The Millennium Development Goals (MDGs) are now coming to an end to be replaced by the Sustainable Development Goals (SDGs). As we know, the concept of sustainable development is supposed to denote development today that doesn’t compromise the ability of the future generations to develop their own world. Well, it’s now 23 years since the Rio de Janeiro Earth Summit and the world is still struggling with the notion of sustainable development, how to reach there, and how to measure progress.

What kind of development do we then need that meets these criteria? We commonly think that sustainable development encompasses economic, social and environmental sustainability. What I’m focusing on here is the environment that we all here depend on. Some people argue that with economic development also environmental problems are mitigated – in many ways, this argument is at the heart of the concept of sustainable development. We also hear how environmental pollution takes a toll on human health and the economy, for example through lost productivity. There is of course truth to this argument and as our societies have got richer, many of our local pollution problems have been reduced. Yet, while we have solved many local environmental problems, we’ve continued to perpetuate the global environmental problems.

We’re now facing what has been termed the Sixth Extinction, a loss of species at a rate that has not taken place ever during the time that humans have inhabited the planet. A quarter of all mammals are under threat of disappearing from the face of earth. For other species the situation is even grimmer: for instance, 6 out of the 7 species of marine turtles are on the verge of extinction. This extinction is almost entirely due to human activities. Deforestation caused by conversion of land for agriculture, forestry monoculture, industrial sites and settlements destroys habitats and modifies ecosystems that thus become hostile to a large number of species, large and small. Land lost under highways and transportation infrastructure not only takes space but fragments habitats so that wildlife cannot survive. What we all consume daily contributes to this. Some 80 percent of deforestation globally is caused by the production of three commodities only: palm oil, soy beans and beef.

The situation is not much better on the high seas. Three-fourths of the world’s fisheries are either fully or over-exploited. There is also a major problem with the acidification of oceans.

Climate change is the poster child of global environmental degradation and may well in the future become truly the biggest challenge to human existence. Many people argue that it already is – ask anyone living in a low-lying small island state that is being usurped by rising seas, battered by increasing storms, and whose water supplies are salinized by sea water. For them, sustainable development is a matter of life and death and they are the first victims of global environmental problems that they themselves hardly contributed to. Both mitigation of climate change and adapting to its impacts is very expensive and will divert resources from other important causes.

The role of environmental evaluation and its challenges

Although global environmental issues receive lots of attention, it becomes obvious why we are making so little headway when you look at the financial flows. Drawing on the results of the 5^th Overall Performance Study of the GEF, the total funding for environmental issues from public sources amounts to approximately $10 billion per year (of this, the GEF share is about $1 billion), while the estimated need would be ten times that. Compare that to the public subsidies – again an order of magnitude higher – that lead to overexploitation of natural resources and environmental degradation, and it’s not difficult to see why the environmental trends continue to sink.

So whatever we do and whatever we fund for the environment, should be focused and effective. And this is where evaluation comes in. Evaluation has in recent years become more and more powerful in determining, not only that the projects and programs that we implement are doing what they set out to do and spending tax payers money efficiently but – more importantly – that we are actually making a difference in people’s lives and the global environment.

Evaluation approaches and methodologies are constantly being refined and become increasingly rigorous. However, environmental evaluation has some challenges specific to it, partly because environmental issues are exceptionally complex and there is a poverty-environment-development nexus that is hard to crack and that has many dimensions. The challenges include the often long timeframes of environmental phenomena. Interventions with environmental goals seldom have equally long timeframes and can only be evaluated on their outcomes and potential of setting things on a path that will eventually lead to real environmental impacts, if several other things beyond the interventions materialize and no unforeseen threats arise. Similarly there is the geographical scale, which often differs from the boundaries of the intervention: watersheds are divided between jurisdictions, transboundary conservation puts strains on cooperation, etc. There are challenges with data availability, quality and credibility. There are specific issues pertaining to research designs that pose challenges for assessing attribution of environmental change to the policies and programs.

There are also new risks and uncertainties. Climate change has thrown a curve ball into the game and suddenly old models don’t apply. Uncertainty, risk, tipping points rule. We need to think carefully about what this means for evaluation and how we build in these factors into our approaches and methodologies.

Evaluation vs. valuation

To me evaluation is not value free. Whatever policy, strategy, program, project or intervention we evaluate, we evaluate from a value perspective. Fundamentally, if we agree with the goals of the intervention, then that forms the value bottom line against which we evaluate. Yet in practice this is not so easy to determine. One of the fundamental issues in sustainable development pertains to trade-offs. If the goal of sustainable development is to make people’s lives better while at the same time preserving the environment and the other, non-human lives on the planet, the goals (at least in the short term) are not always compatible. How do you weigh the relative value of, say, a flush toilet to a 1,000 families vs. the horn – and life – of an endangered white rhino? What if, instead of the flush toilet, the comparison concerns a flat screen TV (or two)? How far do we go to satisfy the human needs – and desires – at the cost of dwindling biodiversity?

Environmental economists have made a lot of progress in setting monetary value to a variety of things, such as ecosystem services, in the hope that making a rational economic argument would convince people to change their behavior. A good case has been recently made that the value of sharks (largely consisting of endangered species) as tourism attraction is now approaching their value as sources of shark fin for the soups of wealthy consumers in Hong Kong and around the world. I hope the shark fishermen – and diners – hear this message loud and clear and very soon. Still, to me personally, the economic argument is an insufficient one. I cannot place monetary value on life, whatever its form. If the world loses its last shark or rhino or wood bug, that’s an irreplaceable loss. Apart from the fact that we don’t even know how such a loss would affect ecosystem integrity, life and the diversity of life to me has intrinsic value. How does one build these perspectives into evaluation is challenging; however I believe that evaluators have a responsibility to think about such questions.

About the book and directions in environmental evaluation

I embarked on doing the book Evaluating Environment in International Development in order to clarify some of these issues and to bring together cutting edge work and research in the field of environmental evaluation in an international context to benefit the practice and, ultimately, to enable us to use evaluation to better define what works, in what context, and for whom, in devising interventions to help the environment that provides us all our sustenance.

The volume is divided into three main sections. The first sets the scene. It consists of three essays, including my introduction, and strategic chapters by Rob van den Berg in which he argues for evaluating environment from the point of public goods and Howard Stewart highlighting the importance of placing environment into the context of development and people’s aspirations. The second section focuses on approaches and challenges of the environment as the evaluand. It contains five chapters, starting with Andy Rowe’s exploration of the principles of evaluating sustainable development interventions; and ending with a chapter on evaluating normative work on the environment by Segbedzi Norgbey and Mike Spilsbury. The final section highlights lessons from international cooperation through eight distinct case studies. Vijaya Vadivelu’s work evaluating climate change and disaster risk reduction is an important contribution here.

The authors highlight different approaches to evaluation. What is common to many is variations of the so called theory based approach to evaluation. We use the theory of change – or logic model – to understand how interventions are designed and intend to achieve their often lofty goals. There are also examples of impact evaluation where rigorous quasi-experimental designs are used to tease out the effects and impact of a specific intervention.

Some of the directions emerging from reflection on environmental evaluation pertain to the need for moving away from evaluating individual interventions to evaluating the environment and development outcomes and impact – we have to keep our eye on the Big Picture and whether we are actually making a difference. Given the complexities, differing time and geographical scales, and data challenges, there is a clear need for putting together multidisciplinary teams and utilizing mixed methods for data collection and analysis. For instance, the joint evaluation by the GEF and UNDP Independent Evaluation Offices of the effectiveness of protected areas and protected area systems in biodiversity conservation that is now in its final phase has effectively combined a variety of methods, ranging from remote sensing to map forest cover change, and global databases for tracking wildlife abundance change, to quasi-experimental methods, replica watches and further to interviews and field visits for verification. This brings me to the point about big data and technologies, such as remote sensing and GIS, which I believe can truly help us see patterns and to track impacts on the ground. They are, however, not a panacea and more qualitative approaches are definitely needed to identify causalities, why things happen, and to understand the specifics of each case.

2015 and the way forward – a wish list for environmental evaluations

2015 is an important year both for the environment and for evaluation. Several large international events take place this year. In March there was the Third UN World Conference on Disaster Risk Reduction in Sendai, Japan. In November there will be the UN Climate Change Conference in Paris, expected to come up with a successor agreement to the Kyoto Protocol. In between, in September, the nations are supposed to agree on the SDGs. The SDGs have placed a lot of emphasis on targets and measurable indicators. What we evaluators have advocated for is to go beyond monitoring and indicators, and build in evaluation into the SDG processes so that we may better understand what is happening and why; what the causalities are that foster or the barriers that hamper the achievement of the goals?

Evaluation is gaining traction on the international scene. Last year, the UN General Assembly passed a resolution on National Evaluation Capacity Development to promote the use of evaluation especially in the developing countries. The year 2015 has also been declared the International Year of Evaluation.