Monitoring and Evaluation NEWS.
MandE NEWS
> Documents > Sustainability Monitoring

PLANNING FOR AND MONITORING OF PROJECT SUSTAINABILITY: A GUIDELINE ON CONCEPTS, ISSUES AND TOOLS
By M. Adil Khan (akhan@sltnet.lk) UNDP Senior Advisor Monitoring and Evaluation December, 2000

Table of Contents

Preface

  1. ABOUT THE DOCUMENT
    1. Purpose of the Document
  2. WHAT IS SUSTAINABILITY
    1. The Concept
    2. Dimensions of Project Sustainability
  3. SUSTAINABILITY ANALYSIS
    1. What is a Sustainability Analysis
    2. What is a Sustainability Strategy
  4. SUSTAINABILITY MONITORING / MEASUREMENT OF SUSTAINABILITY
    1. What is Sustainability Monitoring
    2. What are the Sustainability Monitoring Indicators
    3. When and How Sustainability Monitoring to be carried out

Annexes

  1. Planning for Sustainability : A Check List
  2. Project Preparation & Management and Principal Factors Affecting Project Sustainability
  3. Sustainability Monitoring Indicators and Measurement Tool
  4. Beneficiary Contact Monitoring Indicators: A Survey Tool

Preface

This document is one of several of guidelines, handbooks, source materials etc. that are planned to be produced under the UNDP supported results-based Monitoring and Evaluation activity at the Monitoring and Progress Review Division of the Ministry of Plan Implementation.

Project sustainability is a major challenge in many developing countries. Large number of projects implemented at huge costs often tend to experience difficulties with sustainability. All major donors, such as the World Bank, the Asian Development bank and the bilateral aid agencies have been expressing concerns on this matter. According to several recently conducted studies, while the trend with implementation is showing significant improvement, the trend with post implementation sustainability is rather disappointing - increasingly, less projects are being sustained. This means that while huge expenditures are being incurred by these countries in implementing projects, poor sustainability is depriving them from the returns expected of these investments. This further means that while the debts from development expenditure are increasing, gains from these expenditure have either not been forthcoming fully or been accrued at a lower rate.

Several factors are responsible for poor sustainability. Some are simple. Some are quite complex. Some are within the control of the project management, while others come as external threats.

Some of the factors can be (and indeed ought to be) taken care of right at the design stage of a project, whereas, others can be tracked and corrected during implementation, through monitoring. It is, therefore important that the factors that affect sustainability are articulated well and incorporated, as far as possible at the design stage. Later, the same factors can be followed up through monitoring.

This document presents various definitions of sustainability, highlights issues that affect sustainability and introduces tools which can be used for enhancing sustainability both at the planning as well as at the monitoring stage of a project. These tools are generic in nature and by no means are complete. Moreover, these tools also refer to several analysis, such as economic analysis or estimation of Economic Rate of Return (ERR) to determine the economic viability of project induced products and its returns. Similarly, social analysis is needed to measure equity element of a project etc. While a simple tool for Social Analysis has been provided (Annex 4) no details of estimating ERR is included in this document. Special skills will have to be acquired either within the project management itself or be made available from outside the organization, to conduct economic analysis.

Hopefully, the planners, managers and project sponsors find this document useful. Suggestions from users of this document will be highly appreciated as these will help in improving the quality of the document further, in future.

I wish to thank the Director General, Monitoring and Progress Review Division (of the Ministry of Plan Implementation) and his staff for the professional guidance given to me, for the preparation of this document.

November 30, 2000 Dr. Adil Khan, UNDP Senior Advisor, Monitoring and Evaluation, MPR Division, Ministry of Plan Implementation


1. About this Document

This document is a guideline that describes:

1.1. Purpose of this document

It is expected that this document will assist the planners, the managers and the monitors of projects to become aware of issues that are important for project sustainability and help in incorporating the elements of sustainability right at the design stage. A tool has been also provided (Annex 1) for assessing the degree of presence of the elements of sustainability. This tool can be used, while designing a project. Similarly, a tool has also been provided (Annex 3) for monitoring the sustainability.

Section 2 of this report discusses the 'concept' and various 'dimensions' of sustainability. Section 3 talks about various analysis that are needed to be undertaken to compute the values of different dimensions of sustainability. Section 4 describes the methodology of sustainability monitoring.

2. What is Sustainability

1. The Concept

Sustainability can be defined as the ability of a project to maintain its operations, services and benefits during its projected life time.

However, the issue of sustainability should also be seen within time and changing social, economic and political contexts. A project that is seen as worth sustaining today, may not be so in future. For example, in case of Sri Lanka paddy production which formed the mainstay of the agricultural economy only a few years ago, does not appear to be all that profitable nor is it sustainable, under the current market economic conditions. This and several similar examples from elsewhere illustrate that what was once considered important and sustainable, may not be so today. However, what is also important to note is that if a government for reasons better known to itself, decides to provide support to a certain activity and maintain its sustainability without regard to its economic viability, then that is a choice that the government has made and that the issue of sustainability of such an activity should be seen purely from the perspectives of a decision taken by such a government. Therefore, in case of situation of paddy production in Sri Lanka, government may decide to sustain paddy production by providing price and other forms of subsidy to the producers and ensure its production sustainability (and not economic sustainability). Such sustainability through subsidy will no doubt benefit the paddy producers and thus serve a social purpose, but only at the cost of other perhaps more profitable investments that could be made in the economy.

The paddy example cited above demonstrates further that while subsidy by the government has the ability to sustain a certain production process and thus benefit few producers, in this case the paddy farmers, such an act may equally constrain activities which have greater potentials and better and more sustainable returns from other endeavours. However, in some situation government may still be prepared to provide support to an activity which is economically unsustainable, but politically or otherwise justifiable. Thus when discussing sustainability, the question that needs to be asked:

What is it that one wishes to sustain?

In public sector activities this decision is crucial and should be made right at the planning stage. This will then help incorporating those elements that are relevant for 'sustainability'. However, in general project sustainability is defined as the percentage of project initiated goods and services that are still being delivered and maintained after five years of termination of implementation of the project; the continuation of local action stimulated by the project and generation of successor services and initiatives as a result of project built initiatives. This definition implies that sustainability concerns itself with:

The World Bank defines sustainability, "as to be the ability of a project to maintain an acceptable level of benefit flows through its economic life".

The core indicators that contribute to sustainability vary from sector to sector. For the economic sector projects, the core indicator will be economic and financial returns, whereas, the main indicator for social sector projects will be the extent and degree to which the delivery of goods and services, have been continued and the proportion of target area population that continue to receive the benefits from project activities.

2. Dimensions of project sustainability

There are several dimensions to project sustainability. Depending on the nature of a sector or a project each of these dimensions has the capacity to influence project sustainability in one or way or another. These dimensions are listed below.

Consideration of all these dimensions are Key to sustainability of projects. Experience suggests that weakening of any one of these has the potential to jeopardize the sustainability of the entire project, in the long run.

3. Sustainability Analysis

The multi-dimensional attributes of sustainability - as stated above, imply that to enhance project sustainability, a rigorous sustainability analysis is needed at the time of formulation of a project or a programme. It is expected that such an analysis which is to be followed up by development of a sustainability strategy will assist in incorporating the elements of sustainability, right at the design stage of a project.

1. What is a Sustainability Analysis ?

Sustainability Analysis is the identification and analysis of degree of presence or absence of the factors that are likely to impact, either positively or negatively on the prospects of sustained delivery of project benefits.

Annex 1 (Planning for Sustainability: A check list) presents a tool for checking the aspects of sustainability, at the time of designing of a project. The 'Check List' which include a member of analysis, such as economic and financial analysis; social analysis etc are important and should be undertaken to ensure incorporation of sustainability enhancement inputs during the preparation and the design stage of a project. These analysis include the following:

Relevancy refers to review of consistency ( or lack of it) between the objectives of the proposed project with national, sectoral, provincial and district priorities. Quite often, it is seen that when a project is taken up without due regard to various priorities set by the government, its ability to attract required support from various parties and its capacity to operate in a conducive environment, gets severely restricted. Therefore, 'Relevancy' test is expected to help analyzing these issues and assess the relationships between the proposed actions and their consistency with different priorities that have been set by the government.

Acceptability issues relate to the level and degree of acceptability of a project to the community, the local representatives, the executing agency etc. Weak acceptability by anyone or more of these parties has the risk of compromising long term sustainability of a project.

Economic/Financial viability refers to economic and financial profitability of project induced products and services. For these products to be of benefits, both to the producers as well as the economy the product cost must reflect real market costs and the product prices, the real market prices, and that the latter should be consistently higher than the former. In some cases, the project induced products and services may not reflect the market cost and nor the price which may mean that which the project will benefit the direct participants or the target population, the it will incur economic losses, at the national level.

It is now widely recognized that under the current situation of globalization and liberalization, any project induced products which cannot be produced and sold under market determined cost and prices and cannot earn profit under these conditions, are neither likely to be sustained nor would these be beneficial to the economy.

Environmental Sustainability relates to project induced environmental impacts - both positive and negative. If negative impacts are foreseen and no mitigational measures are planned, then ultimately the project may yield benefits at a reduced rate or worse still and depending on the extent of environmental costs, such negative impacts may in fact contribute to the net losses to the economy.

Implementation and Monitoring strategy refers to consideration of project management arrangements - e.g. is the implementation period realistic ? is there a well defined implementation plan with clearly defined functions and responsibilities and have necessary provisions been made thereof. Quite often weak management and inadequate monitoring provisions contribute to implementation problems which than weakens the project sustainability, eventually.

Post implementation operation and maintenance (O&M) refers to management support (either by the executing agency or the community or both) required after implementation of a project. Quite often projects tend to encounter sustainability problems due to weak or inadequate O&M support.

The sustainability analysis is to be followed by development of a sustainability strategy, so as to ensure that all sustainability enhancing elements are incorporated right at the design stage of a project.

3.2. What is a sustainability strategy ?

The sustainability strategy is a follow up activity of sustainability analysis and is expected to indicate the way various elements of sustainability are to be identified, assessed and incorporated into a project or a programme, right at the design stage. The strategy is expected to specify various complements / constraints to sustainability and make provisions for their incorporation / tackling during: (i) formulation/design; (ii) implementation, and (iii) operation and maintenance stages of a project.

Elements that compliment project sustainability have already been mentioned in the previous section. One needs to undertake necessary analytical research to define these variables and incorporate mitigating factors accordingly. Sometime it is also helpful to specify factors that constrain sustainability. Definition of constraining factors is also a useful way to determine a sustainability strategy.

See Annex 2, for a flow chart on, Project Preparation and Management and Principal Factors Affecting Project Sustainability. This document describes issues or dimensions that constrain sustainability and illustrates how these issues are also inter-linked. Annex 2 also highlights several External Events that affect project sustainability.

It is, therefore, important that the project planner becomes aware of these elements and develops a strategy for enhancing sustainability. For example, if a certain project envisages joint responsibility between the executing agency and the community to undertake post-implementation operation and maintenance, design stage to achieve this.

4.0. Sustainability Monitoring / Measurement of Sustainability

The next step in project sustainability is sustainability monitoring. The aspects of sustainability monitoring (some call it sustainability assessment) comes into play, as soon as a project goes into implementation. Many also term this exercise as process evaluation and lately, some call it, results-based monitoring and evaluation.

4.1. What is sustainability monitoring ?

Sustainability monitoring and development of a strategy for sustainability monitoring form the core of a project or a programme management. It is evident from the check list of sustainability analysis, that there are a whole variety of factors that can affect sustainability. Therefore, it is imperative that a well planned monitoring mechanism is put in place to assess the status of sustainability, at a regular interval. This will help tracking sustainability related problems early and provide necessary feedback for adjustments and enhance the prospects of sustainability. It is useful to base such monitoring on pre-determined indicators.

4.2. What are the Sustainability Monitoring Indicators ?

The Sustainability Monitoring indicators are signposts which reveal status of sustainability at a certain stage or point of time of a project. Since the issue of sustainability concerns a variety of factors and since these are multi-dimensional (e.g. 'economic', 'community', 'equity', 'institutional', 'logistics' and 'environment'), the monitoring indicators representing each of these dimensions needs to be identified separating and measured; community.

For example, to monitor the "economic" dimension, of a project sustainability, the indicator that has been used in Economic Rate of Ration (ERR) - that means against the economic dimension of sustainability, one to re-estimate the economic rate of return (ERR) of the against the appraisal value. If the current value is significantly lower than the appraisal value, then the project is likely to encounter sustainability difficulties.

Similarly, for assessing the elements of Community participation, a comparison between apprised targets (i.e. proportion of beneficiaries expected to participate in the project activity) and the level currently achieved (i.e., the proportion of beneficiaries who are actively participating) will reveal the status of achievement of this vital indicator of sustainability. Likewise, with application of a variety of indicators, the status of achievements of 'equity', 'institutional', 'logistics' and 'environmental' dimensions can also be ascertained. In this regard, it is equally important to recognize that a multi-disciplinary team may be needed to monitor the multi-dimensional aspect of sustainability.

A scoring system ranging from 1 (poor) to 3 (strong) against each indicator is being suggested in this document. The 'mean value' of individual scores is expected to give the overall measure of sustainability of a project. At a given time if the mean value is estimated to be less than 2, for a project it is likely that the project is facing serious sustainability problems. Anything above 2 is positive.

See Annex 3 for "Sustainability Monitoring Indicators and a Measurement Tool".

4.3. When and how Sustainability Monitoring to be carried out ?

Sustainability Monitoring or Sustainability Assessment which also forms the core of process evaluation is expected to commence right from the start of implementation of a project. However, not all dimensions of sustainability are expected to reveal themselves at an early stage of a project. For example, the 'economic'; and 'environmental' dimensions of sustainability is expected to reveal themselves at a more mature stage of a project, say after six months to a year of operation of it. But continuous attention to a variety of other sustainability issues such as the 'institutional', 'logistics' and 'community' etc. will help detecting deviations (if any) at an early stage and ensure introduction of corrective measures ahead of time. A simple survey tool, with semi-structured questionnaire is annexed here to assist assessment of sustainability of 'institutional', 'logistics', 'equity' and 'community' aspects of a project. See Annex 4 for Beneficiary Contact Monitoring Indicators: A Survey Tool

Timing

Sustainability monitoring is expected to be carried out on a bi-annual basis, with the help of a multi-disciplinary team. However, it is also expected that some aspects of sustainability monitoring such as 'institutional', 'logistics' and 'community' aspects should be undertaken during the more routine phase of project monitoring.


References

Bamberger, M. and Cheema, S. (1990). Case Studies of Project Sustainability, Implications for Policy and Operations from Asian Experience. The World Bank, Washington D.C. 1990.

Khan, M. Adil (1993). "Managing Project Sustainability: Key concepts and Issues in Development Administration", in Asia-Pacific Journal of Rural Development. CIRDAP. Dhaka. 1993 Spring issue.

Khan, M.Adil, Western J. and Hossain, þ (1992) Sustainability of Social Sector Projects: The Asian Experience. The World Bank. Washington D.C. 1992


Annex 3

Sustainability Monitoring Indicators and A Measurement Tool

Dimensions

Indicators

Sustainability Status

Score1

Appraisal target

Currently Achieved

Economic

Economic Rate of return of the project induced products

     

Community

  • Proportion of community participating

  • Local support

     

Equity

Pattern of distribution of project target benefits, by income and gender groupings of target populatin

     

Institutional

Manpower/resources for project management

     


Logistics / O&M
  • Condition of facilities provided
     
  • Condition of plants and equipment's
     
  • Maintenance Procedure
     
  • Operating budget
     
  • Operating manpower

- Government

- Community

  • Cost recovery (% targeted for recovery)
     

Environmental

Positive / Negative Impacts

     
     

Mean Value

 

Annex 2

PROJECT PREPARATION & MANAGEMENT AND PRINCIPAL FACTORS

AFFECTING PROJECT SUSTAINABILITY

Source : World Bank


Annex 4

Beneficiary Contact Monitoring (BCM) Indicators: A Survey Tool for Social Analysis

The following indicators, expressed in the form of a semi-structured questionnaire are known as Beneficiary Contact Monitoring Indicators. These indicators are useful for assessing the 'equity' and the 'community' dimensions of sustainability of a project.

These questions can be asked either by surveying a sample population selected randomly or through a focussed group discussions:

The Questions:


Annex 1

Planning for Sustainability:
A Check List

(It is suggested that at the time of formulation of a new project this check list is filled and annexed to the document project.)

Project:
Sector:
Ministry:
Department:

  1. Relevancy

Yes

No

Not Sure

  1. Is the Project consistent with National Priority

  • Is the Project consistent with Provincial / District Priority
     
  • Is the Project consistent with the Sectoral Priority
     
  1. Acceptability
     
  1. Has the community been consulted
     
  • Has the project identified/defined clearly the targeted beneficiaries
     
  • Does the community consider the project as beneficial
     
  • Will the project disadvantage/discriminate any particular group
     

2.4 Have the local elected representatives/

community leaders been consulted and agreement

obtained

     
  1. Have the local elected representatives / community leaders endorsed the project
     
  • Have the implementation agencies been consulted and agreement obtained
     

Have the capacity of the implementation agency / agencies been assessed

     
 

Yes

No

Not Sure

  • Has the implementation agency / agencies got

the necessary resources, logistics (trained

manpower, vehicles etc) to implement the

project.

     
  1. Economic / Financial Viability
     

3.1. Would the post-implementation project

operations be without subsidy 2

     

3.2. Are project benefits greater than costs

     

3.3. Will the project services be given free,

continuously3

     

3.4. Is there a cost-recovery plan4

     

3.5. Has the funding sources for the project been

identified

     

3.6. Will there be sufficient funds available to

implement the project

     

4. Environmental sustainability

     

4.1.Is the project environmentally Okay

     

5. Implementation and monitoring strategy

     

5.1. Is the implementation period realistic

     

5.2. Has the project developed an implementation

plan with clearly defined responsibilities

     

5.3. Has the project developed monitoring and

evaluation plan

     

5.4. Has the project developed all the success

indicators (financial, physical and impact

     

5.5. Has the project identified various levels of

review mechanisms for monitoring progress

     
 

Yes

No

Not Sure

  1. Post-Implementation Operation and Maintenance (O & M)
     
  • Has the project clearly determined the responsibility for post-implementation operation and maintenance

     


6.2. If post-implementation O&M is

to be with the government:

  • Whether budgetary provisions been made
  • whether institutional

arrangements been made

     

6.3. If post-implementation O&M is to be with the

community/beneficiaries

  • has the community/beneficiary agreement

been obtained

     

Return to MandE NEWS