MONITORING AND EVALUATION PRACTICES IN MSF HOLLAND

Systematic planning, monitoring and evaluation of projects, programme and policies did not emerge in MSF Holland until the introduction of the logical framework in 1994. A manual for Project Planning was finalised in November 1995.

MSF adopted a field driven approach to programme management in April 1997, when operational management responsibilities were decentralised. Final responsibilities for policy and strategy remain with four senior programme managers in Amsterdam. Delegation of responsibilities and the need to keep some central overview resulted in an push for improved internal accountability. At the same time, it was recognised that MSF H was not making the most of opportunities for learning. Evaluations were ad hoc and lacked a systematic approach.

The need for a more systematic approach to planning, monitoring and evaluation was driven by internal agendas, meaning an independently defined approach to the establishment of the system. MSF recognised the danger of becoming too independent and loosing touch with outside world. To address this, a unit for monitoring and evaluation was established in April 1997 with a mandate to:

  1. actively participate in external discussions on evaluation of humanitarian programmes and translate academic and "external" concepts to the operational realities of MSF; and
  2. support field operations with tools and technical (methodological) advice in planning, monitoring and evaluations.

Some of the networks we actively participate in:

 

Monitoring in MSF Holland

The M&E Unit was given the task to construct a project monitoring system, including a conceptual framework, tools and reporting formats. A number of issues were identified:

The monitoring system highlighted something we have known for some time: the turbulence in the context of our work affects both the thoroughness of planning and the implementation of our planning. This is why context analysis and the project environment take a prominent place in our monitoring system. Monitoring interaction of project planning and environment is known as the "Observation Process" in MSF-Holland.

The project environment is monitored through the observation process. This relates to the assumptions, preconditions and overall objective sections of the logical framework. Developments need to be monitored in order to anticipate adjustments in project planning. Three broad domains are defined:

  1. The context of the project, consisting of the political, social, cultural and economic aspects of the environment of the project. It provides the project with opportunities and constraints in reaching its objectives. The context also includes human rights issues if they are a cause of public health problems.
  2. The needs of the target population. Information in this domain comes from MSF health surveillance, but also that of others, recognising that health statistics and information can be highly politicised.
  3. The response domain includes: a) the capacity of the target population, b) all actions undertaken by other actors in response to the identified needs, and c) MSF's organisational capacity (including our mandate, resources and policies).

B. Evaluation

MSF H adopted the OECD/DAC definition for evaluations, recognising the strategic importance of funding frameworks to the humanitarian community and the need to be coherent and compatible with other actors. The definition was originally formulated for work in relatively stable situations and for the evaluation of sustainable development programmes. The turbulence in the context of humanitarian work calls for some adjustments. A number of additional evaluation criteria, were endorsed during a symposium in 1996.

In MSF Holland, three types of evaluations are common:

1. Country Programme Evaluations

Over the last twelve months about ten country programme were evaluated, mainly by external consultants. Country Programme Evaluations helped MSF in identifying bottlenecks and other issues related to its reorganisation to decentralised programme management. This provided very useful insights in how MSF (management) systems are functioning. Later in 1999, we are planning to synthesise the main findings.

Although we the M&E unit will continue to support senior management with these types of evaluations, it is now felt that the information needs of country managers and project co-ordinators require more emphasis.

2. Project Management Cycle or Self Evaluations

Routine evaluations integrated into the project management cycle. Simpler and easier to manage processes relying on minimal external support. The principle is that those responsible for implementation are given the skills and tools required to evaluate their own work. The advantage is maximum control over the process and therefore maximising the potential for learning for those who need to implement the lessons learned.

3. Thematic Evaluations

Thematic evaluations can focus on any topic. In MSF H they often relate to operational / technical issues. Criteria for approving a theme for evaluation are:

MSF Holland will continue to play an active role in developing thinking about concepts and practise of monitoring and evaluation of humanitarian work in the foreseeable future. We are increasingly aware of the importance of the humanitarian context to the way we think about evaluation. We also need to develop ways of decentralising control over evaluation processes and to have them more in support of project management. The Monitoring and Evaluation Unit has recently become part of a larger Context and Evaluation Department, providing a basis for integration with context analysis at a more strategic level in MSF-Holland.

 

Peter Giesen

E-mail Peter_Giesen@amsterdam.msf.org

Context and Evaluation Department, MSF Holland

ANNEX 1 OVERVIEW OF EVALUATIONS WITHIN MSF HOLLAND

1997

1998:

  1. KIVU: evaluation of the emergency response in 97 by 1 external evaluator

1999:

Planned for 1999:

Most of these reports are public and available on request.

Return to Mande NEWS

There have been Visit counter visits to this page since July 15th 1999.