The last two papers were reproduced by Elizabeth – Wade Brown of CAFOD, credits are on the papers
Wateraid 22nd January 1999. Organised by Jean MacGrory
New Economics Foundation are doing a scopping study for DFID, further information from Peter Raynard on neweconomics@gn.apc.org
Annual Thematic reviews by CAFOD are available. Contact Elizabeth Wade- Brown on ewade-br@cafod.org.uk
Christian Aid are producing a synthesis of evaluation reports. Contact Aiden Timlin on atimlin@christian-aid.org
Christopher Riley and Colin Kirk gave a presentation of the DFID evaluation departments recent consultation with CSOs, (mainly those in the UK).
There are planning follow up workshops. A copy of the overheads was given out by Colin – further copies should be available from him onc-kirk@dfid.gtnet.gov.uk
Andy Clayton of INTRAC gave a presentation on the Participatory Action Learning Study (PALS) that was undertaken for DFID.
Their recommendations included the need for more flexible project cycle management, and better use of the logical framework, which was felt to be more of an internal bureaucratic tool.
There were also broader recommendations for how DFID recruits and trains its own staff. Further details are available from a DFID publication on PALS.
By Underground
to Stockwell (Northern and Victoria Lines)
to Brixton (Victoria Line)
By Bus
Routes 2, 322 and 345 (formerly 45A)
Stop between CAFOD and Stanford Road (going North)
Stop at Stockwell Green (just before Combermere Road, going South)
By Rail
Via Vauxhall and Waterloo, then underground, or direct to Brixton (British Rail)
Joint Dialogue Process between South Partners and Misereor, EZE and Bread for the World
Final Workshop to Phase I of the PME Dialogue Process: The PME Inventory
Bleyerheide (NL), 19.11 – 28.11.1995
The PME Dialogue Process is a joint initiative of the German Churches' aid agencies: Misereor, Bread for the World and EZE. For the first phase of this Dialogue Process, these agencies invited about 80 of their partner organizations in the South to share their experience of PME. This PME inventory was undertaken by local experts as Regional Facilitators of the Dialogue Process in 8 regions throughout the 3 continents of the south, who visited the partners to jointly understand, document and analyse their PME practice.
The result of this PME inventory, the Phase I of the Dialogue Process, is a very interesting and comprehensive compilation of PME approaches, systems, methods and tools, which are being used for different purposes and within a wide scope of development work. At the same time, it gave a very enlightening insight into the evolution, logic and context of PME in and for development work.
The conclusion of Phase I of the PME Dialogue Process was done in the form of an international workshop of the Regional Facilitators and representatives of the agencies involved. The following document attempts to summarize some key findings of the PME Dialogue Process up to now as a general message to the agencies and partners involved. More concrete information and facts about the PME practice of the partners of Phase I is to be found in the synthesis documents and the partner reports from the different regions which will be made available in the form of a publication during the next year.
The partner organizations need to have a clear and complete comprehension of the situation and context of the people with whom they work in order to develop their broader vision of social transformation and change. They could derive their organizational mission, goals and nature of work from this vision.
The next step is to clearly define their objectives which can be object of PME. In this respect, the use of PME could help to transform a usually rather ambitious VMG into a realistic working perspective.
A similar process of strategic orientation ought to be brought forward in the agencies. This process needs to involve the context and situation of the agencies' partners in the South as well as the context and framework conditions for development policy in the North.
PME is not an end in itself, but a means to achieve the goals of development as set by the people, the partner organizations and the agencies. Therefore it should be used in function of the purpose and impact of development work, and the result of its use is more important than the method used.
pme update No. 3 December 1995
Editor: PME-Coordination Compiled by
c/o MISEREOR Klaus Langefeld
Mozartstr. 9 Sophienstr. 58,
D-52064 Aachen D-60487 Frankfurt/Main,
Germany Tel Fax –49-69-704827
Reproduced by CAFOD
It is possible that partners are making use of sophisticated PME techniques without having effective projects and development impact. On the other hand, partners could be doing good development work inspite of using PME methods which are not sophisticated/ This shows that there is no automatic and mechanical relationship between the sophistication of PME and the impact of development work. Thus the need is for an improved and more impact-oriented PME which does not necessarily mean more PME.
PME is not an issue only of the partner organization, but it is also an issue of the grass root groups. These groups have their own logic and implicit methodology of doing things and organizing their daily life. The difference between this "People's PME" and the partners' PME has to be taken into account if a joint development effort is to be undertaken. To make this possible, there should be mutual participation of the people and the partners in the key moments of each other's PME process which leads to a joint ownership off the project.
In general terms, the ownership of a project and of its PME depends on the nature and the level of the activities being implemented. Some projects like a health service tend to be rather in the ownership of the partner organization that is offering this service, and people can achieve certain levels of participation through a participatory PME process.
Other projects that evolve from the people's own initiative imply an ownership of the project with the people. Nevertheless it has to be borne in mind that the ownership of such projects ought to remain with the people, and that it should not be transferred to the partner organizations because of its more elaborate levels of PME. Other key factors comprise of an appropriate management of the power relationship, the adaptation to the cultural context and respecting and maintaining the peoples groups' autonomy.
Yet another important element with regards to participation is the internal democracy in the partner and the peoples organizations. This requires preparation and training as well as communication and socialization of information on both sides. Further there is also a need to enhance capacities of adaptation of methods by the partners and by the people. In this way, participation could become an institutional attitude of the partner and a continuous practice amongst its staff. Thus the gap between the intention and the reality in the process of participation can be reduced.
PME is an inherent process to all development work which facilitates learning from practice and experience. The two-fold functions of PME are to enhance managing and learning processes in the development work of an organization. This has a dialectic relationship with the internal logic and rhythm (the "pace" )of the organization.
Under the "managerial loop", things have to be organized and streamlined under a given structure and framework, and PME has to go along the pace of the organization, as defined in terms of specific objectives, available resources, etc.
Under the "learning loop", PME facilitates a learning process which helps to structure the development work and to shape the pace of the organization, thereby contributing to specify framework, rhythm, objectives, etc. of their activities.
The partnership between agencies and counterpart organizations has to be developed on the basis of a clear policy and a realistic approach on both sides. This might help to demystify the relationship between them.
There is a need for a comprehensive understanding of each other on the part of both agencies and partner organizations which goes far beyond PME. Clarity about things in common and differences is essential. For a mutually satisfying interaction, the agencies and the partners need to have their own process of organizational development (OD).
On the other hand, since the basis of this relationship is financial, it is a dominant factor in conditioning PME on both sides. Nevertheless, the ways of implementing financial support ought to be revised in order to establish more appropriate funding procedures. Specifically the gap between expecting from the partners a strategic orientation and the development of long-term vision, mission and goals but only giving short-term funding, mostly limited to projects, is to be reduced. This situation is a greater cause of concern for small and medium-sized organizations. It has often been found that partners evolve a something that could be described as a dual PME system; one for satisfying the agencies' needs with regards to project preparation (proposals), reporting and project completion, and another one for its development work at the grass root level. All the same, the partners get very little feedback from the agencies to these efforts to meet their requirements.
It needs to be further explored what kind of influence the agencies have vis-à-vis the partners' PME and what degree of their involvement would be appropriate.
There is a diversity of PME methods and tools prevalent. At the same time PME is just one of several working methods in an organization. As earlier mentioned use of PME tools does not automatically lead to the effectiveness of the programme. Its [sic] important to know how to organize information on and access to PME-methods. Also a critical attitude and understanding is required vis-à-vis the conception and use of PME-tools. Methods need to be simplified and flexibility needs to be incorporated in the use of the tools.
One question that comes back is what are the characteristics of good PME methods and tools, or what characterizes a good PME system? There is no one answer to this question. A PME method, tool or system becomes "good" if used in an appropriate manner taking into account the context of work, vision, mission goal of the organization and the capabilities of the people and staff members involved.
There is a need for the information of already existing PME tools. Access to such information and the tools used in practice need to be facilitated, and while adapting these tools, the local context should be taken into consideration.
Planning, Monitoring and Evaluation together comprise the PME system. Some basic issues in the elements of Planning, Monitoring and Evaluation are discussed below.
Many questions have been raised in the area of planning that need to be explored in depth. Some of these are whether detailed planning should be done at the proposal stage? Does planning help to perform well the development work or is it undertaken for the sake of writing project proposals to satisfy the agencies? It is also realised that planing does not mean merely operationalizing the activities, but it involves a wider perspective of the context, the capabilities of the actors concerned and the vision-mission-goal of the organization. The planning process also has to explore various alternatives or options to address the developmental issues, and arrive at a strategic decision towards its feasibility.
Prior to specific planning, a situational analysis namely identification of problems, needs and potentials needs ot be undertaken. In case of an ongoing project, planning of further activities usually starts with a review of the earlier years' work done. On one hand, this diagnosis of the situation is a phase of analysis and reflection and serves to be a link between the process of evaluation and planning. On the other hand, the diagnostic approach needs to be practised as an organizational attitude on an ongoing basis underlying the project cycle of planning, implementation, monitoring and evaluation.
It has been observed often there is an incompatibility between the planning of financial resources and planning of the activities. Many organizations do not have the capabilities to prepare appropriate budgets which incorporate the real needs and the timeliness of the required resources.
As mentioned earlier, under chapter 6, there is more emphasis given to short-term planning of projects than to long-term and strategic planning. However, strategic planning and development work is more than merely "adding projects".
Monitoring is found to be the weakest element in the PME process. Although many partners started to collect a lot of information through formats (eg daily work performance sheets), they do not know how to use the information gathered through monitoring. There is confusion for what purpose much of this information is being collected: Is it for partners' purposes, for the agencies or for the beneficiaries?
Monitoring needs to maintain an appropriate balance between self-reflection, feedback and control. A daily work performance sheet could be a useful instrument for self-discipline but, if imposed, it can become a means for external control.
Often the information gathered through monitoring is not being processed and made use of and sometimes real "data cemeteries" get created. This is especially true for larger organizations and organizations with more complex areas of work.
Indicators are considered to be an important element of monitoring, Nevertheless, indicators cannot be defined without taking into account the social, cultural, economic and organizational context of the partner and its work. Therefore the idea of having standardized indicators or indicator banks is to be rejected.
Although indicators play an important role for monitoring, they should not be overestimated. The mere fact of having good indicators and formats does not necessarily lead to an appropriate monitoring.
Monitoring of financial resources should become part of the overall monitoring practice.
Most evaluations focus on the results rather than on effects and impact. There are a lot of reasons why impact is not being addressed. First of all, a clear concept and definition of "impact" is missing. Consequently, it is quite difficult to work out specific impact indicators. Usually there is no linear and tangible relationship between project output and impact. It is much more difficult to get information on impact than on project output. It has been seen that more often process review is not being undertaken, which is an essential component in impact evaluation.
While self-evaluation is seen to be a reflective and learning process, the limitations of self-evaluations also need to be kept in mind. Self-evaluations should not become a means of mere self-reconfirmation but rather a means of critical self-reflection. In order to incorporate a wider perspective, self-evaluations ought to include the participation of external resource persons.
The partners need to be aware of the advantages of external evaluations as a means of critical feedback from outside the organisation and therefore take the initiative by themselves. Hence external evaluations makes more sense if they are no longer donor-driven but self-initiated. At the same time, the role of the agencies in external evaluations needs to be redefined and reworked.
Finally, evaluations should also point out the role and influence of the agencies in the projects which they have funded.
The Regional Facilitators of the PME Dialogue Process:
The Agencies' Representatives:
20th – 24th October, 1997
Report compiled by CORAT/Misereor, January 1998
Report Extracts, CAFOD
From the 20th to 24th October 1997 a workshop on Planning Monitoring and Evaluation and Organisational Development was held in Norton, Zimbabwe. This workshop was initiated by the German church agencies (Misereor, EZE and Bread for the World – MIBREZ) and convened by CORAT - Kenya, IRED – Zimbabwe, CDRA - South Africa and Gilbert Namwonja - Kenya in order to bring together partners of Southern and Eastern Africa to share experiences and ideas on planning, monitoring, evaluation (PME) and organisational development (OD), but also on the relationships between the different stakeholders in development.
At the workshop in Norton implementing NGOs, support organisations, consultants from the South and Northern "donor" agencies were present which allowed for rich discussions and a mutual learning process. This paper is [contains extracts from] a summary of the discussions and outcomes.
The origins of Misereor's involvement in the question of PME date back to the 1980's when the German church agencies felt the need to make a contribution to the ongoing discussions on the issue of evaluations in development aid. The beginning of this involvement was a critical reflection on the concept of evaluation and on the experiences of the church agencies in this field. These reflections resulted in a publication "Evaluation in the Church related development work" in March 1993 to guide the staff of the church agencies and the partners in the South on this question.
In the beginning of the '90s – being confronted with heated discussions on methods like ZOPP, PRA and others – two new insights were gained by the MIBREZ group in Germany: a) evaluation cannot be discussed in isolation but has to be seen in relation to planning and monitoring and b) in order to progress in the methodological discussion the practical experiences of the Southern partners have to be taken into account.
At this point and following these insights the MIBREZ group started the "PME Dialogue Process". A first step in this process was an analysis of the PME "systems" or methods used by a number of selected partners in Africa, Asia and Latin America. The MIBREZ group and eight regional collaborators in Bleyerheide (The Netherlands) discussed the results of this study in November 1995 where some important findings were summarised as the "Bleyerheide Declaration".
Some salient points of this declaration are:
a) That PME does help transform the vision, the mission and the goals of an organisation into realistic work objectives.
b) That PME focuses on purpose and impact of development. It is not an end in itself
c) That there has to be participation at all levels, especially at the partner organisations' and the grassroots level.
d) That PME facilitates a learning process as organisations are challenged and pulled to act. Impact assessment is key and this has to be built into the process of learning.
e) That there is a whole menu of methods in use. The method chosen should be simple and flexible.
f) Those monitoring systems are often weak. A lot of information gathered continues to pile up in "data cemeteries".
In order to bring back these and other outcomes of the PME-Dialogue Process to a larger forum, it was decided to organise workshops in Asia, Africa and Latin America, where PME and the upcoming question of OD could be discussed with partners and where progress in the methodological field could be achieved. In Africa two workshops were initiated by the MIBREZ group and organised by local partners; one in Ivory Coast in April 1997 for West and Central Africa and one in Zimbabwe for Southern and Eastern Africa. As it became clear during the PME Dialogue Process, the donor agencies and the implementing agencies are not the only stakeholders in development and in the methodological discussions. Therefore support and consulting organisations were invited to contribute to this discussion in Norton.
…..
As mentioned above it seemed to be necessary at the workshop to make an in-depth analysis of the different stakeholders and actors in development work, to look at their roles as they are right now and as they should be. In this chapter this analysis is presented: ie the stakeholders that were present at the workshop (the support organisations, the donors, the consultants and the implementing organisations) reflected together on their particular challenge and role in development aid.
a) NGOSOs must not use client organisations to justify their existence; never raise funds to provide services to organisations and then look around for the organisation to provide the services to.
b) NGOSOs should be honest with donors and organisations about what OD and PME can achieve and how long it may take. Unrealistic promises should not be made.
c) NGOSOs must assist organisations to set up accounting and reporting procedures, which they can use, and which give donors the information they need. Otherwise donors will impose tools.
d) The tools used by NGOSOs must make sense and be appropriate to the organisation they work with and not primarily to the donors.
e) Unwilling or unable organisations should be linked with organisations that have been through PME or OD processes so that they can share experiences and (positive) stories.
f) NGOSOs must be self-reflective and have OD and PME processes within their own organisations to explore their own preconceptions and preconceived ideas – this enables them to be honest and clear about their own position when they work with other organisations.
g) NGOSOs should encourage the use of tools that allow individual self-reflections and group/organisational reflection in order to be able to address a wide range of problems.
h) NGOSOs must actively promote that donors finance budgets for PME and OD to the organisation whilst ensuring that the organisation remains in control of the process. One way of doing this is for NGOSOs to encourage their clients to see PME and OD costs as essential components in their development budget.
Because of the wide scope of expectations that grassroots groups, NGOs, service organisations and donors have, the role of the consultant is very challenging. While some clients ask for expert advice others want the consultant to facilitate an open process. Some expect the consultant to be a provocateur and to shake up an organisation while with other clients the consultant is not allowed to touch the given power structures.
The different roles which the consultants experience themselves are technical adviser, deliverer of technical skills, trainer of process skills, facilitator of a limited process, questioner who confronts an organisation with fundamental questions, counsellor, etc.
The consultants among the participants see their role and their challenges as outlined in the following points:
a) Consultants should pose new and difficult questions to an organisation.
b) They should be a mirror for an organisation and thereby initiate self-reflection. Their role is to bring a fresh eye and a fresh heart to help a group/organisation see anew.
c) They should reveal paradoxes and tensions within organisations.
d) One major challenge is to be loyal to the development process of the people one is working with and not to the expectation of the paymaster by just doing the job as described in a contract.
e) Consultants should retain independence and integrity.
f) In their work the methodological aspect is important. To improve on methods and adapt them to the circumstances is a constant challenge.
g) Consultants should be concerned with questions concerning the sustainability of the client organisation.
The donors see themselves trapped in the crazy development circus. On the one side there is the public in the western societies that question aid agencies about the use of funds. Their idea of development is to feed the hungry and to heal the sick. Everything spent on development structures, animation programmes, consultancy, quality management, etc. is questioned very critically. On the other side the partners ask for support in organisational development, PME and the donors themselves want to move away from charitable work to development work.
One the one hand there is a need to spend more time n dialogue as mutual trust and understanding is still lacking. On the other hand there is an ever present time pressure on the development agency staff to manage the piles of paper that come in [in] order to comply with the control requirements.
the donors feel very much trapped in the diverse expectations of their donors (government and the public), partner NGOs, grassroots groups, etc. In this dilemma they see their role and their challenges as follows:
a) Donors should create an environment which allows for dialogue and mutual trust. They should not have hidden agendas.
b) In order to build up a relationship of mutual trust an investment of time has to be made. Donors need to consider this in their time budget.
c) Donors should be aware of the time that development processes take and shouldn't have unrealistic expectations.
d) The achievement of plans and objectives should be measured in terms of impact, not in terms of outcomes. Outcomes can be of short-term nature, and considerable attention should be paid to impact, which is of long term benefit.
e) Donors should produce information material on how they see their partners and on what they expect from them. The information deficit should be overcome.
f) Donors should encourage and support the exchange of experiences in organisational development between partners. They should refer partners to organisations who faced similar challenges and went through similar processes.
The service organisations see themselves as catalysts in the development process. As catalysts, they are expected to stimulate development and provide such services which will create a conducive environment for development.
In addition, the service organisation is expected to play the advocacy role, brokering with the donors, networking with the government structures and other NGOs who are involved. In view of the above, the service organisations face a number of challenges:
a) They should always be flexible and critical where necessary in challenging the methods and tools to ensure that they are appropriate
b) They need to create a climate of open dialogue and trust so that people can feel free to accept of reject the services they offer.
c) They should encourage round table consultations where some partner agencies are brought together for problem solving and capacity building.
d) They should be courageous to try out better ways of working.
e) They should promote networking in order to enhance impact and effectiveness of their programmes.
Since 24th March 1999 there have been
visits to this page.
Return to:
