FORD FOUNDATION ACTION-RESEARCH PROGRAMME INTO ASSESSING THE IMPACT OF DEVELOPMENT FINANCE:
AN INVITATION TO PARTICIPATE.
James Copestake, Susan Johnson, Allister McGregor, Martin Greeley, John Gaventa, Jutta Blauert, Paul Mosley
September 1999.
This note provides background information about a recently launched programme of the Ford Foundation. The primary objective of the programme is to improve methodologies used within development finance organisations for assessing the impact of the financial services they provide in order to improve the quality of those services and their impact on poverty.
The programme is the initiative of the affinity group of Ford Foundation lead officers directly responsible for Ford’s support to development finance organisations around the world. Its aim is to promote action research into impact assessment of development finance within a network of collaborating agencies drawn from across South Asia, Sub-Saharan Africa, Eastern Europe, and Latin America. A similar programme is also operating in North America. Participation in the programme is not restricted to Ford’s existing partners. Rather the key criterion for participation is a genuine commitment to the above objective.
The first year of the programme (Stage 1) will be dedicated solely to identifying organisations willing to participate, and to developing with them action-research activities for implementation during Stage 2 of the programme. Responsibility for identifying and working with interested organisations to develop proposals for Stage 2 has been delegated to a team of development finance researchers based in the UK. Organisations interested in participating in the programme should contact them in the first instance at the addresses indicated below.
A workshop is planned for May 2000 to be attended by representatives of development finance organisations, Ford Foundation and researchers. The workshop will provide an opportunity to review the best and most innovative impact assessment methodologies available. It will also provide a forum for presentation and discussion of plans for action research to be carried out during Stage 2. Detailed plans for Stage 2 will consequently only be finalised after the workshop.
The remainder of this paper sets out some initial thoughts on the role of impact assessment within development finance organisations and how it might be improved. It contains a rather ‘mixed bag’ of responses to the idea of the programme when it was first proposed by Ford Fondation nearly a year ago. Its intention is to act as one point of departure for discussions between the research team and the staff of development finance organisations interested in participating in Stage 2 of the programme.
The research team does not believe that there is one universal model of how impact assessment should be done. Instead it believes that the best approach depends upon the specific goals, character and operating environment of a particular organisation. Thus the ability of the programme to accommodate and be moulded by the views and interests of different development finance organisations will be a key determinant of its success. At the same time, the team believes that cross-fertilisation of ideas about impact assessment between organisations can still do much to promote more reliable, cost-effective and inclusive impact assessment activities.
The next two sections of this paper set out some general thoughts on two key questions: assessment for whom, and of what? The remaining sections suggest four broad areas which new impact assessment methodologies need to address.
2. Assessment for whom?
Many providers of microfinance services start out in a position of financial dependence upon one or more sponsors. In return for their support, sponsors have a powerful influence over the direction and pace of providers’ development. However, as a provider grows, scope for autonomous decision making should open up. Most prominent in much recent debate has been the choice between (a) a rapid movement towards financial self-sufficiency and (b) a slower growth path, seeking perhaps to retain donor goodwill (and hence more protracted financial support) through a more explicit poverty focus. This is reflected in a broad range of decisions: over terms of lending and deposit taking, targeting, product diversification, business expansion and so on.
The pace at which decision making power shifts from sponsor to provider depends in part upon the "affective" nature of their relationship or partnership (Copestake, 1996). But it also depends upon the provider’s ability to diversify sources of funding, and upon the willingness of sponsors to accept and indeed encourage such autonomy. Sponsors' own bureaucratic needs, perceptions of provider's competence, accountability to other stakeholders, and market power all influence this. Perhaps most important of all is the nature of the accountability (through ‘exit’ as well as ‘voice’) of both sponsor and provider to actual and intended users or ‘clients’ of the financial services on offer. The question of ‘assessment for whom’ can thus be located within a triad of accountability/governance relationships between users, providers and sponsors.
The choice of impact assessment methodology can only be meaningfully addressed in the context of these wider relationships. For example, if there is an intention that ownership and strategic management of a providing agency should become more autonomous, then it follows that the provider’s own leadership should approve, if not increasingly control, the information systems through which performance is judged. Where, to give another example, there is a commitment to internal decentralisation and empowerment, then the influence of staff and clients on what information is collected (and by whom, as well as what for) becomes paramount.
3. Assessment of what?
Strategic decision-making within a development finance organisation broadly requires information about (a) financial performance of the organisation, (b) the relationship between services provided and the "health" of existing clients, their enterprises and their households, (c) the wider operating environment. More precise information requirements will vary between stakeholders according to their distinct priorities, particularly the importance attached to the goals of achieving (1) organisational survival and growth (2) broader social and economic impact, referred to here (for the sake of brevity) as poverty reduction. Thus a distinction can be drawn between six categories of information, as shown below.
|
Goal Information field |
|
2. Poverty reduction. |
|
A. Financial performance |
1A |
1B |
|
B. Client "health" |
1B |
2B |
|
C. Wider environment |
1C |
2C |
Row A.
The nature of the task in respect of Box 1A has become conceptually clearer and less controversial (Von Pischke, 1996) though practical problems with introduction and management of necessary information systems (and harmonisation with those of mainstream banking) remain. Effective financial monitoring is necessary for those concerned with poverty reduction too (Box 2A) so that impact through one activity (such as microfinance) can be compared with impact of allocating the same resources to other poverty reduction activities. The more controversial issue is what data needs to be collected over and above the financial performance of the development organisation and its sponsors themselves.
Row B
Client level data is needed for two reasons. Firstly, because the provider's and the client's health or "sustainability" are ultimately inderdependent. In the case of credit, for example, it has a powerful influence on overall loan portfolio quality. Secondly, the goodwill of sponsors may depend upon evidence on broader social and economic impact. In this case, it is necessary to understand not only client health, but also the extent to which that health is affected by (or attributable to) the services they receive from the provider. For example if an organisation is growing by capturing customers from competitors then that need not affect goal (1), but it reduces achievement in relation to goal (2).
General understanding of how such information can best be collected within what may for brevity be referred to as positivist, interpretivist and participatory traditions has improved considerably in recent years (eg. Hulme, 1997 and other AIMS working papers). There is also general recognition that the three traditions are in many ways complementary. But there is a continued need to refine both "methods" and "mix" further in order to reduce costs, thereby permitting an outward movement of the potential "impact possibility frontier". (Mosely and Hulme, 1996). In particular, the scope for more participatory impact monitoring and decision-making needs to be researched not only as an end in itself, but also as a means to reducing costs. This requires a more detailed and holistic understanding of elemental information exchange encounters - be they formal interviews or PLA activities.
Wider environment
In relation to Box 3A, the key issue is how agencies monitor opportunities and threats to existing and potential services that they provide – ranging from macroeconomic and political shocks, to changing micro-level investment opportunities. Effective social networks are widely recognised to be more important than formal data sources. But many larger finance agencies do of course employ market analysts. Thus there may be scope for improving formal planning and review processes appropriate to microfinance for use before and after launching a new product, sub-sector specific business development, opening a new branch, changing interest rates and so on.
In relation to Box 3B, it is necessary to assess the extent to which any one agency contributes to changes in the impact of the wider financial system that it is part of, rather than viewing it in isolation. Economic theory provides a partial framework for tracing such wider impacts through changes in market structure, linkages, local multiplier effects and so on. However, in the case of financial service contracts there is a particularly acute need for a "real market" analysis that incorporates impacts on the wider social and political institutions and norms (including gender and ethnic relations) within which financial contracts are embedded (Johnson, 1998).
A clearer conceptual framework for thinking about and monitoring such wider impacts is also necessary to clarify the role of finance within wider development strategies. This was evident, for example, in a recent workshop organised by Proshika in Bangladesh which explored how to influence the widening and deepening of financial markets in ways that assisted the poor. (Sharif and Wood, 1998)
4. Measurement of indirect and wider poverty impact
Few studies have attempted rigorously to assess the distributional impact of microfinance. The few that have (including Hulme and Mosley 1996, Copestake et.al, 1998) have done so in a strictly limited way, looking mainly at the direct impact on the livelihoods of the households of borrowers’ only. But there are other poverty impacts that matter a great deal:
There are also potential negative long-term effects (e.g. taking too large a loan induces unsustainable indebtedness induces asset sales reduces income); and both the positive and the negative effects may well differ as between men and women. The key point is that poverty assessment that only looks at borrower households merely scratches the surface of the issue. It is also important to separately assess the impact of the different components of financial services: lending, savings, insurance, and so on: conventional a priori wisdom suggests that savings may have higher utility for the very poor than borrowing, but this has never been properly tested.
Once satisfactory measures of impact have been obtained, they can be correlated with beneficiary income to ascertain whether impact increases with beneficiary income as suggested in a preliminary manner by Mosley and Hulme (1998) and whether the relationship between income and impact (the ‘impact frontier’) can be moved by appropriate changes in design such as reforms in loan collection procedures, incentives to repay, insurance arrangements and so on. Detailed assessments of poverty impact of this kind are not generally required by lending institutions for purposes of day-to-day management or obtaining sponsorship from commercial banks (see below). They are, however, required by sponsors with a ‘social’ remit such as aid donors and NGOs and there is a need to develop methodologies that can do this to the satisfaction of such sponsors at an acceptable cost.
5. Local information networks
A major reason why microfinance underperforms is not that good lending technologies do not exist – whether in lending or in impact assessment of such lending – but rather that best-practice technology is not known by many institutions, especially perhaps among the many NGOs set up with a local poverty alleviation remit. For this purpose stronger networks are needed to diffuse information on ‘what works locally’ among institutions and supply training facilities as required, of the type supplied by K-REP in Kenya and PRODEM – BancoSol’s research and training arm – in Bolivia. The case for setting up such a network in the Eastern Africa region has been made elsewhere (Mosley 1998a). The need for such networks will vary from place to place, being perhaps less in Asia and Latin America than elsewhere on account of networks set up by CASHPOR (Credit and Savings for the Hardcore Poor) and ACCION respectively. It is important for local networks to circulate ideas from around the world, rather than just locally generated ones, recalling the valuable influence of BRI Indonesia savings technology on BancoSol Bolivia and the inspiration of Grameen Bank in many places outside Bangladesh such as K-REP and BancoSol itself. But at the same time there is a need for further thought on how such relationships can be made to work effectively between organisations who are to varying degrees financially and technically dependent on distinct global microfinance franchises.
6. Linkages between Microfinance and the ‘Commercial’ Banking Sector:
Many microfinance institutions are highly profitable – over and above the often-cited cases of BRI Indonesia and BancoSol -, and a major question on the frontier of microfinance research is why, given this fact, microfinance institutions have attracted so little inward investment from commerical banks and other for-profit institutional investors. Microfinance institutions, if able to move into the development finance mainstream in this way, would both be able to expand lending to the poor more rapidly and more sustainably under the pressure of the market. Montagnon (1998) has investigated some of the reasons for the lack of linkage. One of his main findings is that microfinance institutions simply do not produce the information required to convince commercial investors of their bankability; in other words, the right impact indicators and business plan formats to achieve credibility with the market; but what indicators they do need in order to be convinced is left as an open question for lack of data. The action research could take on the task of interviewing commercial financial institutions concerning the impact data which they require, and then find out from microfinance institutions what the cost of obtaining those data would be. It could then, in the case of those rather rare institutions which had succeeded in obtaining inward investment from for-profit financial institutions, examine what the impact of that involvement was. Appropriate regions for this enquiry would be, once again, Africa, eastern Europe and India; although in India the context is different because the so-called ‘commercial’ bank sector is, in spite of liberalisation, still heavily state-dominated. Government is, however, increasingly committed to a creative partnership with NGOs and other agencies in developing microfinance (cf. the 24/10/98 Lucknow workshop on Kickstarting Microfinance) and from the point of view of influencing policy, India would appear to be a vital focus for this research.
7. Participatory processes, empowerment and inclusion.
Development finance organisations can contribute toward poverty reduction not only by improving access to financial services, but also by doing so in ways that are inclusive and empowering of poor people. This in turn raises questions about processes of strategic decision making and institutional development within development finance organisations themselves. With respect to impact assessment, it is necessary to analyse what determines the extent to which they are willing and able to privilege primary stakeholder or client perspectives.
There is much evidence to suggest that many poor people are currently being excluded from participation in microfinance schemes; very often the reason for this is associated with the changing organional culture, including the aggressive role of donors in promoting particular priorities (sustainability) and particular approaches to assessment. This changing institutional culture – once described as "from compassion to capitalism" – is most visible in the recruitment, training, and performance assessment of staff through a pronounced shift to a concern with financial sustainability rather than poverty reduction per se. Of course, there are also reasons why potential borrowers opt to self-select out of schemes. Understanding the relative importance and relationship between these supply-led and demand-led reasons for exclusion is a key issue.
In contrast, it is increasingly recognised that participatory approaches to service needs identification, service delivery and constant improvement in institutional responses is a prerequisite for effective and sustained poverty reduction, sustainable livelihoods and quality of life more broadly understood. Such participation is necessary also as tools for local communities to help monitor the effectiveness of development finance organisations on individual and local economies. Improved assessment methods that empower poor people and help to make DFIs more responsive internally can address these concerns.
It is consistent with this view that research into impact assessment itself is only likely to be successful if carried out in an open, participatory and flexible way (Edgcomb & Garber, 1998). Outputs from any action-research process will NOT be a single blue-print methodology, but a contribution to the continuously developing set of tools and protocols. This approach is necessary in part to move away from the distinction between (ex post) assessment and (ex ante) appraisal that is a relic of the old project cycle (Picciotto & Weaving, 1994). The term "impact monitoring" has become fashionable precisely because it is indicative of a move towards more seamless information systems in which impact assessment feeds more clearly and continuously into different levels of client, provider and sponsor decision making.
References
Copestake, J G (1996) NGO-Donor collaboration and the new policy agenda - the case of subsidised credit Public Administration and Development 16:855-66.
Copestake, J, S Bhalotra, M Godwin, H Grundel, S Johnson & D Musona (1998) Impact Assessment of the PULSE microfinance programme in Lusaka, Zambia. Centre For Development Studies, University of Bath, Working Paper. Available at www.bath.ac.uk/centres/cds
Craig, D. and Porter, D., (1997). "Framing participation: Development projects, professionals, and organisations", Development in Practice, Vol. 7, No. 3. pp 229-236
DSA (Development Studies Association), 1998, unpublished symposium on The Tyranny of Participation.
Elaine Edgcomb and Carter Garber (1998) Practioner-Led Impact Assessment: A Test in Honduras. USAID Office of microenterprise initiatives. AIMS Brief No. 17.
Evans, G.E., Adams, A.M., Mohammed, R., Norris, A.H. (1999). "Demystifying Nonparticipation in Microcredit: A Population-Based Analysis, World Development, Vol. 27, No. 2, pp. 419-430
Fowler, A (1997) Striking a balance: a guide to enhancing the effectiveness of non-government organisations in international development. London: Earthscan
Gaile, Gary and Jennifer Foster, 1996, Review of methodological approaches to the study of the impact of microenterprise credit programmes, Washington DC: USAID, AIMS programme.
Greeley, M., (1997). "Poverty and Well-Being: Problems for Poverty Reduction and Role of Credit", Who needs Credit? Poverty and Finance in Bangladesh, Geoffrey D Wood, Iffath Sharif (eds.). The University Press, Bangladesh.
Hassan Zaman (1998). "Who benefits and to what extent? Evaluating the targeting, poverty alleviation and empowerment objectives of BRAC’s micro-credit program. DPhil thesis, Dept. of Economics, University of Sussex
Hossain, Mahabub, 1984. Credit for the rural poor; the experience of the Grameen Bank of Bangladesh. Dhaka: Bangladesh Institute of Development Studies.
Hulme, D & P Mosley (1996) Poverty and finance. Vol.1 (cha? on management). London: Routledge.
Hulme, D (1997) Impact assessment methodologies for microfinance: a review. Prepared for the virtual meeting of the CGAP working group on impact assessment methodologies.
Hulme, David, 1997, Impact assessment methodologies for microfinance: a review, Washington DC: USAID, AIMS programme.
Hulme, David, and Paul Mosley, 1996. Finance against poverty. 2 vols. London: Routledge.
Johnson, S (1998) Programme impact assessment in micro-finance: the need for an analysis of real markets. IDS Bulletin, 29(4), 21-31, October.
Johnson, S., (1998). "Policy Arena. Microfinance North and South: Contrasting Current Debates", Journal of International Development, Vol. 10, No. 6, pp 799-809
Khandker, Shahidur, and Mark Pitt, 1995. Grameen Bank: what do we know? Unpublished conference paper (now a World Bank occasional paper).
Mayoux., L. "Impact Assessment and Women's empowerment in micro-finance programmes: issues for a participatory action and learning approach". Background paper for CGAP meeting, April 1997
Microenterprise Summit, 1997, Statement of Objectives, Washington DC.
Montagnon, Peter, 1998. Credit where credit is due; bringing microfinance into the mainstream. London: Centre for the Study of Financial Innovation.
Mosley, Paul, 1997a, The use of control groups in impact assessment for microfinance, Geneva: ILO.
Mosley, Paul, 1997b, A simple guide to impact assessment for development economists, London: DFID.
Muazzam Husain, A.M., (ed.) (1998). "Poverty Alleviation and Empowerment. The Second Impact Assessment Study of BRAC’s Rural Development Programme". BRAC, Dhaka, Bangladesh
Noponen, H. "Participatory Monitoring and Evaluation - A prototype Internal Learning System for Livelihood and Micro-Credit Programs" (forthcoming) Community Development Journal: An International Forum, Oxford University Press
Pearce, John (1996) "Measuring social wealth. A study of social audit practice for community and co-operative enterprises". New Economics Foundation, London
Piccioto, R & R Weaving (1994) A new project cycle for the World Bank? Finance and development, December, pp.42-44 [copy attached to this outline].
Shams Mustafa, Ishrat Ara, Dilruba Banu, Altaf Hossain, Azmal Kabir, Mohammad Mohsin, Abu Yusuf, Sarwar Jahan (1996). "Beacon of Hope. An impact assessment study of BRAC’s Rural Development Programme".
Sharif, I & G Wood (1998) Towards new frontiers of finance: summary and findings. Paper presented at the 2nd international workshop on poverty and finance: emerging institutional issues in microfinance, Dhaka, Bangladesh, 11 July 1998.
Sorenson, Christian (1998) "New uses of PRA in connection with appraisal of micro-enterprises in Eritrea". ACORD, Asmara/London,
USAID (1998) Concept Paper: ownership and governance of microfinance institutions. Microfinance Best Practices Publication available at http://www.mip.org.
Von Pischke, J D (1996) Measuring the trade-off between outreach and sustainability of microenterprise lenders. Journal of International Development