Since March 15th, 1999 there have been
visits to this page.
Meeting Notes by Carol Puzone
In January, members of the "xc-eval" listserve received the following (abbreviated) email : "MDF (Management for Development Foundation, Ede, The Netherlands) and i2K (Information to Knowledge, Ottawa, Canada, a business trust of CARE Canada) announce that they are joining efforts in the field of computer-assisted tools for planning, monitoring, reporting and evaluation of development projects".
In mid-February, an email request went out to members of the xc-eval discussion group requesting further information on MER, or any other existing M&E software packages. Two responses came in. One from i2k/CARE-Canada who manages the MER system and the other from NEDWORC in the Netherlands who manages the PROMES system. To cut a long story short -- there seemed to be interest in the two software packages, and it seemed to make sense to have a group of M&E specialists viewing the demos together. Another request went out. Did anyone want to view demos together of the two software packages?
What resulted from all of this communication was a meeting where MER and PROMES were (personally) demonstrated here in New York to a small group of interested parties. Disclaimer. There probably are other, similar M&E packages available on the market. This meeting is only representative of the two companies who saw, and responded to, the request for information on the xc-eval list.
A. Where? Office of the United Nations Capital Development Fund, New York.
B. When? Wednesday 17 February 1999, 1-5 PM.
C. What (meeting content)? Dr. Jan Lavrijsen and his colleague Mr. Ton van der Krabben traveled from The Netherlands to provided a demonstration of the PROMES computer software system (Project management, monitoring and evaluation system). Gerard van der Berg of i2k/CARE-Canada traveled from Ottowa to provide a demonstration of the MER system (Monitoring, Evaluation and Reporting)-a design, monitoring and evaluation methodology that includes an optional software tool component. A total of 14 participants came to the meeting representing 4 NGOs, 3 UN related agencies and 1 University affiliate. During the software demonstration meeting, the group saw powerpoint presentations/demonstrations of PROMES and MER amidst general dialogue of M&E systems, and M&E automation needs in different organizations.
A question and answer period followed, although questions were asked throughout the course of both presentations. Many participants had to leave the meeting early, so there was really no final, "comparative" group discussion held, however similarities and differences in the capabilities of each system generally became apparent in the Q&A section. There is also a section on next steps for the New York Based M & E community.
d) Contact for Presenters? (in alphabetical order):
Gerard van der Burg of i2K/CARE-Canada
B.P./P.O. Box 9000, 6
Antares Drive
Ottawa, Ontario K1G 4X6
Phone: (613) 228-5695
Fax:
(613) 226-5777
Email: gerard@kcenter.com
Dr. Jan S.G. Lavrijsen, and Mr. Ton van der Krabben of Nedworc
P.O.
Box 816
3700 AV Zeist - The Netherlands
Phone-31-0-306932912
Fax-31-0-306932911
Email
for Dr. Lavrijsen: JANL@IAEHV.NL
Email
for Mr. van der Krabben: Krabben@worldonline.nl
e) Who Participated? The following organizations were represented at the meeting.
If you would like detailed names and contact numbers of individuals at the meeting, please contact Carol Puzone at ORBIS International as follows:
Carol A. Puzone, M.P.H.,
ORBIS International
330 W. 42nd
Street
Suite 1900
New York, NY 10036
(212) 244-2525 - Phone
(212)
244-2744 - Fax
Carol@ny.orbis.org -
Email
A brief summary of the presentations from 17 February are provided below. These summaries are derived partly from notes taken during the meeting, but also from documents provided by NEDWORC and i2K/CARE-Canada to participants. The systems are presented in the order in which they presented.
History.
In 1985 the inventors of the PROMES system developed a small system for the Sri Lankan Ministry of Plan Implementation to rapidly produce reports for integrated participatory rural development programmes. In 1992 a request followed from Rwanda to create a similar technical system, but it was never implemented due to civil unrest and subsequent slowing of NGO activity in the region. In 1994 a system was created for Monitoring and Evaluation in Benin, and the system that was developed in Benin was subsequently exported to Niger, Mali and Burkina by the Netherlands Development Organization. This module then evolved into PROMES, which was released in 1998.
Description.
PROMES is an acronym for 'project management, monitoring and evaluation system'. PROMES is a computer software package that facilitates the management, project-internal monitoring and evaluation of development processes, projects and programs in a decentralised setup. Development projects may either follow a more traditional sectoral approach, or be based on the principles of the participatory or process approach that can be multi-sectoral.
Promes is built on a broad interpretation of M&E. Promes does not divorce the monitoring and evaluation functions from normal project planning and management ones. Instead it integrates them as in the Project Cycle Management (PCM) method. Promes takes a holistic view of development and merges its important components, i.e. stakeholders, objectives and projects, into a whole. PROMES provides a combination of logframe, sampling, data collection and numerical analysis into one database package.
The main modules in PROMES are: Actors, Projects, Expenditure and Goals. They respectively deal with human resources, project management, financial administration and objectives/process monitoring and evaluation. PROMES supports and implements the logical framework approach. However, PROMES is not a panacea for all issues related to participatory M&E by an intermediate organization. PROMES is a tool to be used to make M&E easier. PROMES is not a final solution for all of your M&E problems.
System Requirements.
Windows-95 Platform. At a minimum, a pentium class computer with 32 Mb of memory, a hard disk with at least 30 Mb of free space, a floppy drive, a VGA colour screen, a keyboard and a mouse. Installation of Office 95/97 is recommended due to availability of data exports from PROMES to Microsoft Office.
Cost of software: Single user license for Non-NGO: $3,000. Availability of large quantity discounts is decided on a case-by-case basis. Single user license for local NGOs: $1,500 US. LAN licenses available at a small extra fee per workstation. Large discounts for educational institutes.
Training Costs: 10-day all-in course for max 15 participants localNGO: USD 7,000, Non-NGO: USD 10,000. Course includes a module to systematically prepare for system introduction into an organisation.
For Individuals: Course fee per participant is USD 900. Each participant is responsible for their own travel and accommodation costs and for bringing a laptop or hiring a PC/laptop.
Costs shown are for courses held centrally in the Netherlands. Other arrangements are also possible and the course venue can be anywhere in the world.
History. The MER system evolved from the M&E and information management needs of a multi-sector, multi-donor project being undertaken through CARE-Honduras in the early 1990s. By 1996, the MER system had been taken up by CARE-Honduras at an organizational level, and other CARE country representatives were seeing its potential for broader utility within the organization. With this interest from other country offices, and Gerard van der Burg's move back to CARE-Canada from CARE-Honduras, the MER Global Initiative began which included Beta Testers in 9 countries. The expanded team added other specifications they wanted the MER software to meet, participated in the development of training materials in basic M&E and software use, and tested MER in projects in their own countries. At the same time, i2K was established as a special unit (and later as an autonomous business trust) of CARE-Canada, focusing on M&E, knowledge management, and information technology for development organizations.
Description. The MER System is an integrated package of tools for the management of information associated with Monitoring, Evaluating and Reporting. It is comprised of manuals and training materials that promote systematic project design, log frames, M&E plans, data handling and analysis, and reporting systems. The accompanying MER Software is an integrated information management, analysis and reporting tool to help project managers automate the collection, analysis and reporting of data from routine project monitoring and periodic surveys for baseline and evaluations. The MER system offers flexibility in that it allows for systematization with or without the available automation software.
The volume and complexity of information resulting from development interventions make it necessary to follow a logical sequence of steps to facilitate effective and efficient management. The MER management software identifies four phases for information management: a) project definition b) information system design and planning c) data management and d) results reporting.
The MERlite system offers a scaled down version of MER which handles monitoring dta at the project level. MERlite was created to provide a quick and easy interface to data collection and monitoring for satellite, remote, and small projects or offices. MERlite can help project managers and monitoring teams to identify their monitoring needs, and accelerate the process of monitoring. I2K/CARE-Canada also offers a LiveLink Intranet to facilitate collaboration and sharing of knowledge within and across organizations. The PlanBud software (budgeting on the basis of Logical Framework) has been developed by MDF, and will offer an interface with MER in the future.
System Requirements. IBM compatible computer with pentium processor, 32 Mb of Ram, 20 Mb of free hard disk space, a 1.44 MB floppy drive, and Windows 95 operating system. A computer screen, a keyboard and a mouse. Microsoft office 95/97 is recommended because of export capabilities within MER.
Costs of software. Single user license for NGOs: $1,350 US + $450 upgrades, maintenance, and technical support.
Training Costs. Module I. (Systematic Program/Project Planning (no software). NGO: USD 4,000 for up to 15 participants; Non-NGO: USD 6,000 for up to 15 participants. Module II (MER Software Training). NGO: USD 7,000; Non-NGO. USD 10,000.
For Individuals: 1999 MER Workshop series. Costs per participant for Module I are US$500, and for Module II are US$750. Each participant is responsible for their own travel and accommodation costs. Contact I2K/CARE-Canada for dates and locations of the workshops.
Note: This section was extremely hard to make "stand-alone" for those people who did not attend the meeting. If you are unfamiliar with M&E and basic database design, much of the following exchange will look like sanskrit. Please contact NEDWORC and I2k/CARE-Canada for further information about either system. The questions asked by participants are broken down into 3 categories: a) Human Resources b) Software Capabilities and c) Training Issues.
· If there is an M&E person at headquarters, can the project manager incorporate this software into their existing activities, or is it necessary to have an M&E person in each field office? How much training is required to implement the software systems?
PROMES: Normally when introducing these systems in the field a trained user/specialist is needed. 2 weeks of basic training followed by an exchange of experience with PROMES management 1, 2 or 3 months after implementation is the recommended training period for this person.
MER: Staffing needs vary project to project. MER provides basic M&E training for project staff based on a templated , modulated strategy. A one week training course is offered before or during system implementation. Then, staff are left to work with the system unassisted for 4-6 weeks, followed by a training review based on a needs assessment.
· How are data forms input into the software systems?
General response: It depends on your organization. Many field staff still hand in written reports that are then counter-checked at a higher level (i.e., a supervisor). Reports are then submitted up to the next level - and the information is entered into the system. The answer to this question lies in proper project management .
· In additional to single projects, can each system distinguish and track "programs" with multiple projects, and various country projects as part of a larger "super project" (e.g., a vaccination campaign with multiple sites, etc)?
PROMES: Yes, as long as same application classifications are provided in order not to "compare apples and oranges". PROMES can handle dozens of single node objectives, partial and full logframe trees; logframes can be interlinked into user-defined 'virtual trees'. The system is based on systemic thinking.
MER: Version 2.5 coming in March 1999 will have "nested logframe" capability. This will enable both the tracking of individual projects/ativities, as well as the roll-up of any and all information from this level to a wider organizational level program. This will allow for organization-wide tracking and analysis.
· Do the software systems have project planning and management capabilities?
PROMES: Yes, based on the logframe design.
MER: Yes, based on the logframe design.
· Are the programs "demand driven" by the user (i.e., can you add/delete/rename program elements according to the needs of your organization/program/project?) Are definitions of impacts, purposes, outputs, outcomes pre-defined?
PROMES: You can extensively custom design the system to meet your needs, however some of the terminology used in the logrames is fixed.
MER: You can custom design the system, including language and definitions to meet your individual needs.
· Do the systems have a "lessons learned" component?
PROMES: The system could have this capability, but we tried it in the past and found it very difficult. However, this can be achieved within an organization if data structures and their meaning are unequivocally defined and harmonized. The system can have this capability across organizations, when all agree upon a minimum level of standardization. Promes also offers memo files for storing "lessons learned".
MER: Within MER there is a logbook that can be used as an "uploadable link". The logbook remembers relationships between keywords via information mapping.
· Can the systems track budget allocations and expenditures? Can budget line items be customized (e.g., to the UN system)?
PROMES: The system can track projected vs. actual expenditures, and you can cross reference the system to standard UN line items and make user defined, customized reports.
MER: You can create user defined budget reports. In the future, you will be able to import/export between PlanBud (budgeting on the basis of Logical Framework) and MER.
· Can you collect data via the internet?
PROMES: Many countries and regions within them are not on the Internet yet. Datasets can be exchanged thorough the Internet using Email or FTP.
MER: Yes, through LINK (Linking Information to Knowledge, a system for sharing project data and related information through a global intranet/extranet)
· Can the sitemaps (the systems) be used at multiple sites?
PROMES: Currently, PROMES is decentralised and site specific, but future versions will provide flexible user-defined consolidation routines.
MER: Yes, through the link capability.
· Can the system handle qualitative data?
PROMES: The system assigns values to qualitative data. Currently, it is essentially a database system. Future versions will include a system-wide keyword and word pattern search engine.
MER: Yes, MER can do qualitative analysis. The software includes a full coding system that enables the user to code text answers for analysis.
· How many countries are currently using the systems?
PROMES: 11 countries
MER: 12-15 countries
· When next is the next version of each system coming out?
PROMES: We are waiting for feedback on the existing program in order to develop and release a new version. Work is being done on the development of a multi-language version and the inclusion of significant enhancements already on the wishlist. The next major upgrade is scheduled for later this year.
MER: Version 2.5 will be released in March 1999
· Can you have various program sectors in the same program table?
PROMES: Yes, can have multi-level classification system.
MER: Yes, in version 2.5.
· How does the system track "% completed" of a project or program?
PROMES: Each project has a data entry sheet. Based on this the system automatically calculates how much of the project is completed. Many of the data fields to track "% completed" are calculated fields. For example, the budget balance is a calculated field based on actual expenditures vs. projected budget.
MER: Based on data that is input into the system. The user defines what "Output" should be achieved over a period of time, and then the user defines indicators (as many as you want), and gives the indicator a relative weight (0-100). You can then see how far the project has deviated from its original goal. Eventually, the PlanBud utility can be directly interfaced to track actual vs. projected budget expenditures.
· Can you differentiate between outputs and outcomes, activities and indicators?
PROMES: Yes, the system is based on the concept of the "logframe".
MER: Yes, the system is based on the concept of the "logframe".
· How does the system handle missing data fields on level (i.e., can you create mandatory fields? Will the system "flag" a manager if data is missing?
PROMES: Yes, the system has mandatory 'must have' and skip fields and will flag and guard them at data entry time.
MER: Yes, the system can be customized with mandatory fields and skip
patterns.
· Can both systems custom design reports?
PROMES: Yes, PROMES comes with standardized reports, plus the foxfire report writer which allows for an unlimited creation and customization of user-defined queries, reports, graphs, export datatables, textfiles, mail merges etc. Automatic launch of, for example Excel, to work with the export file/graph is possible.
MER: Yes, the system comes with standardized paste-in reports, and a powerful report writer.
· Can you upload existing data into the system (e.g. baseline survey data)?
PROMES: External data is usually keyed in to the system manually. Promes has an automated data import link with the Project Accounts Management package.
MER: Yes, you can do it through a mapping process, but this can sometimes be a time consuming process.
Is the system multi-lingual?
PROMES: Available in the next version
MER: Yes, it is currently offered in English, French and Spanish. There is also an add-in module available for character based languages.
· Can you specify which users are allowed to access certain areas of the system (e.g., you might not want data entry personnel using the data analysis area)
PROMES: Yes; powerful and extensive user-definable system access, menu
access and record locking facilities that can be set for individuals and groups
of users.
MER: Yes, the system provides full security features.
· Are the math functions for formulas easily understood?
PROMES: Yes, it is standard math
MER: Yes, it's standard math
· Are on-going trainings held in use of the systems?
PROMES: Yes, trainings are held on an as-need basis.
MER: Yes, trainings are held at the organization level on an as-needed basis. A regional schedule for individual training for 1999 is also available on request.
· Is there a risk of computerizing before mind-set is ready for this?
PROMES: Yes, this is a danger. PROMES is a tool, not a solution.
MER: Yes, that is why the MER system is designed to be implemented either with or without software
· How do the two software systems help monitoring of projects? Do they add discipline to monitoring?
PROMES: This system cannot impose discipline because this would be too strict a system. There is nothing in the system to impel people to monitor. It is people and organisational discipline that will ultimate determine the success or failure of a M&E system
MER: If you systematize, and then automate, the process can add discipline to monitoring. Automation alone won't do it.
A. If you are interested in learning more about either system, please contact the presenters directly.
B. If you have any questions about the meeting, please contact Carol Puzone @ ORBIS International
C. Another major output of the meeting was that New York Based NGOs, UN Agencies, are thinking of joining together to create a quarterly working group of M&E professionals in the New York area. This kind of an informal (note, informal), working group would help all of us to stay on the cutting edge of M&E in the development community, not to mention the potential partnerships that may arise as the result of new professional friendships. Please contact Carol Puzone at ORBIS International for further information.
[Many thanks to Carol Puzone for her work in preparing these very detailed minutes]
Since March 15th, 1999 there have been
visits to this page.
Return to:
