Impact Assessment

Review of Impact Assessments of Selected Enterprise Development Projects, Sebstad, et al, DAI, July 2004

    The study was conducted under USAID's Accelerated Microenterprise Advancement Project (AMAP-BDS). The report presents findings from a review of evaluations of enterprise development projects with a BDS component. While the studies cover a wide range of programmatic issues, the review focused primarily on impact and approaches used to study it. It documents evaluation objectives, key issues addressed, methodologies, findings, and lessons for future impact studies.

    The studies cover 9 programs in Africa, 6 in Asia, 3 in the Middle East and North Africa, 6 in Latin America and the Caribbean, and 3 in transitional countries that were funded/ conducted primarily by USAID, World Bank/IFC, IDB/MIF, and DFID. The evaluations covered private sector programs focused on agriculture/ agribusiness, trade and investment promotion; business development services, market linkages, cluster and value chains; direct firm level assistance, and financial sector reform. Several programs had multiple components that cut across BDS, market links, export promotion, and/or trade categories.

    The indicators show that studies examined impact at various levels: 1) BDS market development - commercial viability, client satisfaction, market and provider growth/development, and MSE integration into BDS markets; 2) product market development - indicators on overall growth and productivity, employment generation, trade, competitiveness and MSE integration into product markets, 3) market links; 4) enterprise development - sales, profitability and upgrading; (5) household level impacts; and 6) individual level impacts.

    Though not an exhaustive list, these evaluations seem to reflect much of the work to date on the impact of BDS programs.

    Summary of results
    The wide range of projects, activities, performance and contexts in these evaluations makes it difficult to draw general conclusions about the impact of MSE and other enterprise development programs. The studies reveal modest levels of change across variables, but the limited number of studies using strong impact assessment (IA) methodologies (quasiexperimental quantitative and well documented qualitative) makes it difficult to attribute change to projects and draw conclusions about impacts. The changing focus of evaluations has further limited conclusions.

    Findings suggest several ways that future impact studies of enterprise
    development programs can be improved:
    - Use more systematic and rigorous methodologies;
    - Conduct IA in the context of broader assessment frameworks that establish links between project inputs, outputs and outcomes before moving on to the question of impacts;
    - Focus more on issues related to integrating MSEs into value chains and clusters;
    - Increase attention to program impacts as they relate to poverty reduction; and
    - Improve dissemination of research and evaluation findings.

    Associated Activities and Documents
    Impact Assessment
    »Inventory and Analysis of Donor-Sponsored MSE Development Programs, USAID AMAP, DAI 2005
    Synthesis Documents
    »The New Generation of PSD Programming: The Emerging Path to Economic Growth with Poverty Reduction, USAID AMAP, 2006