plos PLoS Currents: Disasters 2157-3999 Public Library of Science San Francisco, USA 10.1371/currents.dis.706b7dc0e8382b55a20d6f7d0cf14257 Research Article Characteristics of an Effective International Humanitarian Assistance: A Systematic Review Moslehi Shandiz Department of Disaster Public Health, School of public Health, Tehran University of Medical Sciences, Tehran, Iran; Department of Disaster and Emergency Health, National Institute of Health Research, Tehran University of Medical Sciences, Tehran, Iran Ardalan Ali Disaster and Emergency Health Academy, National Institute of Health Research, School of Public Health, Tehran University of Medical Sciences, Tehran, Iran; Harvard Humanitarian Initiative, Harvard University, Cambridge, USA

Associate Professor and Department Chair at the 1) Department of Disaster & Emergency Health, National Institute of Health Research, Tehran University of Medical Sciences, Tehran, Iran and 2) Department of Disaster Public Health, School of Public Health, Tehran University of Medical Sciences, Tehran, Iran

http://nihr.tums.ac.ir/disaster
Waugh William Department of Public Management and Policy, Andrew Young School of Policy Studies, Georgia State University, Georgia, USA Tirone Daniel C. Department of Political Science, Louisiana State University, Baton Rouge, Louisiana, USA https://sites.google.com/site/danielctirone/ Akbarisari Ali Department of Health Care Management, School of Public Health, Tehran University of Medical Sciences, Tehran, Iran
25 2 2016 ecurrents.dis.706b7dc0e8382b55a20d6f7d0cf14257 2019 Moslehi, Ardalan, Waugh, Tirone, Akbarisari, et al This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Introduction: The objective of this study is to identify the effectiveness characteristics, review the definition of them, and develop a conceptual mapping of existing domains in the field of International Humanitarian Assistance (IHA).

Methods: We conducted a systematic review and searched the major databases (Science Direct, Scopus, Springer and Pubmed) and grey literature, including references of potentially eligible articles and conference proceedings through March 2015. Articles were included if they focused on IHA effectiveness. Reviewers independently identified the eligible studies and extracted data.

Results: 10 studies were included and 48 characteristics were identified. There is a lack of scientific studies and agreement on the characteristics of IHA effectiveness. 

Conclusion: This study could be the step toward an understanding of IHA effectiveness characteristics and its definitions with the findings making a base line for more research in this area. 

effectivenesshumanitarian aidhumanitarian assistancerelief The authors received no specific funding for this work.
Introduction

According to the trends in the data, the number of reported disasters and the numbers of people killed or otherwise affected by natural disasters have increased over the past 50 years 1. Based on the Office for the Coordination of Humanitarian Affairs (OCHA) report of global humanitarian response 2014, 17 countries were affected; 568 aid organizations participated in Humanitarian Assistance (HA); and there was a large gap between funded and requested fund programs1,2. It is estimated that 102 million people are in need of HA, while it was 81 million in December 2013. Global financial requirements to cover humanitarian needs are US$17.3 billion compared with US$12.9 billion in 20133. Such increases in the level of HA funds are leading to the increased importance of effective allocation4,5.

In 1962, Morgenthou defined the HA as a foreign aid which governments extend to nations faced with natural disasters6. In this definition the author did not mention the man-made disasters such as wars and armed conflicts. OCHA defines the HA as: “Aid that seeks, to save lives and alleviate suffering of a crisis-affected population” 7. The aim of HA is to protect the lives of the affected people during and after disasters8. International HA (IHA) will be provided when the governments of the affected countries are unable to provide relief in an effective way. The assistance is provided in emergency situations to save and protect human life in both natural and man-made disasters and to maintain human dignity9. This assistance should be allocated in an effective way in order to maximize its impact.

Etzioni in 1964 was one of the first researchers who defined the goal attainment approach of effectiveness10,11. Georgopoulos and Tannenbaum in 1957 and Yuchtman and Seashore in 1967 emphasized a financial approach 12,13. The approach of reputation was evolved from Georgopoulos and Mann’s project in the hospital system14. This approach relies on subjective performance measures reported by informants or organizational stakeholders15. After that the literature focused on the definition of more complex methods of effectiveness, such as multi-dimension models 16,17,18, competing values models 19, contingency models 20, and balanced scorecard21 approaches. These methods tried to unify the aspects of goal attainment, resource control and reputation approaches 22.

HA effectiveness is defined as: objectives achievement23 and the extent to which the interventions’ immediate objectives were achieved, measured by the timeliness of payments and the targeting of aid on the needs of affected individuals24. As recognized at the Consultation Workshop for Humanitarian Effectiveness in 2013, identifying ways of defining, measuring and improving the effectiveness of HA was one of the thematic areas of focus for the World Humanitarian Summit (WHS)25. Defining clear targets for improving aid effectiveness was the result of Paris Conference on Aid Effectiveness in March 2005 26 . During the Rome Conference on Harmonization and Alignment in 2003, about 50 countries spent two days discussing the ways they could improve the effectiveness of their works26. Also at the Monterrey Conference on Finance for Development in 2002, it was mentioned that increasing aid volume will not reduce the poverty. Instead, the aid should be more effective27.

IHA organizations are expected to save lives with limited resources28. Measuring the effectiveness of IHA became an OCHA priority in 2013-201425, increasing humanitarian actions more than 6 fold from 1990 to 201229. The shortage of empirical studies about effectiveness of IHA lead us to conduct this study. So the objective of this study is to identify the effectiveness characteristics, review their definitions and develop a conceptual mapping of existing characteristics in the field of IHA.

Methods

Study design

The approach is comprehensive and involves a search strategy including international, relevant studies. It followed the Preferred Reporting Items for Systematic Review and Meta-analysis statement (PRISMA) guidelines. In addition, we reviewed the references in potentially eligible articles, conference proceedings, dissertations, organizational reports and documents of the major IHA organizations to minimize the publication bias.

Search methods for identification of studies

Search scientific articles:

We conducted a systematic search in electronic databases (from their inception to the latest available entry date) on 3 March 2015:including Science Direct, Scopus, Springer and Pubmed.

The search language was restricted to English. The search keywords were selected after consulting the experts. Databases were searched with the following keywords: humanitarian* AND assistance* OR relief* OR aid* AND effectiveness*. In addition, the references of relevant articles were reviewed. This helped to evaluate the effectiveness of the search strategy and to identify any articles overlooked due to unusual keywords.

Search other resources:

This search was conducted following organizational reports and documents: OXFAM, OCHA, Save the Children, Swedish International Development Cooperation Agency (SIDA), Canadian International Development Agency (CIDA), Active Learning Network for Accountability and Performance (ALNAP), Department for International Development (DFID) and Humanitarian Practice Network (HPN).

Study eligibility

Inclusion criteria:

Eligible studies included English-language articles with no limitation in time of studies. No other filters were applied. Articles were included if they focused on IHA and effectiveness characteristics. Both quantitative and qualitative studies were included. Also, organizational reports and documents were included.

Exclusion Criteria:

Non English studies, the irrelevant studies to IHA effectiveness and articles with similar keywords but covering unrelated topics were excluded.

Data collection and analysis

Selection of studies:

After excluding duplicated studies, references found through the search were evaluated for inclusion criteria by reading their titles and abstracts. After an initial screening of abstracts of those that remained, the first author (SM) read and analyzed their full texts to determine their eligibility. This was done under the supervision of second author (AA). The first author had passed IHA course at the Harvard Humanitarian Initiative. Quality assessment was performed using study design appropriate Critical Appraisal Skills Program (CASP) tools. Disagreements were resolved by consensus or by consulting other authors.

Data extraction and analysis:

We extracted the following data from each study report: organization; characteristics or elements influence the IHA effectiveness; and the definition of characteristics. Data extraction forms were designed using Microsoft Excel. Extracted data were presented in descriptive tables.

Results

A total of 494,675 potentially relevant articles were identified during the search. The analysis contained 92 articles that met the inclusion criteria. A full text review of these articles led to 10 studies. The study selection flow diagram is shown in Figure 1 and Figure 2 shows the Description and Mapping of IHA effectiveness characteristics.

Figure 1_DIS-42

Table 1_DIS-42

Discussion

Based on our results, we could find the main characteristics of IHA effectiveness, but there was no standard definition for each of them. We tried to develop a conceptual mapping of existing characteristics of IHA effectiveness.

There is a matter of terminology in IHA subject and relief programs28. Although there were a lot of meetings for identifying IHA effectiveness 25,26, there is no single and accurate definition of IHA effectiveness 27. Organizations tend to report the positive aspects and ignore the negative points. Some may report the international efforts and the others may report the outcomes to define their effectiveness87. Governmental, intergovernmental, NGOs and other organizations may define the effectiveness due to their own perspectives 88. There is a global agreement on defining criteria which should be used to measure the IHA effectiveness. These criteria should help the decision makers to plan, implement and evaluate the IHA in a standard way 28. So, OCHA held a consultative workshop in 21 March 2013 to identify ways of defining, evaluating and improving IHA effectiveness 25.

We categorized and present the data found from IHA organizations documents and reports (the grey literature). The study results appear to be over-predicted and the number of studies found was lower than the value we expected. This confirms previous findings in the literature. Some studies agreed that HA evaluators face with the lack of appropriate data28. Also, all the humanitarian organizations do not upload their evaluation reports and documents on their websites to disseminate the results with other organizations or researchers 87 . Lack of access to reports and documents from IHA organizations can reduce the quality of aid provided 89,90.

The most striking result to emerge from the data was the lack of systematic reviews in the subject of HA. There is a global need for conducting HA systematic reviews. Because conducting systematic reviews could play an important role in increasing the effectiveness of IHA in all the phases of a disaster91. As mentioned by Bonnix Kayabu and Mike Clarke 83% of their study participants said that systematic reviews are useful in disasters and 69% of their participants strongly agreed that findings of systematic reviews with the subject of IHA could have a positive role in humanitarian programs91.

There was no standard category for these characteristics. In the Paris Declaration there were five defined categories: mutual accountability, managing for results, harmonization, alignment and ownership 51. In OECD Development Assistance Committee (OECD-DAC) there were also five defined categories: relevance, appropriateness, connectedness, efficiency, effectiveness and impact 25. This model was criticized because of being too developmentally focused and the absence of effectiveness indicators 25. We could not categorize all these characteristics in these defined categories.

Limitations:

The most important limitation of this study lies in the fact that we only used English articles, documents and reports. We may have missed some important published articles, documents and reports in other languages. The findings also might not be representative of all IHA organizations, since we had analyzed the documents of major IHA organizations.

Conclusion:

In this review we have presented the IHA effectiveness characteristics from previous reports and documents. The evidence from this study supports the idea that there is a lack of scientific studies in the field of IHA effectiveness. Also the findings of this study indicate that there is no agreed definition for IHA effectiveness characteristics, although there is a global need for conducting IHA effectiveness evaluation. We developed a conceptual mapping of existing characteristics of IHA effectiveness. Although this study is a first step toward our understanding of IHA effectiveness, the findings add to growing body of literature on IHA effectiveness.

Supporting Information

PRISMA Checklist

Competing Interest

The authors have declared that no competing interests exist.

Correspondence

Ali Ardalan. Email: aardalan@tums.ac.ir, ardalan@hsph.harvard.edu

This article is a part of a PhD dissertation from Tehran University of Medical Sciences, Tehran, Iran.

References Riddell, R.C., Does Foreign Aid Really Work? 2007, Oxford: Oxford University OCHA, Overview of Global Humanitarian Response 2014. 2014, OCHA: Geneva OCHA, Global Humanitarian Overview: Status Reprt. 2014, OCHA: Geneva Hallam, A., Cost-effectiveness Analysis: A Useful Tool for the Assessment and Evaluation of Relief Operations? 1996, Relief and Rehabilitation Network: London. Arola, S.A., Humanitarian Aid: Problems and Solutions, in International Relations. 2007, Webster University: London. p. 57. Morgenthou, H., A Political Theory of Foreign Aid. The American Political Science Review, 1962. 56: p. 301- 309. OCHA, Glossary of Humanitarian Terms. 2003, United Nations: New York. Australian Development Cooperation, International humanitarian aid, Europe, Editor. 2009: Vienna. p. 7- 9. Europian commission, Council Regulation No 1257/96 Concerning Humanitarian Aid, E. Commission, Editor. 1996. Etzioni, A., Modern Organisations. 1964, NJ: Englewood Cliffs. Spar, D. and Dail, J., Of Measurement and Mission. Accounting for Performance in Non-Governmental Organizations. Chicago Journal of International Law, 2002. 3(1): p. 171-182. Georgopoulos, B.S. and Tannenbaum, A.S., A Study of Organizational Effectiveness. American Sociological Review, 1957. 22(5): p. 534-540. Yuchtman, E. and Seashore, S.E., A System Resource Approach to Organizational Effectiveness. American Sociological Review, 1967. 32(6): p. 891-903. Georgopoulos, B.S. and Mann, F.C., The Community General Hospital. 1962, New York: MacMillan. Jobson, J.D. and Schneck, R., Constituent Views of Organizational Effectiveness: Evidence from Police Organizations. The Academy of Management Journal, 1982. 25(1): p. 25-46. Cameron, K.S. and Whetten, D.A., Some Conclusions about Organizational Effectiveness. Organizational Effectiveness: A Comparison of Multiple Methods, ed. K.S. Cameron and D.A. Whetten. 1983, New York: Academic Press. Zammuto, R.F., A Comparison of Multiple Constituency Models of Organizational Effectiveness. The Academy of Management Review, 1984. 9(4): p. 6060-616. Foster, M.J. and Lock, A.R., Factoring Effectiveness Factors! The Journal of the Operational Research Society, 1990. 41(2): p. 111-117. Quinn, R.E. and Rohrbaugh, J., A Spatial Model of Effectiveness Criteria: Towards a Competing Values Approach to Organizational Analysis. Management Science, 1983. 29(3): p. 363-377. Lewin, A.Y. and Minton, J.W., Determining Organizational Effectiveness: Another Look, and an Agenda for Research. Management Science, 1986. 32(5): p. 514-538. Kaplan, R.S. and Norton, D.P., Using the Balanced Scorecard as a Strategic Management System. Harvard Business Review, 1996. 74(January/February): p. 75-85. Herman, R.D. and Renz, D.O., Theses on Nonprofit Organizational Effectiveness. Nonprofit and Voluntary Sector Quarterly, 1999. 28(2): p. 107-126. OCHA, Measuring the Effectiveness of Humanitarian Action: Risk, Adaptation and Innovation in Humanitarian Action, in OCHA Policy Conference. 2011: NY. Metz,M., et al., Comparing the efficiency, effectiveness and impact of food and cash for work interventions: lessons from South Sudan. Humanitarian Exchange Magazine, 2013(57). OCHA, Consultative workshop for Humanitarian Effectiveness. 21 March 2013, OCHA: New York. Stolk, B., A Research on Aid Effectiveness: Donor Performance on Coordination and Harmonisation in Sri Lanka, in Centre for International Development Issues Nijmegen. 2006, University of Nijmegen, the Netherlands. hort, C. Harmonisation: Improving Aid Effectiveness and Supporting Poverty Reduction Strategies. 2003; A message from the Secretary of state to DFID staff]. Available from: ODI, Accountability in Disaster Response: Assessing the Impact and Effectiveness of Relief Assistance, in Relief and Rehabilitation Network. 1995, ODI. Ramalingam, B., et al., Counting what counts: performance and effectiveness in the humanitarian sector. 2012, ALNAP: London. p. 1-90. UNISDR. What is Disaster Risk Reduction? 2014 [cited 2014 December]; Available from: Taylor, G., et al., The state of the humanitarian system. 2012, ALNAP: London.p. 53 - 68. Action, P. Reducing Vulnerability. 2014 [cited 2014 December]; Available from: UNISDR and ESCAP, Reducing Vulnerability and Exposure to Disasters, The Asia-Pacific Disaster Report 2012. 2012, UNISDR: Bangkok. UNISDR, Hyogo Framework for Action 2005-2015: Building the Resilience of Nations and Communities to Disasters, in World Conference on Disaster Reduction. 2005, UNISDR: Kobe, Hyogo, Japan. UNISDR, Building Disaster Resilient Communities. 2007, UNISDR: Geneva. UNDG. Results-based Management. 2014 [cited 2014 December]; Available from: Buchanan-Smith, M. and Telford, J., AN INTRODUCTION TO EVALUATION OF HUMANITARIAN ACTION (EHA), ALNAP: London. DFID, Andrew Mitchell: The outcome of the High Level Forum on Aid Effectiveness, in Reducing hunger and malnutrition in developing countries and International aid and development. 2011. CIDA, A Review of Evidence of the Effectiveness of CIDA's Grants and Contributions. 2006, CIDA: Quebec.p.1-126. UNDP. Democratic Governance 2014 [cited 2014 December]; Available from: DFID, Aid effectiveness: Mitchell co-chairs 'post-Busan' panel, in Increasing the effectiveness of UK aid and International aid and development. 2012, Department for International Development. Fortin, M. Goal Achievement. 2014 [cited 2014 December]; Available from: Wheelwright, V. The Power of the Long-Term Perspective. 2011 [cited 2014 December]; Available from: IFRC, World Disaster Report Focus on Resilience. 2004, IFRC: Geneva. Kreidler, C., The role of donors in enhancing quality and accountability in humanitarian aid. Humanitarian Exchange Magazine, 2011. October(52) SIDA, The Accra Agenda for Action From a Democracy, Human Rights and Gender Equality Perspective: A Broadened and More Inclusive Aid Effectiveness Agenda. 2010, SIDA.p.1-12. Dictionary, Economic Growth. 2014 CIDA, Canada Introduces a New Effective Approach to its International Assistance. 2009, CIDA: Toronto Russell, J. Aid effectiveness. 2013 February 2013 [cited 2014 8 September]; Available from: CIDA. Securing the future of children and youth. [cited 2014 December]; Available from: OECD, The Paris Declaration on Aid Effectiveness and the Accra Agenda for Action. 2008. CIDA. CIDA's Policy on Poverty Reduction. 1996 [cited 2014 December]; Available from: Ocxford. Oxford Dictionary. [cited 2014 December]; Available from: UNOPS, Delivering Sustainable Results. 2012, UNOPS Save the Children, Modernizing Foreign Assistance Insights from the Field: HAITI. 2009, Save the Children.p. 1-20. OECD, The Busan Partnership for Effective Development Cooperation. 2012, The Fourth High-Level Forum on Aid Effectiveness (HLF4) CIDA, Key Issues for Canada at Busan. 2011, CIDA. Security, H. Plan and Preper for Disasters. 2013 [cited 2014 December]; Available from: Danailov, S. and Michel, S., Ensuring the predictability of emergency response: the DRC Rapid Response Mechanism. Humanitarian Exchange Magazine, 2008(39). UNDESA. Youth Participation. 2014 [cited 2014 December]; Available from: Agency, C.I.D., Minister Oda announces next step to CIDA's Aid Effectiveness. 2010, CIDA: Ottawa. DAC Criteria for Evaluating Development Assistance. 1991 [cited 2014 December]; Available from: Wateraid. Delivering Services. [cited 2014 December]; Available from: Young, J., et al., RAPID OUTCOME MAPPING APPROACH: a guide to policy engagement and influence, Overseas Development Institute: London. Business, S. Managing People. [cited 2014 December]; Available from: ALNAP. Humanitarian Leadership. [cited 2014 December]; Available from: Yourdictionary. Timeliness. [cited 2014 December]; Available from: Definition of Human Security. 2011 [cited 2014 December]; Available from: Moslehi, S., et al., The Challenges and Recommendations of Accessing to Affected Population for Humanitarian Assistance: A Narrative Review. Global Journal of Health Science, 2015. 7(3): p. 111- 115. Wikipedia. Multi-Agency Coordination. [cited 2014 December]; Available from: Martin, M. and S. Moser, Exiting Conflict, Owning the Peace: Local Ownership and Peacebuilding Relationships in the cases of Bosnia and Kosovo. 2012, Friedrich-Ebert-Stiftung: Berlin. OECD. Strengthening and Using Country Systems. [cited 2014 December]; Available from: Food Security. in World Food Summit 1996: WHO. Adams, L.M., Exploring the Concept of Surge Capacity. The Online Journal of Issues in Nursing, 2009. 14(2). OCHA. Surge Capacity. [cited 2014 December]; Available from: Terman, D.L. and Behrman, R.E., Adequate and Equitable Funding. Financing Schools, 1997. 7(3). Phillips, C. What is costeffectiveness? 2009 [cited 2014 December]; Available from: OECD. Challenges, opportunities and approaches for increasing joint donor [cited 2014 December]; Available from: WHO. Humanitarian Health Action. 2007 [cited 2014 December]; Available from: UNEP. Public Awareness. [cited 2014 December]; Available from: Education about Social Welfare [cited 2014 December]; Available from: Burns, K. and Lillejord, J., What is Gender Equality in Humanitarian Action? 2012 [cited 2014 December]; Available from: CIDA, Gender Equality and Humanitarian Assistance: A guide to the issues. 2003, Canadian International Development Agency (CIDA): Quebec. Ayton-Shenker, D. The Challenge of Human Rights and Cultural Diversity. 1995 [cited 2014 December]; Available from: Victims Rights. [cited 2014 December]; Available from: Women's Empowerment. 2008 [cited 2014 December]; Available from: Oliver, M., Evaluation for Emergency Response: Humanitarian Aid Agencies and Evaluation Use. 2008, Georgia State University: Georgia. Delgatty, C.R., Evaluating the Effectiveness of Short Term Humanitarian Aid. 2011, Texas State University-San Marcos: San Marcos, Texas. Mills, E.J., Sharing evidence on humanitarian relief: Needs a publicly accessible, searchable, and comprehensive database. BMJ, 2005. 331: p. 1485- 1486. Chalmers, I., Government regulation is needed to prevent biased under-reporting of clinical trials. BMJ, 2004. Kayabu, B and Clarke, M., The Use of Systematic Reviews and Other Research Evidence in Disasters and Related Areas: Preliminary Report of a Needs Assessment Survey. PLOS Currents Disasters, January 3013.