LEARNING LESSONS - The COVID-19 Global Evaluation ...
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
LEARNING LESSONS Independent
Evaluation
Getty Images
January 2021
International Finance Institutions’
Support to the Deployment of COVID-19
Vaccines in Low- and Middle-Income
Countries: 10
10 Lessons from Evaluation
2020 witnessed unprecedented scientific progress in developing COVAX: The COVID-19 Vaccines
a vaccine to control the coronavirus (COVID-19) pandemic. Global Access (COVAX) Facility
This has led to the development of a range of effective vaccines was created in April 2020 to provide
spanning from traditional formulations to new ones, such as
equitable access to COVID-19
the novel mRNA or DNA-based vaccines. Many international
vaccines for all participating
organizations have committed their support to ensuring that the
vaccines are deployed and administered as quickly as possible members. COVAX offers a self-
across the globe. financing component for high-
income countries and an advance
For instance, international financial institutions (IFIs), like the market commitment that is open
World Bank and the Asian Development Bank (ADB), have to 92 lower- and middle-income
set aside substantial resources to support the deployment of countries, including 29 ADB
COVID-19 vaccines, including through the use of available member countries. COVAX is part
purchase mechanisms and the strengthening of health-related of the larger initiative, the Access
systems. In October 2020, the World Bank approved a $12 to COVID-19 Tools Accelerator,
billion envelope to finance the acquisition and deployment of to distribute new treatments,
COVID-19 vaccines and strengthening of related systems by its
therapeutics, and vaccines to fight
client countries.1 In December 2020, ADB approved the Asia
Pacific Vaccine Access Facility (APVAX), to provide a resource
COVID-19.
envelope of $9 billion and a framework to While COVAX has been designed to mitigate
support fast, high quality, safe, and equitable the complexity and improve the fairness in
vaccine access by its developing member the acquisition and distribution of COVID-19
countries (DMCs). ADB also emphasizes the vaccines, countries are complementarily
importance of working in congruence with pursuing additional mechanisms to ensure the
routine vaccination and other related health immunization of their population. Building on
services.2 the experience of ADB and other development
partners, in general, there are four interrelated
CHALLENGES TO VACCINE primary steps that countries and international
DEPLOYMENT PROGRAMS organizations need to be aware of with
respect to the deployment of vaccines. These
Delivering vaccines is a complex process, steps are outlined in Figure 1: planning and
comprising many moving parts. Financing coordination; budgeting; selecting; purchasing;
is only one part of the challenge of safely and deploying.
vaccinating priority populations. Through
the provision of technical assistance and LESSONS FROM PAST EFFORTS TO
in partnership with specialized technical SUPPORT VACCINE DELIVERY
agencies, IFIs can blend their finance with
other services to help countries access crucial Evaluation and research studies of financial
knowledge and experience. Countries need and technical responses to past public health
to make decisions on vaccine selection and crises, such as the 2009 influenza A (H1N1)
deployment to match their specific cultural, pandemic and introduction of new vaccines
political, and practical realities. in low- and middle-income countries, can
provide important lessons in making the
Unique Challenges with the COVID-19 Vaccine
In many ways, the distribution of a COVID-19 vaccine will be unique in the history of vaccination programs.
Some of the unique characteristics of this program include:
• Multiple vaccine choices. There are different vaccine choices, requiring different logistical
arrangements. Having a large number of vaccine candidates means that countries will be introducing
multiple vaccine products with different administration schedules, different shipping/storage/handling
requirements and different adverse event profiles against the same pathogen.
• Need for speed and scale. Unlike routine vaccination programs, the COVID-19 vaccine must be
deployed quickly, ideally to whole populations, to reach the desired level of immunity. The difficulty
will be to deliver quickly, and with a clear set of priorities determining who should get the first doses,
because the vaccination program will be prolonged. This is different from having adequate doses and
conducting a nationwide campaign targeting all ages simultaneously.
• Demanding cold chain and logistical process. While the requirements vary by type of vaccine, some
involve more than one dose and other special challenges, such as an ultra-cold chain for transportation
and a limited shelf life outside the cold chain.
• Need for close and continuous follow up of adverse events. The risk of adverse events is high
because of the short period for observation since vaccine development. Adverse events may occur
shortly after, or months after vaccination, and can be minor or serious, with an impact on vaccine
uptake in the long term. The virus itself is prone to mutations that need to be closely monitored.
• Risk of misinformation transmitted through social media. Given the universal focus on COVID-19,
vaccination campaigns will be uniquely vulnerable to false news and misinformation.
2
Figure 1. Simplified Model of a Vaccine Delivery System
Budgeting Selecting Purchasing Deploying
• Needs assessment • Regulatory approval • Legal requirements • Local distribution
• International financing by stringent regulatory • Transparency in • Communication
• Domestic financing authorities / WHO procurement • Demand
• Domestic regulatory • Tracking AEFI
approval
• Country readiness
Planning
Planningand
andCoordination
Coordination
COVID-19 vaccines available to populations Effective coordination of
around the world. As countries and other 2 partnerships, capitalizing on the
specialized international organizations strength of each partner, improves
assisting them plot the way forward to ensure both the national and international systems
the global deployment of COVID-19 vaccines, needed for the deployment of vaccines. In
here are 10 lessons, relevant to each one of the 2006, ADB provided support to its DMCs for
above mentioned primary steps. the Prevention and Control of Avian Influenza
in Asia and the Pacific Project, working with
Planning and Coordination several international organizations as well as
national and international nongovernment
Effective communication with organizations. Evaluation shows that the
1 receiving agencies at the country coordination among stakeholders that
level and clear discussion on the characterized this program served to
options available for support by IFIs, strengthen the sustainability of both national
promote ownership and improve and international vaccination systems.4 A
effectiveness. Evaluations have frequently similar lesson was drawn from the World Bank
attributed deficient performance to the lack of vaccine project designed to support the
stakeholder consultations and poor deployment of vaccines in a conflict-affected
understanding of expected roles in program population in Yemen. This project showed that
design and implementation.3 Developing the World Bank made extensive use of
complex policies often requires inputs from partnerships on the ground. This required
numerous stakeholders, and it is common to great flexibility on the part of the Bank, and
encounter issues of ownership if, for reasons good coordination among the different
of speed, consultation or coordination is weak. agencies playing direct roles in both
This often leads to confusion regarding procurement and distribution. It also required
responsibility and a lack of understanding of a continuous assessment to determine who
what is expected of each actor. Financing would be best suited to carry out expected
agencies must avoid the assumption that and required functions.5 In sum, several
“everybody is on board” and that commitment evaluations point to the fact that projects are
exists. less successful when coordination on the
3ground is weak and when IFIs rely on technical These include expenses to assess pre-
agencies only for procurement purposes, introduction, country readiness and
without capitalizing on the overall comparative supervision to identify and address problems
advantages of each organization.6 that can negatively affect the immunization
program. In these countries, immunization
Budgeting budgets often included expenses for vaccine
purchase, cold chain equipment, training of
Understanding of the national healthcare workers, and updates to the health
3 public financial management management information system but
process reduces delays and overlooked crucial operational costs that are
improves effectiveness. While COVID-19 key to successful deployment of a new
represents a major crisis, national budget vaccine, such as fuel for refrigerators and
procedures continue to operate, and transportation. This problem was more
governments have to allocate resources prevalent in countries with decentralized
through existing mechanisms. Vaccination health systems because operational costs were
projects in the past have been delayed or even typically funded by local governments and not
cancelled due to insufficient understanding of the national immunization program.10 These
government procedures. For example, in 2009, issues are likely to be more pronounced with
Mexico found itself at the epicenter of the COVID-19 vaccination programs, due to their
influenza A (H1N1) pandemic that had the large scale and complex and costly logistical
possibility to mutate to a deadly strain. The arrangements, including sequential
World Bank provided the government with a vaccination starting with population groups at
$538 million loan, including $426 million for highest risk.
the purchase and deployment of vaccine.7
Using emergency procedures, the project was Selecting
prepared in less than two months. However,
due to the intricacies of the government’s In the case of bilateral deals,
budgeting process, the loan was never made 5 assessing and considering the
effective and was cancelled. Bilateral aid complexity of regulatory and legal
agencies had a similar experience in Ghana, arrangements among stakeholders on the
where a lack of understanding of government ground is key to avoiding delays in vaccine
budgeting procedures led to significant delays selection. Past experience shows that
in the deployment of childhood vaccines.8 differences in country and organization
regulatory processes and legal requirements
Providing sufficient resources to can delay the selection and approval of
4 cover the full costs of delivering vaccines. During the 2009 influenza A (H1N1)
the vaccines to the population and pandemic, differences in national regulatory
reflecting these costs in the national processes, and the lack of a single legal
budgets lead to greater effectiveness. While framework acceptable to all parties led to
resources for purchasing vaccines may be bottlenecks and delays in the selection and
available, many other financing bottlenecks approval of vaccines. In more than half of the
can occur while budgeting their deployment. recipient countries, World Health
Data from the introduction of 15 vaccines in Organization (WHO) prequalification of a
10 countries in Africa and Asia showed that vaccine was not sufficient to obtain national
several steps involved in introducing vaccines regulatory approval. Even with
can lead to significant costs beyond normal prequalification, each country had a unique
country immunization program budgets.9 regulatory process and there was little
4harmonization even for emergencies. Only
countries that did not require national
registration for products donated by the
United Nations deemed the WHO
prequalification.11 The situation may be
different if countries get vaccines through
COVAX, but in the case of bilateral deals,
multiple vaccines available for selection each
made by a different manufacturer and each
requiring a separate approval and procurement
process complicate the challenge. Moreover, Vaccine delivery will be a global faultline that will run through
unlike the H1N1 situation, where country the first half of this decade, according to Agathe Demarais, of
the Economist Intelligence Unit. Key reasons include: securing
vaccine allocation was made taking into vaccine ingredients, production constraints, delays in delivery,
consideration whether there were sufficient poor medical infrastructure in some countries and lack of trained
doses available from a specific manufacturer, health workers to administer injections, among others. Photo
credit: Guillermo Legaria/Getty Images
many countries will be simultaneously
administering more than one vaccine product
in the fight against COVID-19. several small island nations or others small
states. This is similar to the approach that
Purchasing ADB is currently supporting in the Pacific to
introduce new vaccines.13 The Pan American
A high level of transparency in Health Organization’s (PAHO) experience
6 procurement is necessary to avoid with a revolving fund for vaccine procurement
complications and reputational as well as the Gulf Cooperation Council’s
risks. Once vaccines are selected, in case of (GCC) group purchasing program for medical
bilateral purchase agreements, procurement is products suggests that pooled procurement
not straightforward. Experience with can lead to cost savings and ensuring adequate
procurement of vaccines for the pandemic supply. However, it is a lengthy process. Both
influenza A (H1N1), suggests that bilateral programs started small and learned from
purchase agreements are likely to be experience before expanding. PAHO’s
complicated and technically demanding. For revolving fund started in 1979 and now serves
COVID-19 vaccines, this is compounded by more than 40 member countries. It is critical
the multiple options, the enormous demands, for the immunization program for several small
and the need for detailed legal agreements. island states in the Caribbean. The GCC’s
Given the volume of funding involved, there program has been in operation since 1978 and
can also be concerns about accountability in procures a range of medical products. Lessons
the use of resources, so a high level of from these programs suggests two key factors
transparency in the procurement process will of success when using this approach. First, the
be essential.12 design of the programs needs to attract the
participation of a sufficient number of
If using regional pooled countries to make the operations worthwhile
7 procurement arrangements, a and financially viable. Second, the programs
careful plan, prepared well in need to have transparency and ownership.
advance together with the participating Governance is important and both programs
countries is essential to deliver on time. The were backed by strong commitment from
use of pooled procurement of COVID-19 member countries, well-defined regulations
vaccines may be considered, for instance, for and procedures, high-quality leadership and
5continuity of program staff located in an
independent secretariat. Ad hoc or one-off
arrangements are unlikely to be effective.14
Deploying
Building on the infrastructure and
8 strength of existing immunization
programs allows for faster
deployment of new vaccine programs.
Nearly all countries have a routine childhood From installing freezers to setting up cold chain equipment,
immunization program, and some have arrangements are being made at a Delhi government hospital for
seasonal influenza programs. High levels of vaccine storage. Adequate and continuous training for frontline
health workers is key to successful vaccination programs.
coverage with routine childhood vaccines is an Photo credit: Manisha Mondal | ThePrint
indicator of a well-functioning system that can
deploy new vaccines quickly. Analysis of data of a successful vaccination program. Indeed,
from country responses to the 2009 influenza the Centers for Disease Control and
A (H1N1) pandemic indicates that countries Prevention indicate that recruiting and training
that had a seasonal influenza vaccination of providers is one of the most important steps
program were better prepared to receive and prior to actual distribution of the vaccine.17
use donated or purchased vaccines than those USAID experience shows the importance of
without a program. Countries with seasonal bearing on mind that while the physical act of
influenza vaccination programs are more vaccinating an individual is straightforward,
familiar with the regular management of there are many protocols that must be
vaccines and can quickly deploy new vaccine followed to ensure vaccine safety and
programs.15 Enabling factors for successful effectiveness.18 The protocols for newer
deployment of the influenza vaccine included vaccines tend to be more complex and
existing systems for effective planning, COVID-19 is likely to pose a major challenge,
importation, storage and delivery, vaccination, with different brands requiring different
adverse-event monitoring and waste protocols. Vaccine training needs to include:
management.16 Some low-income countries, storage, delivery, and waste management;
which benefit from the assistance of GAVI, monitoring and reporting (including
The Vaccine Alliance and UNICEF, may have monitoring of side effects); and
well-performing national immunization communication.19 This is further complicated
programs that can provide a foundation upon by the often low levels of education and
which COVID-19 vaccine deployment can be high-level of turnover for frontline health
built. Countries with weaker routine workers, necessitating continual training and
immunization programs will likely need early- supervision. This typically requires multi-day
stage and accelerated support to plan in training (typically one to four days for new
advance of the introduction of COVID-19 vaccines) as well as a system of cascading
vaccines. training down to the local level.
Providing adequate training to Successful vaccination campaigns
9 frontline technical staff is key to 10 require proactive communication
the success of vaccination strategies to inform the
programs. Health workers play a critical and population and address vaccine concerns
often overlooked part in the distribution chain and hesitancy. Ensuring that vaccines reach
6their intended beneficiaries requires outreach for Pandemic Preparedness and Response for
and public messaging. There will always be the WHO Executive Board, January 2021,22
misinformation about ongoing efforts, as has early evidence indicates that the risk of under-
been seen many times throughout the world; achievement is high. Delays and inequality in
for example, during the 1976 US swine flu plans for vaccine rollout will likely obstruct the
outbreak and 2009 influenza A(H1N1) in expected impact of the tremendous efforts
Southeast Asia. More recently, the large 2019 of countries and IFIs. The protracted nature
measles outbreak in Samoa is attributed to of this process and the extended support
misinformation about the side effects of that countries will need to complete the
vaccines, which led to the vaccination rate vaccinations have significant implications for
plummeting to 31% in the country.20 While IFIs like ADB, from a strategic, operational,
resistance to new public health interventions and corporate perspective. New lessons
is not unusual, with COVID-19 this is being will have to be drawn in real-time from this
amplified by social media transmitting rumors experience to ensure that responses adapt
and misinformation at lightning speed. In quickly and are up to the challenges posed by
addition, there is the risk that some adverse the COVID-19 crisis.
effects of the vaccines may be exaggerated,
leading to fear and increased vaccine Note: This paper is distilled from evidence-
resistance, with serious implications for public based evaluation knowledge. See also
health. In the Philippines, for example, public the policy brief based on presentations
concerns about the dengue vaccine in 2017 by PATH, a global nonprofit organization
led to a dramatic drop in public trust in dedicated to health for all, at the Policy
vaccines overall – from 93% “strongly Actions for COVID-19 Economic Recovery
agreeing” that vaccines are important in 2015 (PACER) Dialogues organized by the Asian
to 32% in 2018. This was combined with a Development Bank (ADB).
sharp drop in views on vaccine safety from
82% strongly agreeing that vaccines are safe in Acknowledgements: The production of
2015 to only 21% in 2018. The 2019 measles this note was co-led by Maya Vijayaraghavan
outbreak in the Philippines was attributed to (mvijayaraghavan@adb.org), Principal
the drop in vaccine coverage after the dengue Evaluation Specialist, and Erik Bloom
vaccine controversy.21 Communication (ebloom@adb.org), Senior Evaluation
strategies, media involvement and strategic Specialist, Independent Evaluation
engagement of stakeholders for new vaccine Department (IED), Asian Development Bank23
introduction can play a positive role, as the The note was prepared under the guidance of
case of India for the pentavalent and Hib IED Director General Marvin Taylor-Dormond
vaccines demonstrates. Messaging from local and IED Deputy Director General Véronique
leaders, celebrities and other credible Salze-Lozac’h.
individuals, can contribute a great deal.
ENDNOTES
Accelerating learning by incorporating new
[1] World Bank. 2020. Project Paper on a Proposed
lesson in real time Additional Financing to the COVID-19 Strategic
Preparedness and Response Program Using the Multiphase
While evaluation lessons from former crises Programmatic Approach. Washington, D.C.: World Bank.
are all useful and relevant, the current [2] Asian Development Bank. 2020. ADB’s Support to
situation is unprecedented and calls for Enhance COVID-19 Vaccine Access. Manila: ADB.
extraordinary efforts. As stated in the Second
Report Prepared by the Independent Panel [3, 8,9,10,18] USAID. Bottlenecks and Breakthroughs:
7Lessons Learned from New Vaccine Introductions in Low- [14] D. DeRoeck, S.A. Bawazir, P. Carrasco, M. Kaddar,
Resource Countries, 2008 to 2013. Report prepared by the A. Brooks, J. Fitzsimmons, and J. Andrus. 2006.
Maternal and Child Health Integrated Program for the Regional Group Purchasing of Vaccines: Review of the
U.S. Agency for International Development. Pan American Health Organization EPI Revolving Fund
and the Gulf Cooperation Council Group Purchasing
[4] Independent Evaluation Department. 2014. Project Program. International Journal of Health Planning
Completion Validation Report: The Prevention and Control Management. 21. pp.23–43.
of Avian Influenza in Asia and the Pacific Project. Manila:
ADB. [16] B.A. Ruscio, and P. Hotez. 2020. Global and
National Influenza Vaccine Systems to Match the
[5] World Bank. 2019. Implementation Completion and COVID-19 Pandemic Response. Vaccine. 38(50).
Results Report: Yemen - Health and Population Project. pp.7880-7882.
Washington, D.C.: World Bank.
[17] Centers for Disease Control and Prevention. 2020.
[6] Independent Evaluation Group. 2014. The World COVID-19 Vaccination Program Interim Playbook for
Bank’s Partnership with the GAVI Alliance. Washington, Jurisdiction Operations. Atlanta: CDC.
D.C.: World Bank Group; and Independent Evaluation
Group. 2011. The Global Fund to Fight AIDS, Tuberculosis [19] See for example the World Health Organization’s
and Malaria, and the World Bank’s Engagement with the COVID-19 vaccination training for health workers.
Global Fund. Washington, D.C.: World Bank Group.
[20] Adam T Craig, Anita E Heywood, Heather Worth.
[7] Independent Evaluation Group. 2012. ICR Review. 2020. Measles epidemic in Samoa and other Pacific
Influenza Prevention and Control. Washington, D.C.: islands. The Lancet Infectious Diseases: 20(3). pp. 273-
World Bank. 275.
[11, 15] R.M. Porter, S. Goldin, K.E. Lafond, L. Hedman, [21] H.J. Larson, K. Hartigan-Go, and Alexandre de
M. Ungkuldee, J. Kurzum, E. Azziz-Baumgartner, C. Figueiredo. 2019. Vaccine Confidence Plummets in the
Nannei, J.S. Bresee, A. Moen, A. 2020. Does having Philippines Following Dengue Vaccine Scare: Why It
a Seasonal Influenza Program Facilitate Pandemic Matters to Pandemic Preparedness. Human Vaccines &
Preparedness? An Analysis of Vaccine Deployment Immunotherapeutics. 15:3. pp. 625-627.
During the 2009 Pandemic. Vaccine. 38(5). pp. 1152-
1159. [22] Independent Panel for Pandemic Preparedness and
Response for the WHO Executive Board. 2021. Second
[12] CEPA. 2012. Lessons Learnt from the Accelerated Report on Progress. Geneva.
Vaccine Introduction (AVI) Project. Gavi Alliance. UK:
Cambridge Economic Policy Associates. [23] A draft version of this note was kindly commented
by W. William Schluter, Director of the Global
[13] Asian Development Bank. 2018. Report and Immunization Division, CDC, and Ulla Kou Griffiths,
Recommendation of the President to the Board of Director: Senior Advisor Health, Immunization Financing and
Proposed Loan and Grants to the Independent State Systems Strengthening, UNICEF. However, IED is fully
of Samoa, Kingdom of Tonga, Tuvalu, and Republic of responsible for the content of this note.
Vanuatu: Systems Strengthening for Effective Coverage of
New Vaccines in the Pacific Project. Manila: ADB.
Disclaimer
The views and assessments contained herein do not necessarily reflect the views of the Asian Development Bank (ADB) or its Board
of Directors or the governments they represent. ADB does not guarantee the accuracy of the data and accepts no responsibility for any
consequence of their use.
Learning Lessons is a synthesis of key lessons drawn from evaluations of ADB-supported projects and programs as well as from
evaluations published by other international organizations . This synthesis may include contexts derived from literature review. Lessons
presented in this brief are not prescriptive, and users are advised to carefully review these lessons in the context of country, sector, and
thematic conditions.
About the Independent Evaluation at Asian Development Bank
The Independent Evaluation Department evaluates the policies, strategies, operations, Independent
and special concerns of the Asian Development Bank relating to organizational and
operational effectiveness. It contributes to development effectiveness by providing
feedback on performance and through evaluation lessons.
EvaluationYou can also read
