BOARD INVITED REVIEW: Prospects for improving management of animal disease introductions using disease-dynamic models - UNL Digital Commons
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
University of Nebraska - Lincoln DigitalCommons@University of Nebraska - Lincoln USDA National Wildlife Research Center - Staff U.S. Department of Agriculture: Animal and Plant Publications Health Inspection Service 4-10-2019 BOARD INVITED REVIEW: Prospects for improving management of animal disease introductions using disease-dynamic models Ryan S. Miller United States Department of Agriculture-Veterinary Services, ryan.s.miller@usda.gov Kim M. Pepin United States Department of Agriculture- Wildlife Services Follow this and additional works at: https://digitalcommons.unl.edu/icwdm_usdanwrc Part of the Natural Resources and Conservation Commons, Natural Resources Management and Policy Commons, Other Environmental Sciences Commons, Other Veterinary Medicine Commons, Population Biology Commons, Terrestrial and Aquatic Ecology Commons, Veterinary Infectious Diseases Commons, Veterinary Microbiology and Immunobiology Commons, Veterinary Preventive Medicine, Epidemiology, and Public Health Commons, and the Zoology Commons Miller, Ryan S. and Pepin, Kim M., "BOARD INVITED REVIEW: Prospects for improving management of animal disease introductions using disease-dynamic models" (2019). USDA National Wildlife Research Center - Staff Publications. 2268. https://digitalcommons.unl.edu/icwdm_usdanwrc/2268 This Article is brought to you for free and open access by the U.S. Department of Agriculture: Animal and Plant Health Inspection Service at DigitalCommons@University of Nebraska - Lincoln. It has been accepted for inclusion in USDA National Wildlife Research Center - Staff Publications by an authorized administrator of DigitalCommons@University of Nebraska - Lincoln.
Journal of Animal Science 97(6):2291-2307. doi: 10.1093/jas/skz125
BOARD INVITED REVIEW: Prospects for improving management of animal
disease introductions using disease-dynamic models
Ryan S. Miller*,1 and Kim M. Pepin†
*Center for Epidemiology and Animal Health, United States Department of Agriculture-Veterinary Services,
Fort Collins, CO 80526; and †National Wildlife Research Center, United States Department of Agriculture-
Wildlife Services, Fort Collins, CO 80521
ABSTRACT: Management and policy decisions demonstrate how disease-dynamic models can
are continually made to mitigate disease introduc- improve mitigation of introduction risk. We also
tions in animal populations despite often limited identify opportunities to improve the application
surveillance data or knowledge of disease trans- of disease models to support decision-making
mission processes. Science-based management is to manage disease at the interface of domestic
broadly recognized as leading to more effective and wild animals. First, scientists must focus on
decisions yet application of models to actively objective-driven models providing practical pre-
guide disease surveillance and mitigate risks re- dictions that are useful to those managing disease.
mains limited. Disease-dynamic models are an In order for practical model predictions to be
efficient method of providing information for incorporated into disease management a recog-
management decisions because of their ability nition that modeling is a means to improve man-
to integrate and evaluate multiple, complex pro- agement and outcomes is important. This will be
cesses simultaneously while accounting for uncer- most successful when done in a cross-disciplinary
tainty common in animal diseases. Here we review environment that includes scientists and decision-
disease introduction pathways and transmission makers representing wildlife and domestic animal
processes crucial for informing disease manage- health. Lastly, including economic principles of
ment and models at the interface of domestic value-of-information and cost-benefit analysis in
animals and wildlife. We describe how disease disease-dynamic models can facilitate more ef-
transmission models can improve disease man- ficient management decisions and improve com-
agement and present a conceptual framework for munication of model forecasts. Integration of
integrating disease models into the decision pro- disease-dynamic models into management and
cess using adaptive management principles. We decision-making processes is expected to improve
apply our framework to a case study of African surveillance systems, risk mitigations, outbreak
swine fever virus in wild and domestic swine to preparedness, and outbreak response activities.
Key words: adaptive management, disease, domestic, interface, transmission model, wildlife
Published by Oxford University Press on behalf of the American Society of Animal Science 2019.
This work is written by (a) US Government employee(s) and is in the public domain in the US.
J. Anim. Sci. 2019.97:2291–2307
doi: 10.1093/jas/skz125
INTRODUCTION challenging to manage. There are multiple pos-
sible routes of initial pathogen introduction.
Diseases that can be transmitted between
Some pathogens are readily transmitted from
domestic animals and wildlife are especially
wildlife to domestic host species and vice versa,
Corresponding author: ryan.s.miller@usda.gov
1 which can complicate elimination. One example
Received February 19, 2019. is the introduction of African swine fever virus
Accepted April 10, 2019. (ASFv) in 2007 from Africa into Georgia and
22912292 Miller and Pepin
subsequent spread throughout Europe and Asia assimilate and evaluate multiple, complex processes
causing economic losses greater than US$267 concurrently and rapidly.
million in Russia alone (Sánchez-Cordón et al., A major gap in quantitative model develop-
2018). While domestic swine were initially con- ment is to estimate pathogen introduction risks by
sidered the primary species involved in the epi- considering disease processes in both the source
demic, wild boar are now recognized to have an and recipient host populations (Lloyd-Smith
important role in the spread and maintenance et al., 2009). This is important because changing
of ASFv throughout affected regions (Gallardo ecology in either source or recipient host popu-
et al., 2015). An additional example is the re- lation can dramatically alter introduction risk by
cent emergence and rapid global circulation of changing the dynamics involved in the introduc-
the Goose/Guangdong (GsGD) lineage of highly tion pathway. Thus, inference based solely on a
pathogenic avian influenza virus (e.g., subtypes single component population or on retrospective
H5N1, H5N2, and H5N8) (Verhagen et al., 2015). patterns could produce erroneous predictions as
In North America, Clade 2.3.4.4 GsGD lineage conditions change. A second issue is that many
was introduced through wild bird migratory analytical tools remain idiosyncratic, investigating
routes resulting in reassortment with local strains disease dynamics in local source populations. It
and a multiyear (2014 to 2015) outbreak in com- can be difficult to extrapolate findings based on
mercial poultry with economy-wide losses of at locally focused systems for disease management
least US$3.3 billion (Greene, 2015; Hill et al., decisions at broader spatial scales, or policy im-
2017). At least 18 independent introductions from plementation that is typically at state, regional,
wild birds into commercial poultry occurred (Li or national scales. Lastly, despite the prospects
et al., 2018) as well as transmission from com- of analytical tools to better understand disease
mercial poultry back into wild bird populations introduction risks and support disease manage-
(Ramey et al., 2018). While management was ment and policy-making at the wildlife–domestic
eventually effective, it did not prevent reintro- animal interface, decisions often rely on expert
ductions from wildlife species. These major eco- opinion that is based on historical experiences
nomic burdens and complex ecologies illustrate (Joseph et al., 2013).
the need to develop risk assessment systems that To address these gaps we first review intro-
aim to better understand and predict drivers of duction pathways and disease transmission in the
new introductions. context of ecological processes that are crucial for
Key challenges for management of pathogen informing disease management and policy decisions
introductions into domestic animals include esti- at the interface of domestic animal production sys-
mates of introduction risk, surveillance of patho- tems and wildlife. We then describe how disease
gens and what to do with findings, and how to transmission models can improve disease man-
apply biosecurity and other mitigation strategies agement, specifically, for decision-making in risk
to minimize introduction risks. Routes of pathogen assessment, response planning, and surveillance
introduction can include complicated trade net- design. Next we introduce a conceptual framework
works of domestic animals and their products, as for improving management of introduction risks
well as air travelers; both of which are frequently for diseases with complex ecology, focusing on how
poorly described. Introduction can also occur via models can improve decision-making. We then
wildlife species with complex ecology and lead to apply our framework to an important case study,
spillover and spillback between domestic animals ASFv in wild and domestic swine, to demonstrate
and wildlife, driven by ecological processes that opportunities for informing disease preparedness
are often ill-understood. Thus, understanding and and response. We conclude with a discussion of op-
predicting introduction pathways is not straightfor- portunities to bridge current gaps between disease
ward—quantitative models can be important tools research and management.
for interpreting the outcome of multiple, complex
component processes, and for assimilating uncer-
tainty in surveillance data and ecological processes ECOLOGICAL PROCESSES GOVERNING
to provide information to improve management
DISEASE EMERGENCE
decisions (Pepin et al., 2014; Huyvaert et al., 2018;
Manlove et al., 2019). Models can also be the first, New introductions of a pathogen into a naïve
most efficient method of providing information for domestic animal production system can originate
management decisions because of their ability to by contamination from the same domestic animalApplying models in disease management 2293
production system in another area (e.g., movement can occur by importation of infected domestic ani-
of animals within a country or transboundary) or mals from other countries or from farms within the
from another host species located in the same or a same country.
different area. Introduction into a wildlife popula-
tion results from similar processes. We distinguish Cross-Species Transfer
these processes as “lateral” versus “cross-species”
transfer events, respectively. Both pathways can pose As with lateral transfer, transmission mechan-
a risk to a particular domestic animal population isms between host species can involve fomites, vec-
or wildlife population, and involve several different tors, environmental persistence, or direct contact
ecological and epidemiological processes (Fig. 1A) (Fig. 1B). An additional layer of complexity with
that need to be understood for determining optimal cross-species transfer is that donor host (i.e., host
management strategies. population that the pathogen originates) ecology
may be significantly different than recipient host
Lateral Transfer (i.e., host population that receives the pathogen)
ecology, which can impose additional constraints
Domestic-to-domestic animal introductions for establishment and ongoing spread (Pepin et al.,
can occur by multiple different mechanisms, for ex- 2010; Plowright et al., 2017). Additionally, disease
ample, exposure to fomites or carcasses, direct con- dynamics in one species can greatly influence the
tact with domestic animals, or contact via vectors probability of cross-species transfer and in some
(Fig. 1A). Exposure to fomites can occur through cases persistence of the disease in the recipient
many routes including consumption of contamin- host when repeated introductions are required to
ated human food waste, animal feed, or mechanical maintain transmission (Lloyd-Smith et al., 2009).
transport by humans or equipment that have come For example, changing prevalence of a pathogen
into contact with infected domestic animals. Direct (or virulence) in wildlife can influence the risk of
contact with infected domestic animals can be an- transmission to domestic animals. When wildlife
other significant route of lateral transfer, which migrate they can also impose risk over a broader
Figure 1. Conceptual cycle of ecological processes governing lateral transfer of disease and subsequent establishment and transmission among do-
mestic and wild animal hosts (panel A). Lateral transfer can occur through various pathways into either domestic or wild animals. Transmission between
domestic and wild animals can occur directly or indirectly via the environment or vectors. Panel B describes a noninclusive representation of potential
processes for lateral transfer and transmission of African swine fever virus (ASFv). Transfer directly into wild suid species is thought to primarily occur
through contact with contaminated swine products that are imported or carried by travelers. Transfer directly into domestic swine can occur via contamin-
ated products or through infected domestic swine. Whether ASFv is present in either wild or domestic populations various routes of transmission (direct
and indirect) can facilitate cross-species transmission (i.e., spillover or spillback). Direction of arrows indicates expected direction of transmission (i.e.,
source and donor populations). Dotted arrows indicate hypothesized routes of transmission that are currently less supported by available data.2294 Miller and Pepin
spatial area compared with a donor host species individuals contact one another and 2) who con-
that does not move very far (Manlove et al., 2019). tacts who (i.e., which individuals are connected).
Furthermore, the donor host may have originally In wild animals, contact structure can vary season-
become infected by contamination from domestic ally due to birth pulses, seasonal resource patterns,
animals; thus, there are often complex spillover– or weather-related behavior such as hibernation
spillback dynamics that can involve several do- (i.e., dynamic rather than static contact structures
mestic and wild host species. in which the frequency of contacts do not change
through time). Wildlife contact structure is typic-
Successful Establishment and Ongoing ally heterogeneous across multiple scales due to
Transmission spatial structure and movement behavior, resource
distribution, and social relationships (Sah et al.,
The average number of transmissions from a 2018). In contrast, domestic animal populations
single infectious host in a completely susceptible can be very dense and well-mixed at the farm level
population, referred to as R0, and the per-capita but can demonstrate heterogeneity in contact at
rate at which susceptible individuals become in- larger geographic scales due to shipment patterns,
fected, termed the force of infection (FOI), are marketing of domestic animals, and seasonal pro-
useful quantities for understanding, predicting, and duction practices (Gorsich et al., 2016, 2019). These
managing epidemiological dynamics (for an exten- differences in contact structure between domestic
sive review of FOI and R0, see Vynnycky and White, animal populations and wild animals can result
2010; Keeling and Rohani, 2011). Estimates of R0 in very different epidemiology, even for the same
can be used for predicting pathogen establishment; pathogen. Outbreaks in well-mixed populations are
R0 values 1 predict ongoing transmis- outbreaks relative to those in populations with het-
sion. Similarly, estimates of FOI describe infection erogeneous contact (Keeling, 1999; Bansal et al.,
risk for susceptible individuals, and can predict epi- 2007). The interaction between individual host
demic severity through time. R0 is determined by 3 movement behavior, population demographics, en-
components: 1) the probability of infection given vironmental conditions, and infection-induced be-
contact between a susceptible and infectious indi- havioral changes can result in significant changes in
vidual, 2) the average rate of contact between sus- disease dynamics (White et al., 2018). Because host
ceptible and infectious individuals (where 1 and ecology can have such dramatic impacts on disease
2 together describe the “transmission rate”), and dynamics, understanding these components is cru-
3) the duration of infectiousness. The transmis- cial for predicting and managing disease introduc-
sion rate as well as the current number or propor- tions into domestic animal populations (Plowright
tion of infectious individuals determines FOI (i.e., et al., 2017).
for density- or frequency-dependent transmission,
respectively). Thus, host ecology such as demo- APPLICATION OF DISEASE
graphic dynamics, movement and spatial structure, TRANSMISSION MODELS TO MANAGE
social interactions, and physiological condition, as DISEASE INTRODUCTIONS IN
well as pathogen characteristics are important de- DOMESTIC ANIMALS
terminants of R0 and FOI because they ultimately
determine transmission rates. Analytical tools such as disease transmission
models used to model the dynamics of infectious
The Role of Ecological Processes in diseases leverage a well-established and expanding
Ongoing Transmission body of disease transmission theory to con-
struct representations of epidemiological systems.
The frequency and variation of contacts among Disease transmission models provide a means for
infected and susceptible hosts (i.e., contact struc- understanding how multiple, nonlinear processes
ture) (Fig. 1B) have important consequences for such as host demographic dynamics, movement,
transmission rates (Keeling, 1999, 2005; Bansal contact, and host-to-host pathogen transmission
et al., 2007; Sah et al., 2018). Variation in contact determine outbreak probability and severity in a
structure affects the probability that a pathogen will target host species. Disease transmission theory
become established as well as outbreak size (Lloyd- has shown that 3 quantities determine the initial
Smith et al., 2005). Contact structure can be decom- introduction and ongoing transmission dynamics
posed into 2 primary components, 1) the rates that in a naïve host species (recipient): prevalence inApplying models in disease management 2295
the donor host population, contact rate between et al., 2018) but do not have to be spatial (e.g.,
the donor and recipient hosts, and probability of Pepin and VerCauteren, 2016). Nondynamical
infection given contact (Lloyd-Smith et al., 2009). risk models correlate predictors to case data (in
Together, these 3 components define the introduc- donor or recipient host populations) to make pre-
tion force of infection for a pathogen that is a direct dictions about potential risk in the recipient host
measure of infection risk to recipient host popula- population (e.g., Belkhiria et al., 2016) or predict
tions. Understanding the dynamics of introduction risk probabilistically using conditional probability
force of infection has provided valuable insight and historical surveillance patterns (Faverjon et al.,
toward risk assessment, prevention, and response 2015; Fountain et al., 2018). Nondynamical risk
planning of livestock diseases (see below), but re- models have dominated the literature when as-
mains an underused tool (Lloyd-Smith et al., 2009). sessing risks of transboundary introduction by lat-
Applications of disease transmission ecology typic- eral transfer. They are appealing because they allow
ally have considered disease transmission processes for a multitude of introduction mechanisms to be
in either the donor or recipient host populations, compared against each other to identify the most
because dynamics at the interface of donor and likely pathway of introduction and quantify overall
recipient populations is complex such that even introduction risk. They frequently use expert elicit-
conceptual development remains in its infancy ation approaches or preexisting data sources, and
(Plowright et al., 2017). Below we describe potential are often performed in user-friendly software such
applications of disease ecology for informing man- as Microsoft Excel (e.g., Miller et al., 2015). These
agement of disease in domestic and wild animals. features allow for rapid quantification in emergen-
cies or when data are limited making them readily
Risk Assessment accessible across disciplines. Also, their structure
of quantifying risk pathways through a series of
Risk analysis is an often broadly used term conditional probabilities is appealing because it al-
referring to risk characterization, communica- lows the propagation of parameter uncertainty and
tion, and management, that provides support for forms a process-based chain of events that lead to
decision-making and is frequently used in policy introduction. Whether applied to transboundary or
development (Suter, 2016). For animal disease, within country introduction these approaches serve
risk analysis is an important process used to iden- as a standard approach that is repeatable and trans-
tify and characterize potential risks posed by im- parent providing an important link with stand-
plementation of a specific policy or event such as ards of the World Organisation for Animal Health
importation or movement of domestic animals (OIE) (Murray, 2004; Sugiura and Murray, 2011).
(Sugiura and Murray, 2011). Thus, risk analyses More recently, conceptualizing disease introduc-
are foundational for the development of animal tions through a series of conditional probabilities,
health policy (Miller et al., 2013). The application such as the OIE framework, has also been proposed
of quantitative risk assessment models to predict for examining cross-species disease transmission
lateral transfer (Fig. 1A) and cross-species transfer using dynamical models (Plowright et al., 2017).
(i.e., outbreak dynamics) (Fig. 1B) is typically con- Limitations in assessing risk using nondynamical
ducted independently. Disease risk in a recipient models are that risk of lateral transfer is typic-
population is a function of both disease dynamics ally based on historical data, and many of these
in the donor population and recipient populations approaches do not consider spatiotemporal
(see ecological processes governing disease emer- heterogeneities. Using simulations, Enright and
gence). Quantitative risk assessment models in a O’Hare (2017) have emphasized the importance
recipient host population can be broadly classi- of temporal dynamics in accurately capturing risk.
fied into 2 types: dynamical and nondynamical. They showed that ignoring temporal dynamics in
Dynamical models of risk represent time-varying animal movement can lead to overestimation of
disease transmission processes in host popula- predicted outbreak size and nonoptimal response
tions to infer transmission parameters related to plans. Additionally, reliance solely on historical
force of infection using case data in recipient host data can be misleading as ecological conditions
populations (e.g., Bonney et al., 2018), or to predict change. Ignoring ongoing or seasonal dynamics in
outbreak dynamics from data of component pro- the donor host population could cause erroneous
cesses (e.g., Fournié et al., 2013). These approaches predictions of risk. In contrast, dynamical models
are frequently implemented in a spatially explicit can address these limitations because they inher-
context (e.g., Buhnerkempe et al., 2014; Bonney ently incorporate temporal changes and are readily2296 Miller and Pepin
amenable to explicit representation of space. For uncertainty can require significant analytical ef-
example, in a dynamical model, domestic animal fort to explore parameter sensitivity, understand
movement networks can be represented through whether processes are accurately portrayed, and
time (Fournié et al., 2013; Buhnerkempe et al., 2014) examine consistency in parameter estimation or pre-
to examine seasonality in risk and understand how diction (see Cross et al., 2019 for data and modeling
particular changes in either the donor or recipient challenges). Despite these challenges dynamical
host population affect outbreak probability or se- models can provide the most accurate portrayal of
verity (Buhnerkempe et al., 2014; Sokolow et al., risk across space and time because they can expli-
2019). These types of analyses are not as powerful citly account for changing nonlinear processes.
in nondynamical models because they do not cap-
ture how nonlinear processes interact. Planning Response to Outbreaks
Dynamical models are typically used in 2 dif-
ferent ways: prediction from data on parameters Dynamical models have been used to explore
(Halasa et al., 2016; Merkle et al., 2018) or esti- optimal response plans for domestic animal dis-
mation of epidemiological parameters by fitting eases. Buhnerkempe et al. (2014) assimilated move-
the model to outbreak data (Bonney et al., 2018; ment data for cattle shipments within the United
Hayer et al., 2018) and have rarely been used to do States and used a dynamical model to show that
both. One example where both have been success- local movement restrictions might be more effective
fully implemented are models developed by Hobbs at controlling an introduction of foot-and-mouth
et al. (2015) to support adaptive management of disease virus relative to state or national-scale
an ongoing outbreak of brucellosis in wild bison. movement bans. Similarly, Roche et al. (2015) used
They used extensive historical data describing 5 different dynamical models to evaluate the ef-
population dynamics as well as information on fectiveness of different vaccination strategies for
contact structure and disease prevalence through foot-and-mouth disease control and found that for
time. Integrating historical data with current data all models vaccination led to a significant reduction
they developed an iterative method to evaluate the in predicted epidemic size and duration compared
probability of success for alternative management to the “stamping-out” strategy alone. These re-
actions as well as estimating epidemiologically im- sults emerge from consideration of the interaction
portant parameters such as R0 and FOI along with of dynamic host populations and epidemiological
the changes in these parameters as a result of pre- processes through time. In nondynamical models,
vious and current management decisions. disease dynamics in donor host populations are not
In addition to quantifying risk, dynamical represented explicitly, which neglects assessment
models allow an understanding of how different of response plans that aim to limit transmission in
components of disease transmission affect risk donor host populations (Ebinger et al., 2011).
metrics, which in turn allows for process-based In diseases that have wildlife reservoirs, where
planning of outbreak response (Pepin et al., 2014). spillover–spillback dynamics can lead to disease
By targeting processes that determine risk rather persistence, dynamical models have shown that the
than consequential patterns, response plans can be ecology of both the donor and recipient host popu-
robust to changes in the underlying ecology driving lations need to be considered for optimal control
disease transmission. A significant challenge of (Cowled et al., 2012), and that the optimal con-
dynamical modeling approaches is they are often trol strategy may involve mitigation in both the
technically complex to develop and implement, donor and recipient host populations (Ward et al.,
which may limit their use and interpretation across 2015). However, the type of optimal control strat-
disciplines in animal health management (Manlove egies employed in donor and recipient populations
et al., 2016). Because of their analytical complexity, may differ as a function of host ecology and viral
dynamical models can also be computationally and characteristics (Manlove et al., 2019). In wild pigs,
time-intensive, which has further limited their use contact structure can be fragmented (Pepin et al.,
for rapid risk assessment when new threats are per- 2016), such that viruses causing acute infections
ceived. Also, while nondynamical models can rely are not sustained (Pepin and VerCauteren, 2016).
on expert opinion, dynamical models need appro- Optimal control strategies in the donor population
priate data on a variety of processes such as animal can thus differ substantially based on the com-
movement, disease prevalence, and host densities bined effects of infectious period of the virus and
(Merkle et al., 2018). Appropriately formulating the contact structure of the donor host (Pepin and
the model and accounting for multiple sources of VerCauteren, 2016), which changes both the riskApplying models in disease management 2297
landscape and optimal control strategies of spill- be greatest. Gonzales et al. (2014) developed a dy-
over in space and time. A similar result arises from namical model that accounts for within-flock trans-
considering landscape heterogeneity in space and mission as well as the spatial location of flocks and
time—where consideration of the contact rates between flock transmission. The model predicted
in space is important for determining optimal re- transmission risk across space, producing a tar-
sponse plans (LaHue et al., 2016). geted risk-based surveillance strategy that allowed
Additionally, accounting for these for early detection of low pathogenicity avian influ-
heterogeneities as well as uncertainties in successful enza in domestic chicken flocks in Denmark. Their
management has only recently been addressed. model allowed for a dynamic evaluation of ef-
Hobbs et al. (2015) found that accounting for un- fective sampling frequency that optimizes resource
certainty in the ability to implement management to allocation, but is seldom included in conventional
control an ongoing outbreak of brucellosis in wild methods of surveillance design. Additionally, be-
bison, elk, and cattle after accounting transmission cause this approach dynamically evaluates changes
heterogeneities dramatically influenced the prob- in seroprevalence during an outbreak, it can provide
ability of achieving disease control goals. A major insight into changes in transmission risk factors,
gap in the use of dynamical models for response and evaluation of control measures such as vaccin-
planning is a lack of applying these approaches ation. Similarly, in the United States, an adaptive
in a “learning by doing” framework—where the targeted risk-based approach has been used to al-
models are used to predict optimal strategies, then locate surveillance for avian influenza in wild birds
the predicted strategies are implemented, and data (APHIS, 2016) and pathogens of interest in feral
for assessing effectiveness are collected and used to swine (APHIS, 2017). In both cases previous sur-
validate and refine the models (Restif et al., 2012). veillance data were used to determine uncertainty
Incorporating economic principles in dynamical in risk. Surveillance resources were then reallocated
models for evaluating alternative response strat- annually to prioritize greatest risk areas and those
egies is a second gap that is only rarely addressed. with the greatest uncertainty in risk. This adaptive
Economic assessments using dynamical models approach to surveillance allocation was intended to
have typically used the model predictions as in- reduce uncertainty in risk predictions and improve
puts into economic models in post hoc analyses allocation of surveillance through time.
(Thompson et al., 2018, 2019). The application of dynamical models to guide
surveillance planning is relatively new. One limita-
Surveillance Design tion of existing approaches using dynamical models
is that risk-based targeting is often done as a post
Analytical surveillance design has overwhelm- hoc analysis using the predictions of disease spread
ingly been based on sample size statistics (Herzog or contact and movement of at-risk animals from
et al., 2017) or risk-based ranking approaches a dynamical model (Gorsich et al., 2018). While
(Stärk et al., 2006). Because surveillance resources useful, this limits the utility when risk factors im-
(before an emergency) are often limited compared portant for introduction or spread of a pathogen
with response resources (during an emergency) effi- change seasonally and from year-to-year (Walton
cient surveillance plans are crucial. In other words, et al., 2016). Examples of changing introduction
surveillance needs to be “risk-based” and favor risks are seasonal differences in domestic animal
“early detection” (Stärk et al., 2006; Comin et al., shipment, changes in demand of animal products
2012). Dynamical models have potential to inform and live animals, or changes in global movement of
effective risk-based or early-detection surveillance people among countries. The flow of people, ani-
plans because they can concurrently evaluate how mals, and products can change dramatically across
implementation of surveillance and response ap- time and space. For example, Jurado et al. (2018)
proaches affect outbreak severity (Comin et al., found that risks associated with ASFv introduc-
2012), but dynamical models remain underused tion changed seasonally and varied spatially among
(Herzog et al., 2017). years due to changes in frequency of airline travel
Accounting for transmission processes in a among different airports in countries with and
spatially explicit framework is especially useful for without ASFv. Dynamical models mechanistically
determining optimal surveillance strategies (i.e., represent host-pathogen ecology allowing nonlinear
who, when, where, and how much) because they relationships among risk factors to be included ex-
can help target surveillance to species, locations, plicitly meaning that changes in risk factors (e.g.,
and times where transmission risk is expected to changes in movement of airline travelers, shipment2298 Miller and Pepin
patterns of domestic animals, or movement of wild potential risks of introduction and transmission
birds) through time can allow for time-varying al- (Miller et al., 2017). Dynamical models can be used
location of surveillance effort to optimize detection to evaluate potential risks and consequences posed
in response to shifting locations of greatest risk among many pathogens allowing limited surveil-
(Leslie et al., 2014; Walton et al., 2016). lance resources to be allocated to those with the
A frequent objective of disease surveillance ac- greatest potential risks and consequences within
tivities at the wildlife–domestic animal interface is the populations being managed.
monitoring changes in risk to domestic animals or
effectiveness of risk reduction mitigations (Morner THE WAY FORWARD FOR INTEGRATION
et al., 2002; Hoinville et al., 2013). Dynamical
OF DISEASE-DYNAMIC MODELS IN
models have frequently been used to determine
MANAGEMENT
the transmission of pathogens among species in
multi-host disease systems (Craft et al., 2008). Management and policy decisions are con-
Despite the established theory and application to tinually made to mitigate disease introduction and
understand disease transmission among species, transmission risks. It is broadly recognized across
dynamical models have rarely been used to de- a diversity of domains that science-based manage-
termine surveillance in both donor and recipient ment, sometimes referred to as data-driven man-
populations concurrently (Shriner et al., 2016). As agement, leads to more effective decisions but is
cross-species transfer depends on disease dynamic challenging because it requires making the synthesis
conditions in both the donor and recipient popu- of data more accessible and relevant to policy deci-
lations (Lloyd-Smith et al., 2009), surveillance of sions (Gregory et al., 2012; Williams and Hooten,
only a single component could fail to distinguish 2016; Dietze et al., 2018). Additionally, scientific
differences in the magnitude of risk across space or data are less valuable to decision-makers when
through time. For example, analysis of a low patho- there is considerable uncertainty or complexity.
genic avian influenza outbreak found that when the Integration of science in decision-making is further
network of poultry producer relationships was ex- complicated because policy decisions and science
plicitly included, the location of the index case (i.e., frequently have different timelines, incentives, and
location of introduction) strongly effected both stakeholders, which can hamper efficient integra-
outbreak probability and size (G. Gellner, United tion of science into the decision process (Funtowicz
States Department of Agriculture, unpublished and Strand, 2007).
data). Similarly, a continental scale analysis of foot Synthesizing science to improve its usefulness
and mouth disease that explicitly accounts for spa- in disease management requires monitoring data
tial differences in cattle shipment and density found intended to understand factors and processes that
that the duration and size of outbreaks was de- drive disease introductions and ongoing trans-
pendent on which local population the index case mission. Improving the understanding of disease
first occurred (Buhnerkempe et al., 2014). This in- processes that drive pathogen introduction and
dicates that disease risk in the recipient population transmission is fundamentally essential and ultim-
is a function of both disease dynamics in the donor ately leads to better, more efficient policy decisions
and recipient populations. (e.g., biosecurity or response planning). Thus, pri-
When wildlife are a potential donor host species, oritizing “learning” in the adaptive management
disease-dynamical models can be used to optimize cycle (Fig. 3) can be a very important part of man-
when and where to conduct surveillance in both agement and the science intended to support man-
wild and domestic animals to improve risk moni- agement decisions.
toring in wildlife and early identification of intro-
duction events into domestic animals. Conversely, Adaptive Management
when domestic animals are the donor, the utility
and effectiveness of monitoring for potential spill- Science-based disease management requires
over from domestic animals to wildlife can be evalu- a fundamental shift from simply monitoring to
ated, which has been identified as a critical need for surveillance—i.e., using monitoring data to pre-
control and eradication of chronic diseases such dict changes in disease risk and management
as bovine tuberculosis and pneumonia (Miller and effectiveness, and decide/perform management ac-
Sweeney, 2013; Besser et al., 2013). Additionally, tions that mitigate risks. Disease-dynamic models
the selection of pathogens to conduct surveil- provide an analytical framework for both under-
lance in wildlife is frequently not representative of standing and predicting disease processes in aApplying models in disease management 2299
management context using monitoring data. The ASFv can be through direct contact with infected
adaptive management framework then provides a pigs, indirect contact through fomites, or through
method for integrating disease data using disease- soft tick species in the genus Ornithodoros (Mellor
dynamic models to iteratively reduce uncertainty et al., 1987; Penrith and Vosloo, 2009; Guinat et al.,
in decision-making over time resulting in improved 2016). The virus is highly resistant to inactivation
decisions and outcomes (Allen et al., 2011) (Fig. 3). and can remain viable in the environment for many
The adaptive management cycle is typic- days and in undercooked or cured pork products
ally broken into 2 processes. First, a structured for at least 4 months (Plowright et al., 1969; Farez
decision-making process (Fig. 3A) formalizes the and Morley, 1997). The high stability of ASFv in
definition of the problem, objectives, evaluation addition to multiple potential routes of transmis-
of decision trade-offs, and results in a current op- sion that include vectors, fomites, and direct trans-
timal management decision. The second process is mission has made ASFv particularly difficult to
characterized as learning. This process represents control in populations and a significant concern
the management decision implementation, moni- globally (Sánchez-Cordón et al., 2018).
toring of the system and evaluation of progress to- Introduction of ASFv into the western hemi-
ward management objectives and any adjustment sphere could threaten food security and have a
to management decisions through time that im- large economic impact resulting from the presence
prove progress toward objectives. In disease man- of sympatric vector species and susceptible feral
agement applications, adaptive management is a and domestic swine host populations (Brown and
method for integrating surveillance data and ana- Bevins, 2018) (Fig. 2D–F). The Americas account
lysis of disease management iteratively through for 58% of total global pork exports, 83% of global
time to allow learning through disease manage- imports and exports of live domestic swine, and
ment actions (Fig. 3B). Through formal analyses have the third largest standing inventory of do-
of uncertainty, disease surveillance can be guided mestic swine (USDA, 2018). The predicted eco-
to improve learning about the most important fac- nomic impacts resulting from an ASFv outbreak
tors affecting risk or management effectiveness in in North America are more than US$4.25 bil-
order to optimize information for decision-making. lion, with a cost-benefit ratio of ASFv prevention
Adaptive management has been suggested as an ap- programs of more than US$450 billion (Rendleman
proach to manage disease (Miller et al., 2013; Webb and Spinelli, 1999) making ASFv a threat to
et al., 2017), allocate disease surveillance (Gonzales the global economy (Sánchez-Cordón et al., 2018).
et al., 2014), and improve disease interventions Contaminated animal-derived products have been
(Merl et al., 2009; Shea et al., 2014) but has rarely identified as one of the greatest risks for the entry
been formally implemented to manage a disease of ASFv and other foreign animal diseases (FADs)
system. (Mur et al., 2012). Specific to ASFv the annual
probability of transboundary introduction varies
ASFv as an Example through time and by pathway. Introduction via
swine products carried by travelers or through im-
Recently emerged as a significant threat to portation is considered more likely than import-
domestic swine production globally, ASFv is cur- ation of infected live domestic swine (Herrera-Ibata
rently present in many regions of Africa, Europe, et al., 2017; Jurado et al., 2018). If introduced into
and Asia (Fig. 2A). Currently, ASFv is reportable the western hemisphere the risk of establishment
to the OIE, and transboundary introduction into is expected to vary geographically by the pathway
a country free of the disease can have severe eco- of introduction and the presence of wild and do-
nomic consequences resulting from production mestic swine (Herrera-Ibata et al., 2017; Jurado
losses, loss of export markets, and eradication et al., 2018). Currently available risk estimates for
programs. Species in the Suidae family are suscep- transboundary introduction are complicated by
tible to infection with ASFv (Penrith and Vosloo, multiple pathways of introduction via legal and il-
2009). Infection with ASFv in most Suidae, par- legal routes (e.g., illegal importation of swine prod-
ticularly domestic swine, typically results in high ucts), the presence of competent soft tick vectors,
mortality; however, once established in a popu- and the presence of both feral and domestic swine
lation, the disease can manifest as a subacute (Sánchez-Cordón et al., 2018). Dynamical models
clinical form that can be sustained in the popula- offer a potential tool to account for the complex
tion (Gallardo et al., 2015; Nurmoja et al., 2017; geographic and temporal risks resulting from the
Sánchez-Cordón et al., 2018). Transmission of distribution of vector species, densities of feral2300 Miller and Pepin
Figure 2. Current distribution of African swine fever virus (ASFv) and some risk factors that contribute to lateral and cross-species transfer.
Panel A depicts countries currently reporting ASFv in domestic or wild pigs to OIE World Animal Health Information System (WAHIS, 2019) or
the European Animal Disease Notification System (ADNS, 2019). Right side of panel A is the change in countries reporting ASFv through time.
Color scale indicates the last year ASFv cases were reported in the country while gray indicates ASFv has never been officially reported in the
country. Panels B and C describe 2 routes of transboundary lateral transfer into the Americas that include legal imports of swine products (panel B)
(UN Comtrade, 2018) and transport of swine products via air travelers (panel C) (Patokallio, 2018). Gray lines indicate the annual average amount
(kg) of direct shipments and the annual average frequency of airline flights from 2007 to 2018 from countries currently reporting ASFv cases. Panels
D, E, and F describe the global distribution of 7 Ornithodoros ticks (O. moubata, O. hermsi, O. parkeri, O. talaje, O. turicata, O. sonrai, O. tholozani)
(WHO, 1989; Teglas et al., 2005, 2006; Donaldson et al., 2016; Lopez et al., 2016; Sage et al., 2017; ECDC and EFSA, 2018), density of wild suid
species (Lewis et al., 2017), and density of domestic swine (Gilbert et al., 2018).
and domestic swine, and differing routes of lateral different pathways of lateral and cross-species
and cross-species transfer that are expected to vary transfer through time resulting in earlier detection
through time. of ASFv introduction by targeting surveillance of
swine populations most likely to be exposed to con-
Adaptive Management of ASFv taminated materials (Figs. 1 and 2). The resulting
ASFv surveillance data (e.g., Fig. 2) can then be
New populations continue to be invaded by integrated, using dynamical models, with previous
ASFv through multiple pathways yet the specific monitoring data and near-term host population
pathways of greatest importance appear to vary data (e.g., density, movement, and contact) to pro-
among regions and there is uncertainty about how vide new predictions of ASFv introduction risks,
best to mitigate location-specific risks. The concep- spread, and consequences. This provides a method
tual approach presented in Fig. 3 offers an oppor- of integrating analyses assessing lateral and
tunity to address these challenges because analyses cross-species transfer that are typically conducted
and associated risk predictions can be continually separately. Through time this dynamical modeling
updated using newly available data allowing alloca- allows a method of synthesizing across the entire
tion of surveillance resources to specific pathways, system to improve and optimize surveillance guid-
locations, or animal populations (wild or domestic) ance (adaptive surveillance) that best mitigates
to be varied and improved through time. For ex- changes in risk resulting in improved understanding
ample, frequently updating analyses using new data of risks (Fig. 3B). Similarly, mitigations to reduce
can improve allocation of surveillance effort among introduction risks such as placing limitations onApplying models in disease management 2301
Figure 3. Conceptual relationship between iterative disease predictions and adaptive decision-making. Panel A describes a generalized depiction
of the adaptive management cycle that includes both structured decision-making and learning processes (Allen et al., 2011). Panel B describes a
conceptual approach for integrating disease modeling into the decision-making process. Iterative prediction using disease-dynamic models (blue
oval) integrates data describing risk factors to make predictions of introduction risk, spread, and associated consequences. Using these predictions
optimal risk-based targeted surveillance strategies to detect introductions, risk mitigations to reduce introduction and spread, and optimal control
options are determined. Surveillance guidance informs (adaptive surveillance) decisions concerning allocation of surveillance (i.e., when, where,
and how much). Optimal risk mitigations and potential control options inform adaptive planning and preparedness decisions. New surveillance
data are then integrated with new data describing changes in risk using the iterative prediction cycle to provide updated and improved guidance
for surveillance, risk mitigations, and control options. Thus, learning has occurred and better information is available to support decision-making.
importation of products most risky for ASFv or sensitivity of decision-making to forecast uncer-
refined targeting of inspections of imported prod- tainty can be used to identify how better model
ucts or airline travelers from ASFv regions can be predictions would improve decision trade-offs and
iteratively evaluated and optimized through time. when the model is adequate to meet decision needs
Uncertainty analysis can identify which ASFv (Dietze et al., 2018).
surveillance streams or surveyed populations are Changes in ASFv introduction risks and im-
most critical for reducing uncertainty in predic- proved knowledge of introduction risks gained
tions. Value-of-information analysis can aid in through iterative adaptive surveillance directly in-
the prioritization of ASFv surveillance streams or fluence predictions of disease spread, consequences,
populations (wild or domestic) to be monitored by and optimal control options. Correspondingly,
providing a means to evaluate the “return on in- ASFv control options can be improved as fun-
vestment” provided by the allocation of resources damental disease drivers in a region change (e.g.,
in each surveillance data stream. Additionally, the changes in density of wild and domestic swine) or2302 Miller and Pepin
as the types, volume, and origin of imported swine Updating predictions and forecasts iteratively
products change. Predicted ASFv optimal control requires data that can be difficult to collect and can
strategies can be used in an adaptive planning pro- be labor-intensive. One of the largest challenges is
cess that improves the potential alternatives con- the need to collect multiple types of data—data
sidered. For ASFv this could include proactive describing disease occurrence, changes in wild
population reduction of wild swine, which is cur- and domestic animal populations (occurrence and
rently being implemented in some European coun- population density), changes in population con-
tries, or greater targeted removal of invasive feral tact (whether via products, humans, or directly),
swine in countries with active population control and changes in other epidemiologically important
programs. Additionally, changes in predicted intro- processes such as vector distribution and occur-
duction and spread of ASFv can be used to itera- rence (e.g., Fig. 3B). Some of these data, such as
tively update guidance on the predicted culling or monitoring of domestic animal populations, al-
biosecurity practices required to limit spread if ready have systems in place to collect and main-
introduced. This information can be used by man- tain data. However, disease surveillance data within
agers to forecast resource needs and develop more wild animal populations or for common routes of
accurate and useful response plans that are more disease introduction are frequently not available,
dynamic rather than static. or are only available at broad spatial or temporal
In the event that a pathogen such as ASFv is scales that might not match the epidemiological
successfully introduced, predictions of optimal scales of interest, or do not align with epidemio-
disease control strategies serve as a starting point logical risks (Miller et al., 2013, 2017; Cross et al.,
from which to begin managing an outbreak and 2019). Surveillance data collection is often limited
monitoring its progress. “Learning by doing” can by the number of samples, spatial locations, or time
then be used to continually improve surveillance frames that samples can be collected. However,
designs and response plans. This could be particu- the conceptual approach we present here offers a
larly important for FADs such as ASFv that may framework to rigorously address these challenges
not have previously occurred in a country resulting by allowing for the evaluation, comparison, and
in no prior data describing disease dynamics. For identification of those data of greatest importance
example, ASFv has not previously been intro- for management decision trade-offs.
duced into North America but at least 5 experi- Additionally, the implementation of disease-
mentally competent tick vectors for ASFv occur dynamic models can be technically difficult and
and sympatric populations of wild and domestic time-consuming. As a result the development and
swine are present (Fig. 2D–F). An additional ad- application of these approaches has more frequently
vantage of using an iterative prediction cycle to been implemented in an academic environment
inform decision-making is that consequences of to address specific, narrowly focused policy deci-
introduction and spread can be explicitly included sions. An opportunity to increase the application
in models. This allows for evaluation of poten- of dynamical models within the decision-making
tial risk mitigations (either proactive or during an process is to focus on objective-driven models that
outbreak) to be evaluated using cost-benefit ap- provide practical predictions that are directly useful
proaches, providing practical guidance based on to those managing disease. Indeed, dynamic models
current resources. Uncertainties can be included have been used mechanistically for avian influenza
in the cost-benefit analysis allowing for improved (Malladi et al., 2012; Weaver et al., 2012; Bonney
understanding of which data may be needed to im- et al., 2018), brucellosis (Hobbs et al., 2015), and
prove consequence assessments. foot-and-mouth disease (Buhnerkempe et al., 2014;
Roche et al., 2015) providing practical objective-
Practical Challenges driven predictions to support disease management
decisions. Recognizing that modeling is a means to
Our conceptual approach for integrating improve disease management and outcomes is im-
disease-dynamic models directly within the portant and fosters increased use of these tools.
decision-making process using adaptive manage- This is most successful when these approaches
ment are not without challenges. And several key are implemented in a cross-disciplinary environ-
challenges need to be tackled for adaptive disease ment that includes scientists and decision-makers
management to be most successful, especially for representing both wildlife and domestic animal
disease systems that cross the wild–domestic animal health. Imbedding technical expertise within
interface. the decision-making process will likely ensureApplying models in disease management 2303
long-term success of decision processes that use future. J. Environ. Manage. 92:1339–1345. doi:10.1016/j.
model-based support. There is an additional op- jenvman.2010.11.019
APHIS. 2016. Surveillance plan for highly pathogenic avian
portunity to integrate economic principles such
influenza in wild migratory birds in the United States.
as value-of-information and cost-benefit analysis United States Department of Agriculture, Animal Plant
into the surveillance and monitoring decision pro- Health Inspection Service, Fort Collins, CO.
cess using the framework we presented. While the APHIS. 2017. Targeted antibody surveillane for national
adaptive management framework is often dis- diseases of concern in feral swine in the USA. United
cussed with regard to improving decisions through States Department of Agriculture, Animal Plant Health
Inspection Service, Fort Collins, CO.
learning it also offers opportunities to address these Bansal, S., B. T. Grenfell, and L. A. Meyers. 2007. When in-
other challenges by continually reassessing the ap- dividual behaviour matters: homogeneous and network
proaches used to support decision processes. models in epidemiology. J. R. Soc. Interface 4:879–891.
doi:10.1098/rsif.2007.1100
CONCLUSIONS Belkhiria, J., M. A. Alkhamis, and B. Martínez-López. 2016.
Application of species distribution modeling for avian
Using adaptive disease management with dy- influenza surveillance in the United States considering
namical models can support the development of the North America migratory flyways. Sci. Rep. 6:33161.
doi:10.1038/srep33161
optimal surveillance systems, risk mitigations, as Besser, T. E., E. F. Cassirer, M. A. Highland, P. Wolff, A.
well as disease preparedness and response activ- Justice-Allen, K. Mansfield, M. A. Davis, and W. Foreyt.
ities because this approach allows learning to occur 2013. Bighorn sheep pneumonia: sorting out the cause of
from the most current conditions. Further it allows a polymicrobial disease. Prev. Vet. Med. 108(2-3):85–93.
a method of synthesizing across and integrating Bonney, P. J., S. Malladi, G. J. Boender, J. T. Weaver, A.
Ssematimba, D. A. Halvorson, and C. J. Cardona. 2018.
analytical processes that are frequently conducted
Spatial transmission of H5N2 highly pathogenic avian
independently which can facilitate learning about influenza between Minnesota poultry premises during
the system being managed. This can be particu- the 2015 outbreak. PLoS One 13:e0204262. doi:10.1371/
larly important for FADs that frequently have journal.pone.0204262
limited data available that may or may not repre- Brown, V. R., and S. N. Bevins. 2018. A review of African
sent disease dynamics in recipient host populations. swine fever and the potential for introduction into the
United States and the possibility of subsequent estab-
While ASFv was highlighted as an example, the lishment in feral swine and native ticks. Front. Vet. Sci.
conceptual framework described could be applied 5:1–18. doi:10.3389/fvets.2018.00011
to other diseases such as classical swine fever or Buhnerkempe, M. G., M. J. Tildesley, T. Lindström, D. A.
avian influenza. This conceptual framework does Grear, K. Portacci, R. S. Miller, J. E. Lombard, M.
not have to be restricted to FADs and can pro- Werkman, M. J. Keeling, U. Wennergren, et al. 2014. The
impact of movements and animal density on continental
vide significant benefit for managing endemic dis-
scale cattle disease outbreaks in the United States. PLoS
eases as well. Using dynamical models within the One 9:e91724. doi:10.1371/journal.pone.0091724
decision-making process can foster the resilience Comin, A., A. Stegeman, S. Marangon, and D. Klinkenberg.
and flexibility needed to address the uncertainty as- 2012. Evaluating surveillance strategies for the early de-
sociated with disease decisions, thus improving the tection of low pathogenicity avian influenza infections.
ability to tackle inevitable changes and surprises PLoS One 7:e35956. doi:10.1371/journal.pone.0035956
Cowled, B. D., M. G. Garner, K. Negus, and M. P. Ward. 2012.
that arise. Controlling disease outbreaks in wildlife using limited
culling: modelling classical swine fever incursions in wild
ACKNOWLEDGMENTS pigs in Australia. Vet. Res. 43:3. doi:10.1186/1297-9716-43-3
Craft, M. E., P. L. Hawthorne, C. Packer, and A. P. Dobson.
We thank Dana Cole and Julianna Lenoch 2008. Dynamics of a multihost pathogen in a car-
for thoughtful comments on an early draft of the nivore community. J. Anim. Ecol. 77:1257–1264.
manuscript. We also thank Michael Runge, William doi:10.1111/j.1365-2656.2008.01410.x
Kendall, Amy Davis, David Wolfson, and others Cross, P. C., D. J. Prosser, A. M. Ramey, E. Hanks, and
K. M. Pepin. 2019. Confronting models with data: the
that have influenced our ideas related to adaptive
challenges of estimating disease spillover. Philos. Trans.
management. R. Soc. B. doi:10.1098/rstb.2018.0435
Dietze, M. C., A. Fox, L. M. Beck-Johnson, J. L. Betancourt,
M. B. Hooten, C. S. Jarnevich, T. H. Keitt, M. A. Kenney,
LITERATURE CITED C. M. Laney, and L. G. Larsen. 2018. Iterative near-
term ecological forecasting: needs, opportunities, and
ADNS. 2019. European animal disease notification system.
challenges. Proc. Natl. Acad. Sci. 115(7):1424–1432.
European Commission, Brussels, Belgium.
doi:10.1073/pnas.1710231115.
Allen, C. R., J. J. Fontaine, K. L. Pope, and A. S. Garmestani.
Donaldson, T. G., A. A. Pèrez de León, A. Y. Li, A. I.
2011. Adaptive management for a turbulent
Li, I. Castro-Arellano, E. Wozniak, W. K. Boyle, R.You can also read
