Journal of Health Monitoring - RKI
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
JUNE 2020 FEDERAL HEALTH REPORTING
SPECIAL ISSUE
3 JOINT SERVICE BY RKI AND DESTATIS
Journal of Health Monitoring
Evidence-based Public Health for Public Health Action – Proceedings of an international
workshop at the Robert Koch Institute
Berlin, Germany, 14 December 2018
1Journal of Health Monitoring Index
3 Editorial Providing actionable evidence in Public 23 P
roceedings Summary of World Café Discussions
Health – The 2018 international workshop on Table 3: Dissemination
evidence-based public health at the Robert Koch
Institute, Berlin
7 P
roceedings Emerging challenges in evidence-based
public health and how to address them
9 Proceedings Conceptual issues in relation to the
design, implementation and evaluation of interven-
tions
11 P
roceedings Taking stock of existing evidence and
closing evidence gaps – Reflections from the
National Institute for Health and Care Excellence
(NICE)
13 P
roceedings Novel methods for health intervention
research
15 P
roceedings Systematic reviews in public health:
Exploring challenges and potential solutions
17 Proceedings Evidence-based public health (EBPH)
health policy advising and information of the public
19 P
roceedings Experiences from the Department of
Infectious Disease Epidemiology at Robert Koch
Institute
21 P
roceedings Summary of World Café Discussions
Table 1: Assessment
22 P
roceedings Summary of World Café Discussions
Table 2: Evaluation
Journal
JournalofofHealth
HealthMonitoring
Monitoring2018
20203(XXX)
5(S3) 2Journal of Health Monitoring Evidence-based Public Health for Public Health Action – Proceedings of an international workshop at the RKI EDITORIAL
Journal of Health Monitoring · 2020 5(S3)
DOI 10.25646/6499
Providing actionable evidence in Public Health – The 2018
Robert Koch Institute, Berlin
international workshop on evidence-based public health at the
Christa Scheidt-Nave 1, Angela Fehr 2,
Sebastian Haller 3, Giselle Sarganas 1,
Robert Koch Institute, Berlin
Henriette Steppuhn 1, Julia Truthmann 4,
Thomas Harder 3 A one-day international workshop entitled ‘Evidence- Table 2 (EVALUATION) – Which methods, skills, and
1
obert Koch Institute, Berlin
R based Public Health for Public Health Action’ took place data does the RKI as a national public health institute need
Department of Epidemiology and Health Monitoring
2
Robert Koch Institute, Berlin
at the Robert Koch Institute (RKI) in Berlin on Decem- to perform evaluations of public health interventions?
Centre for International Health Protection ber 14, 2018. The workshop was organised by an inter- Table 3 (DISSEMINATION) – How can the RKI as a
3
Robert Koch Institute, Berlin
disciplinary RKI public health research team and aimed national public health institute facilitate dissemination of
Department of Infectious Disease Epidemiology
4
Formerly Robert Koch Institute, Berlin to (1) provide insight into current concepts and method results among public health stakeholders, and what could
Department of Epidemiology and Health Monitoring ological challenges in evidence-based public health be important steps to enhance that (e.g. Cochrane Public
Corresponding author
(EBPH), and (2) identify next steps in enhancing collab- Health Research network, institutional repositories)?
Dr Christa Scheidt-Nave orations on EBPH research and practice within the RKI The workshop was open to RKI staff from all units. A
Robert Koch Institute and with external partners at the national and interna- total of 66 persons participated including invited speakers.
Department of Epidemiology and Health Monitoring
General-Pape-Straße 62–66 tional level. The workshop program is available on the publication server
12101 Berlin, Germany The workshop consisted of two parts. The first part com- of the RKI.
E-mail: Scheidt-NaveC@rki.de
prised a series of invited talks given by experts in the field
Submitted: 15.01.2020 of EBPH from Germany and the United Kingdom (UK). The Part 1: Invited talks
Accepted: 20.01.2020
second interactive part was devoted to group discussions. In their introductory presentations Mark Petticrew,
Published: 04.06.2020
Applying the world café method, participants were asked Department of Social and Environmental Health
Conflicts of interest to discuss approaches to strengthen EBPH at the RKI. Research at the London School of Hygiene and Tropical
The authors declared no conflicts of interest.
There was time for group discussions at three tables in two Medicine, and Eva A. Rehfuess, Institute for Medical
Funding twenty minute rounds. Inspired by the public health action Information Processing, Biometry, and Epidemiology at
The workshop Evidence-based Public Health for Public
Health Action was funded by the Robert Koch Institute.
cycle three key questions were assigned to the table hosts the Pettenkofer School of Public Health, Ludwig Maxi-
in order to stimulate and guide discussions: milian University of Munich, outlined current concepts
and methodological challenges in EBPH. In particular,
Table 1 (ASSESSMENT) – Which tools/methods for sys- both speakers highlighted the fact that public health
This work is licensed under a tematic evidence assessment does the RKI as a national interventions always imply changes in complex systems.
Creative Commons Attribution 4.0 public health institute need to identify and prioritize public With a special focus on public health interventions tar-
International License. health topics? geting the prevention of non-communicable diseases,
Journal of Health Monitoring 2020 5(S3) 3Journal of Health Monitoring Evidence-based Public Health for Public Health Action – Proceedings of an international workshop at the RKI EDITORIAL
Marc Petticrew emphasized the need to study not only Till Bärnighausen, Heidelberg Institute of Global Health
what is easy to measure, e.g. individual health behaviour, (HIGH), University of Heidelberg, introduced innovative
but also the upstream causes. These include personal, methods in population-based implementation and evalu-
societal and economic context factors, such as health per- ation research including regression-discontinuity and
ceptions or market forces. Understanding the complex fixed-effect models. He demonstrated that quasi-experi-
determinants of non-communicable diseases is essential mental study designs play a key role in public health inter-
to provide evidence for the implementation and evalua- vention research, in particular when conduct of RCTs is
tion of effective public health interventions across very precluded for ethical and methodological reasons. If care-
different contexts. Eva A. Rehfuess illustrated the impor- fully designed and appropriately applied to a specific
tance of contextual factors by providing examples of inter- research question, quasi-experimental methods minimize
ventions that proved to be beneficial in one context, but risk of bias and hence provide high level evidence regard-
harmful in another. Thus, deriving evidence from rand- ing the effectiveness of public health measures.
omized controlled trials (RCTs) may result in seriously Stefan Lhachimi, Institute for Public Health and Nurs-
misleading results if contextual factors are not taken into ing Research (IPP), University of Bremen, discussed some
account. Logic models can be used as a graphical tool for of the challenges specific to the conduct of systematic
mapping contextual factors relevant to the design and reviews and meta-analyses on public health interventions.
evaluation of public health interventions. Several concep- He emphasized the need for improving the quality of the
tual frameworks are available to guide the process from evidence basis at the level of primary studies, in particular
evidence synthesis to decision-making in public health. with regard to reducing risk of bias, standardizing defini-
Kay Nolan, Centre for Guidelines at the National Insti- tions for outcomes and interventions, and considering that
tute for Health and Care Excellence (NICE) in Manchester/ study results may greatly vary according to the specific
UK, shared her expertise in EBPH guideline development. study contexts. He also discussed ongoing work to improve
She illustrated two of the NICE core principles. First, given methods in synthesizing the evidence on public health
the complexity of public health problems, it is inevitable to intervention, in order to ensure timely and actionable infor-
systematically search for the best available evidence. This mation for health policy planning and implementation.
requires considering information across the whole spec- Manfred Wildner, Department of Health in the Bavarian
trum of evidence levels. Secondly, identifying evidence gaps Health and Food Safety Authority, Oberschleißheim, and
and areas of uncertainty is a central part of NICE EBPH Pettenkofer School of Public Health, Ludwig Maximilian
guideline recommendations. This will help to guide University of Munich, highlighted the need for a sustain
research priority setting, in order to continuously improve able translation network in EBPH. Based on his longstand-
the evidence base. These principles have the potential to ing experience at the interface of public health policy,
guide the next steps to strengthen EBPH at the RKI. research and practice, he emphasized that an ongoing,
Journal of Health Monitoring 2020 5(S3) 4Journal of Health Monitoring Evidence-based Public Health for Public Health Action – Proceedings of an international workshop at the RKI EDITORIAL
structured and open discourse involving policy, research Table 1 (ASESSMENT) – Which tools/methods for sys-
and practice should be guided by the shared responsibility tematic evidence assessment does the RKI as a national
to provide the best available evidence to health policy mak- public health institute need to identify and prioritize public
ing, to support implementation and evaluation research, health topics?
and to inform the public. In addition to rating evidence in public health, future
Thomas Harder, Department of Infectious Disease Epi- work should focus on health gap analyses and priority set-
demiology, Robert Koch Institute, Berlin, illustrated two ting, in order to generate the evidence that is presently
examples of EBPH research and practice at the RKI. The most needed and actionable. This will also help to make
RKI took the lead in the PRECEPT project (Project on a efficient use of time and personnel resources. Systematic
Framework for Rating Evidence in Public Health). In this approaches to health gap analyses including quantitative
project, an international and multidisciplinary research as well as qualitative methods, such as discourse analysis
team developed and successfully implemented a concep- may be necessary to achieve this goal.
tual framework for rating evidence in public health with Table 2 (EVALUATION) – Which methods, skills, and
focus on the prevention and control of communicable dis- data does the RKI as a national public health institute need
eases. The project was funded by the European Center for to perform evaluations of public health interventions?
Disease Prevention and Control (ECDC). The German There is a need to strengthen and continuously develop
Standing Committee on Vaccination (STIKO) provides and EBPH research methods to generate evidence in public
continuously updates recommendations on vaccinations health. This includes the use of innovative methods to
in accordance with the German Protection against Infec- abbreviate systematic literature reviews and evidence syn-
tion Act (IfSG). These recommendations are based on sys- thesis (e.g. overviews of reviews) as well as the application
tematic reviews of the biomedical literature which are con- of methods for health impact assessment, in order to aid
ducted by the STIKO Executive Secretariat at RKI. Recom- health policy planning and decision-making. In addition,
mendations serve to inform the public, to advise federal quasi-experimental study designs could be included in the
health authorities on vaccination policies and programs, methodological repertoire to evaluate public health inter-
and to support decisions on reimbursement of vaccina- ventions at the population level. This would help to
tions within the statuary health insurance system by the strengthen implementation research at the interface
Federal Joint Committee. between public health research and practice.
Table 3 (DISSEMINATION) – How can the RKI as a
Part 2: Group discussions national public health institute facilitate dissemination of
Group discussions at the three tables delineated sever- results among public health stakeholders, and what could
al key issues with regard to enhancing next steps for be important steps to enhance that (e.g. Cochrane Public
strengthening EBPH at the RKI. Health Research network, institutional repositories)?
Journal of Health Monitoring 2020 5(S3) 5Journal of Health Monitoring Evidence-based Public Health for Public Health Action – Proceedings of an international workshop at the RKI EDITORIAL
Timely and effective dissemination of evidence in pub-
lic health is essential for the implementation of public
health interventions. It requires building strong networks
between research, practice and policy. It also requires har-
nessing methods and technologies to collate, visualize and
communicate the results of evidence-based public health
research to the specific user groups in need of information
for action.
Overall, the workshop highlighted that principles and
methods of EBPH are fundamental to advance public health
research that informs and influences policy and practice,
which has been defined as one of the essential public health
functions by the World Health Organization. This will be
necessary given new public health threats from infectious
diseases and antimicrobial/antibiotic resistance as well as
from an increasing burden of non-communicable diseases
and age-related health conditions at the national as well
as global level.
Journal of Health Monitoring 2020 5(S3) 6Journal of Health Monitoring Evidence-based Public Health for Public Health Action – Proceedings of an international workshop at the RKI PROCEEDINGS
Journal of Health Monitoring · 2020 5(S3)
DOI 10.25646/6500
Emerging challenges in evidence-based public health and how to
Robert Koch Institute, Berlin
address them
Mark Petticrew
Public health has been defined as ‘the process of mobiliz- ods quite differently. This can be a challenge in itself – but
London School of Hygiene and Tropical Medicine
Department of Social and Environmental Health ing and engaging local, state, national, and international such systems-based approaches offer great promise [4].
Research resources to assure the conditions in which people can be Not least, they can help us to think about how to imple-
healthy’ [1]. ment evidence across very different contexts.
Corresponding author
Prof Dr Mark Petticrew
‘Evidence-based public health’ aims to put scientific evi- This is of practical importance because many non-com-
London School of Hygiene and Tropical Medicine dence at the core of this process, integrating it into public municable diseases (NCDs, and their related inequalities)
Department of Social and Environmental Health
health decision-making, alongside other evidence and arise from systems. Such systems include markets for
Research
15–17 Tavistock Place expertise [2]. This is of course challenging. One reason is unhealthy commodities – such as tobacco, alcohol,
London WC1H 9SH, United Kingdom that public health evidence derives from many sources, unhealthy food, gambling and many others. For example
E-mail: mark.petticrew@lshtm.ac.uk
collected using a wide range of scientific methods. That it has been argued that if we want to understand why peo-
Submitted: 25.11.2019 evidence is often difficult to produce, and generalise from. ple are consuming more unhealthy foods – a major public
Accepted: 20.01.2020 This can make primary research (e.g. evaluations), and sec- health challenge in low and middle income countries, as
Published: 04.06.2020
ondary research (e.g. systematic reviews), difficult and well as in better-off countries – we need to study the trans-
Conflicts of interest costly. Straightforward epidemiological concepts – even formations to economic and social systems that are favour-
The author declared no conflicts of interest. apparently simple ones like ‘population,’ ‘intervention’ and ing their increasing availability and affordability [5].
Note
‘outcome’ – can also be more difficult to define in public This presentation will aim to describe how applying a
External contributions do not necessarily reflect the health contexts [3]. complex systems lens to public health can help address
opinions of the Robert Koch Institute.
Overall, then, this has been one of the greatest chal- major public health problems (like NCDs), but can also help
lenges facing public health: how to develop and use mean- us create more useful, actionable evidence. It will use exam-
ingful actionable evidence. In the face of such complexity, ples from tobacco, alcohol, and food, among others [6, 7].
researchers often retreat into using simpler approaches.
However simplistic models of evidence and evaluation are References
often misleading, and can even be harmful. In contrast, 1. Detels R, McEwen J, Beaglehole R (2002) Oxford textbook of public
health. Vol 1. Oxford University Press., Oxford
conceptualising interventions as changes in complex sys-
2. Sackett DL, Rosenberg WM, Gray JA et al. (1996) Evidence based
tems (rather than as discrete, bounded events) can help medicine: what it is and what it isn’t. BMJ 312(7023):71–72
This work is licensed under a with the development and implementation of public health 3. Petticrew M, Viehbeck S, Cummins S et al. (2016) À mêmes mots, sens
différents – les difficultés de la terminologie épidémiologique avec la
Creative Commons Attribution 4.0 interventions. Systems-based approaches require that we recherche en interventions en santé des populations. Rev Epidemiol
think about public health evidence and public health meth- Sante Publique 64 Suppl 2:S43–54
International License.
Journal of Health Monitoring 2020 5(S3) 7Journal of Health Monitoring Evidence-based Public Health for Public Health Action – Proceedings of an international workshop at the RKI PROCEEDINGS
4. Rutter H, Savona N, Glonti K et al. (2017) The need for a complex sys-
tems model of evidence for public health. Lancet 390(10112):2602–2604
5. Stuckler D, McKee M, Ebrahim S et al. (2012) Manufacturing epidemics:
the role of global producers in increased consumption of unhealthy com-
modities including processed foods, alcohol, and tobacco. PLoS Med
9(6):e1001235
6. Petticrew M, Katikireddi SV, Knai C et al. (2017) ‘Nothing can be done
until everything is done’: the use of complexity arguments by food, bever-
age, alcohol and gambling industries. J Epidemiol Community Health
71(11):1078–1083
7. Petticrew M, Shemilt I, Lorenc T et al. (2017) Alcohol advertising and
public health: systems perspectives versus narrow perspectives.
J Epidemiol Community Health 71(3):308–312
Journal of Health Monitoring 2020 5(S3) 8Journal of Health Monitoring Evidence-based Public Health for Public Health Action – Proceedings of an international workshop at the RKI PROCEEDINGS
Journal of Health Monitoring · 2020 5(S3)
DOI 10.25646/6501
Conceptual issues in relation to the design, implementation and
Robert Koch Institute, Berlin
evaluation of interventions
Eva A. Rehfuess 1,2
1
The conceptualization of public health interventions as the intervention and its components and for thinking
L udwig Maximilian University of Munich
Institute for Medical Information Processing, events in complex systems represents a key starting point through other important factors and the interactions
2
Biometry, and Epidemiology for this contribution [1]. Importantly, the design, evaluation between them [3, 4]. With respect to evidence needs for
Ludwig Maximilian University of Munich
Pettenkofer School of Public Health
and implementation of interventions rarely occur in a lin- decision-making, evidence-to-decision (EtD) frameworks
ear sequence but tend to involve iterations in a cyclical propose a set of criteria, thereby making value assump-
Corresponding author fashion. Generating research evidence across these stages tions explicit. The WHO-INTEGRATE EtD framework,
Prof Dr Eva A. Rehfuess
Ludwig Maximilian University of Munich
ideally involves different public health stakeholders in order firmly rooted in World Health Organization (WHO) norms
Institute for Medical Information Processing, Biometry, to generate policy-relevant evidence. and values, proposes six substantive criteria – balance of
and Epidemiology This contribution focuses on two aspects of relevance health benefits and harms, human rights and socio-cul-
Marchioninistraße 17
81377 Munich, Germany
across intervention design, evaluation and implementa- tural acceptability, health equity, equality and non-discrim-
E-mail: rehfuess@ibe.med.uni-muenchen.de tion: the need to ask a range of questions and to consider ination, societal implications, financial and economic con-
a range of evidence beyond evidence of effectiveness; and siderations and feasibility and health system considerations
Submitted: 25.11.2019
Accepted: 20.01.2020 the recognition that public health interventions depend on – as well as the meta-criterion quality of evidence [5].
Published: 04.06.2020 and interact with their context. These two aspects are intro- Context matters – as illustrated by the health system
duced using examples of interventions targeting individu- intervention to deliver antenatal corticosteroids showing
Conflicts of interest
The author declared no conflicts of interest. als, populations and systems. As the ‘rhetoric urging com- largely positive impacts in high-income settings but sub-
plex systems approaches to public health is only rarely stantial harm in low-income settings. Context is defined as
Note
External contributions do not necessarily reflect the
operationalised in ways that generate relevant evidence of ‘… any feature of the circumstances in which an interven-
opinions of the Robert Koch Institute. effective policies’ [2], the contribution also presents selected tion is implemented that may interact with the intervention
tools to address these challenges in primary research, evi- to produce variation in outcomes’ [6]. The Context and
dence synthesis and decision-making. Implementation of Complex Interventions (CICI) frame-
Whether a public health measure – for example a work represents a tool for reflecting on context in a com-
behavioural intervention to promote healthy eating – is prehensive manner and for considering how contextual
introduced by decision-makers and adopted and main- domains – i.e. geographical, epidemiological, socio-eco-
tained by target populations is influenced by many factors. nomic, socio-cultural, legal, political, ethical – might influ-
This work is licensed under a Logic models, ‘a graphic description of a system … ence how an intervention works (or not), and how inter-
Creative Commons Attribution 4.0 designed to identify important elements and relationships vention impacts vary [7]. This has important implications
International License. within that system …’, are a tool for being explicit about for the transferability of interventions from one context to
Journal of Health Monitoring 2020 5(S3) 9Journal of Health Monitoring Evidence-based Public Health for Public Health Action – Proceedings of an international workshop at the RKI PROCEEDINGS
another, with guidance on how to adapt and re-evaluate
interventions currently being developed [8].
This contribution has focused on only two conceptual
issues affecting the design, evaluation and implementation
of interventions – the broad range of evidence needs and
the importance of context. Taking a complexity perspective
and its implications seriously is likely to require ‘far reach-
ing changes to the way population health intervention
research is funded, conducted and published’ [6].
References
1. Petticrew M, Knai C, Thomas J et al. (2019) Implications of a complexity
perspective for systematic reviews and guideline development in health
decision making. BMJ Glob Health 4(Suppl 1):e000899
2. Rutter H, Savona N, Glonti K et al. (2017) The need for a complex sys-
tems model of evidence for public health. Lancet 390(10112):2602–2604
3. Rehfuess EA, Booth A, Brereton L et al. (2018) Towards a taxonomy of
logic models in systematic reviews and health technology assessments:
A priori, staged, and iterative approaches. Res Synth Methods 9(1):13–24
4. Rohwer A, Pfadenhauer L, Burns J et al. (2017) Series: Clinical Epidemiol-
ogy in South Africa. Paper 3: Logic models help make sense of complexity
in systematic reviews and health technology assessments. J Clin Epidemiol
83:37–47
5. Rehfuess EA, Stratil JM, Scheel IB et al. (2019) The WHO-INTEGRATE
evidence to decision framework version 1.0: integrating WHO norms and
values and a complexity perspective. BMJ Glob Health 4(Suppl 1):e000844
6. Craig P, Di Ruggiero E, Frohlich KL et al. (2018) Taking account of context
in population health intervention research: guidance for producers, users
and funders of research. National Institute for Health Research, United
Kingdom
7. Pfadenhauer LM, Gerhardus A, Mozygemba K et al. (2017) Making sense
of complexity in context and implementation: the Context and Implemen-
tation of Complex Interventions (CICI) framework. Implement Sci
12(1):21
8. Centre for the Development and Evaluation of Complex Interventions for
Public Health Improvement (no date) The Adapt Study.
http://decipher.uk.net/research-page/adaptation-of-population-health-
interventions-for-implementation-and-or-re-evaluation-in-new-contexts-
development-of-guidance/ (As at 31.01.2020)
Journal of Health Monitoring 2020 5(S3) 10Journal of Health Monitoring Evidence-based Public Health for Public Health Action – Proceedings of an international workshop at the RKI PROCEEDINGS
Journal of Health Monitoring · 2020 5(S3)
DOI 10.25646/6502
Taking stock of existing evidence and closing evidence gaps –
Robert Koch Institute, Berlin
Reflections from the National Institute for Health and Care
Kay Nolan
Excellence (NICE)
National Institute for Health and Care Excellence
Centre for Guidelines
The National Institute for Health and Care Excellence tion at NICE is ‘What is the best available evidence to answer
Corresponding author (NICE) has been applying the principles of evidence-based the specific question being addressed?’ We know public
Dr Kay Nolan
National Institute for Health and Care Excellence
medicine to public health problems for over twelve years. health questions may benefit from exploration of a broader
Centre for Guidelines This application is in effect the practice of evidence-based range of evidence including that which is further down the
Level 1A, City Tower, Piccadilly Plaza
public health (EBPH) [1]. NICE as an organisation applies hierarchy of evidence. By taking a transparent and system-
Manchester M1 4BT, United Kingdom
E-mail: Kay.Nolan@nice.org.uk common principles for the development of all its evi- atic approach and pulling from evidence from a range of
dence-based guidelines regardless of topic, one of which disciplines, NICE has been able to review and synthesise
Submitted: 27.11.2019
is the use of the ‘best available evidence’. In the develop- evidence for complex public health problems [2].
Accepted: 20.01.2020
Published: 04.06.2020 ment of any public health guideline, NICE takes stock of The evidence used to generate NICE guidelines is
the existing evidence base. diverse and consists not only of the traditional scientific
Conflicts of interest
The author declared no conflicts of interest.
Infrequently is it practical to assess the effectiveness of published evidence but also stakeholder comments, grey
interventions for complex public health problems by tradi- literature, real world evidence and expert testimony [3]. The
Note tional gold standard randomised control trials. The impor- definition of evidence included in the application of EBPH
External contributions do not necessarily reflect the
opinions of the Robert Koch Institute.
tance of context, wider determinants, subpopulation vari- at NICE has, and continues to, evolve over time, what
ability and the role of the non-human biology element often remains consistent is some assessment of the quality of
mean that specific high quality evidence is unavailable. that evidence particularly its biases.
There is often a lack of good outcome studies answering This presentation will aim to describe the approach
the specific ‘what works’ question. There are often fewer taken by NICE in interrogating the evidence base to develop
studies still that answer the questions what works for whom national public health guidelines. The key challenges and
and under what circumstances? The evidence is also often some solutions will be illustrated through examples from
too imprecise to determine the relationship between the evidence synthesis to support guidelines for non-commu-
intervention and the outcome. nicable diseases.
The public health evidence base is rarely perfect, if it were
This work is licensed under a it would likely negate the need for, or change the approach
Creative Commons Attribution 4.0 and role of organisations such as NICE. Perhaps more
International License. importantly the question asked in the evidence interroga-
Journal of Health Monitoring 2020 5(S3) 11Journal of Health Monitoring Evidence-based Public Health for Public Health Action – Proceedings of an international workshop at the RKI PROCEEDINGS
References
1. Pettman TL, Armstrong R, Doyle J et al. (2012) Strengthening evaluation
to capture the breadth of public health practice: ideal vs. real. J Public
Health (Oxf) 34(1):151–155
2. Lomas J, Culyer T, McCutcheon C (2005) Conceptualizing and combining
evidence for health system guidance: final report. Canadian Health Ser-
vices Research Foundation, Ottawa
3. National Institute for Health and Care Excellence (NICE) (2018) Develop-
ing NICE guidelines: the manual.
https://www.nice.org.uk/process/pmg20/chapter/introduction-and-over-
view (As at 31.01.2020)
Journal of Health Monitoring 2020 5(S3) 12Journal of Health Monitoring Evidence-based Public Health for Public Health Action – Proceedings of an international workshop at the RKI PROCEEDINGS
Journal of Health Monitoring · 2020 5(S3)
DOI 10.25646/6503
Novel methods for health intervention research
Robert Koch Institute, Berlin
Till Bärnighausen
Research is important for the ideation, design, and testing health and behavioral outcomes [22–24], novel approaches
of health interventions. Intervention research can be clas- to quantify economic evaluation [25], and new methods for
University of Heidelberg
Faculty of Medicine, Heidelberg Institute of sified into four dimensions: policy and public engagement. For each method, I explain
Global Health the intuition, describe epistemic and statistical back-
1. Identification of health intervention needs, which grounds, and discuss application opportunities, strengths
Corresponding author
Prof Dr Dr Till Bärnighausen
requires large-scale population-representative studies and weaknesses.
University of Heidelberg (e.g. [1–3]).
Faculty of Medicine
2. Design research to create interventions that are desirable, References
Heidelberg Institute of Global Health
Im Neuenheimer Feld 130.3 feasible and viable, which requires ethnographic studies, 1. Haber N, Tanser F, Bor J et al. (2017) From HIV infection to therapeutic
response: a population-based longitudinal HIV cascade-of-care study in
69120 Heidelberg, Germany ideation, and prototype and pilot testing (e.g. [4–7]). KwaZulu-Natal, South Africa. Lancet Hiv 4(5):E223–E230
E-mail: till.baernighausen@uni-heidelberg.de
3. Evaluation research to quantify (i) causal effects and 2. Geldsetzer P, Manne-Goehler J, Theilmann M et al. (2018) Diabetes and
Hypertension in India A Nationally Representative Study of 1.3 Million
Submitted: 16.01.2020 impacts – causal impact evaluation – [8–10], (ii) mech- Adults. JAMA Intern Med 178(3):363–372
Accepted: 06.05.2020 anisms of action – performance or process evaluation 3. Geldsetzer P, Manne-Goehler J, Marcus ME et al. (2019) The state of
Published: 04.06.2020
– [11], and (iii) social value of health interventions – eco- hypertension care in 44 low-income and middle-income countries: a
cross-sectional study of nationally representative individual-level data
Conflicts of interest nomic evaluation [12]. from 1.1 million adults. Lancet 394(10199):652–662
The author declared no conflicts of interest. 4. Governance and policy translation research to guaran- 4. Yapa HM, Bärnighausen T (2018) Implementation science in resource-
poor countries and communities. Implement Sci 13(1):154
Note
tee the ethical and political ‘goodness’ of our approach-
5. Adam M, McMahon SA, Prober C et al. (2019) Human-Centered Design
External contributions do not necessarily reflect the es to real-life health intervention research [13, 14] and of Video-Based Health Education: An Iterative, Collaborative, Community-
opinions of the Robert Koch Institute.
to ensure rapid incorporation of novel insights into Based Approach. J Med Internet Res 21(1):e12128
health policy and routine practice [15]. 6. Isler J, Sawadogo NH, Harling G et al. (2019) Iterative Adaptation of a
Maternal Nutrition Videos mHealth Intervention Across Countries Using
Human-Centered Design: Qualitative Study. JMIR Mhealth Uhealth
7(11):e13604
For each of these four dimensions of health intervention
7. Muller N, Emmrich PMF, Rajemison EN et al. (2019) A Mobile Health
research, I present several novel methodological approaches Wallet for Pregnancy-Related Health Care in Madagascar: Mixed-Methods
and illustrate them with examples from real-life studies in Study on Opportunities and Challenges. JMIR Mhealth Uhealth
7(3):e11420
resource-poor communities in Africa and Asia. My focus 8. Hu J, Geldsetzer P, Steele SJ et al. (2018) The impact of lay counselors on
is on innovative study designs: new forms of design HIV testing rates: quasi-experimental evidence from lay counselor rede-
ployment in KwaZulu-Natal, South Africa. Aids 32(14):2067–2073
This work is licensed under a research for health interventions [16], a range of innovative
9. McGovern ME, Herbst K, Tanser F et al. (2016) Do gifts increase consent
Creative Commons Attribution 4.0 experimental and quasi-experimental approaches for causal to home-based HIV testing? A difference-in-differences study in rural
KwaZulu-Natal, South Africa. Int J Epidemiol 45(6):2100–2109
International License. impact evaluation [17–21], new methods to validly measure
Journal of Health Monitoring 2020 5(S3) 13Journal of Health Monitoring Evidence-based Public Health for Public Health Action – Proceedings of an international workshop at the RKI PROCEEDINGS
10. Chen S, Sudharsanan N, Huang F et al. (2019) Impact of community 25. Verguet S, Kim JJ, Jamison DT (2016) Extended Cost-Effectiveness Analy-
based screening for hypertension on blood pressure after two years: sis for Health Policy Assessment: A Tutorial. Pharmacoeconomics
regression discontinuity analysis in a national cohort of older adults in 34(9):913–923
China. BMJ 366:l4064
11. Marchal B, van Belle S, van Olmen J et al. (2012) Is realist evaluation
keeping its promise? A review of published empirical studies in the field
of health systems research. Evaluation 18(2):192–212
12. Barnighausen T, Bloom DE, Cafiero-Fonseca ET et al. (2014) Valuing
vaccination. n: Proc Natl Acad Sci USA 111(34):12313–12319
13. Barnighausen T, Eyal N, Wikler D (2014) HIV Treatment-as-Prevention
Research at a Crossroads. PLoS Med 11(6)
14. Eyal N, Lipsitch M, Barnighausen T et al. (2018) Risk to study nonpartici-
pants: A procedural approach. Proc Natl Acad Sci USA 115(32):8051–8053
15. Barnighausen T (2017) Population health intervention research: three
important advancements. Int J Med Sci Public Health 62(8):841–843
16. Berner-Rodoreda A, Barnighausen T, Kennedy C et al. (2018) From doxas-
tic to epistemic – a typology and critique of qualitative interview styles.
Qualitative Inquiry (efirst)
17. Barnighausen T, Tugwell P, Rottingen JA et al. (2017) Quasi-experimental
study designs series-paper 4: uses and value. J Clin Epidemiol 89:21–29
18. Barnighausen T, Oldenburg C, Tugwell P et al. (2017) Quasi-experimental
study designs series-paper 7: assessing the assumptions. J Clin Epidemiol
89:53–66
19. Bor J, Moscoe E, Mutevedzi P et al. (2014) Regression Discontinuity
Designs in Epidemiology Causal Inference Without Randomized Trials.
Epidemiology 25(5):729–737
20. Ortblad K, Musoke DK, Ngabirano T et al. (2017) Direct provision versus
facility collection of HIV self-tests among female sex workers in Uganda:
A cluster-randomized controlled health systems trial. Plos Med 14(11)
21. Geldsetzer P, Francis JM, Sando D et al. (2018) Community delivery of
antiretroviral drugs: A non-inferiority cluster-randomized pragmatic trial
in Dar es Salaam, Tanzania. Plos Med 15(9)
22. Barnighausen T, Bor J, Wandira-Kazibwe S et al. (2011) Correcting HIV
Prevalence Estimates for Survey Nonparticipation Using Heckman-type
Selection Models. Epidemiology 22(1):27–35
23. Haber N, Harling G, Cohen J et al. (2018) List randomization for eliciting
HIV status and sexual behaviors in rural KwaZulu-Natal, South Africa: a
randomized experiment using known true values for validation. BMC
Med Res Methodol 46(18):1–12
24. Harling G, Gumede D, Mutevedzi T et al. (2017) The impact of self-inter-
views on response patterns for sensitive topics: a randomized trial of
electronic delivery methods for a sexual behaviour questionnaire in rural
South Africa. BMC Med Res Methodol 125(17):1–14
Journal of Health Monitoring 2020 5(S3) 14Journal of Health Monitoring Evidence-based Public Health for Public Health Action – Proceedings of an international workshop at the RKI PROCEEDINGS
Journal of Health Monitoring · 2020 5(S3)
DOI 10.25646/6504
Systematic reviews in public health: Exploring challenges and
Robert Koch Institute, Berlin
potential solutions
Stefan K. Lhachimi
Systematic reviews (SR) utilizing meta-analysis of ran Second, primary studies used for SR-PHIs are often not
University of Bremen
Institute for Public Health and Nursing Research domized controlled trials (RCTs) are a foundation block of randomized. Even in cases using RCT designs, studies lack
evidence-based medicine, with Cochrane reviews often important characteristics that ensure a low risk of bias. For
Corresponding author being considered the gold standard. Systematic reviews of example, behavioural interventions cannot be blinded and
Prof Dr Stefan K. Lhachimi
University of Bremen
public health interventions (SR-PHI) are certainly desirable contamination often cannot be avoided. A newly developed
Institute for Public Health and Nursing Research and conducted with increased frequency [1]. Nevertheless, risk of bias tool (ROBINS-I) aims to mitigate this challenge
Grazer Straße 4
public health interventions (PHI) differ substantially from by potentially allowing to upgrade certain non-random stud-
28359 Bremen, Germany
E-mail: Stefan.Lhachimi@uni-bremen.de clinical interventions but a precise demarcation remains ies to ‘moderate quality’ [8].
elusive. Commonly, PHIs are some kind of public policy or Third, primary studies for PHIs are less standardized:
Submitted: 25.11.2019
behavioural intervention that aims to protect or improve Evidence-based medicine explicitly aims for replication of
Accepted: 20.01.2020
Published: 04.06.2020 health at the population level; not merely a primarily clin- previous findings to increase robustness of the evidence
ical intervention that has ‘public health relevance’ due to base. In contrast, public health studies differ often in their
Conflicts of interest
The author declared no conflicts of interest.
disease burden or patient numbers (e.g. vaccination). precise intervention-specification, and often no agreed
Drawing from a set of published and ongoing Cochrane standards exist on primary endpoints or minimum signif-
Note reviews on fiscal policies employed as public health inter- icant differences.
External contributions do not necessarily reflect the
opinions of the Robert Koch Institute.
ventions – unconditional cash transfers (UCT) [2, 3] and Fourth, transferability of the findings of an SR-PHI to a
taxation of high caloric foods (‘sin taxes’) [4–6] – we will different setting can be unclear: An understudied aspect
present several challenges of SR-PHIs. of transferability of a given PHI is the political feasibility in
First, conducting SR-PHIs is time consuming: For SRs a different polity (e.g. are unconditional cash transfers truly
published by the Cochrane Public Health Group in the pre- acceptable for the general population). Another challenge
vious two years, the median duration between publishing is that jurisdictions/population differs substantially in their
the protocol and publishing the review was approx. 56 health and behavioural profile. Hence, applying particular
months (range: approx. 22–78 months). Usually, the search interventions can yield very different outcomes in terms of
must be broader in terms of types of literature (in particu- magnitude [9]. Tools originating from quantitative health
lar grey literature or policy papers) and search strategy. impact assessment are an option to quantify prospectively
This work is licensed under a Recent research on abbreviated search strategies suggests, the consequences of a public health population for a par-
Creative Commons Attribution 4.0 however, that the scope for reducing the search effort for ticular intervention [10].
International License. SR-PHIs is currently lower than for clinical interventions [7].
Journal of Health Monitoring 2020 5(S3) 15Journal of Health Monitoring Evidence-based Public Health for Public Health Action – Proceedings of an international workshop at the RKI PROCEEDINGS
In conclusion, the relative complexity and duration of
SR-PHIs are in stark contrast to the need of giving well-
timed policy advice. For some of these outlined challenges
methodological work is already ongoing. Nevertheless,
designer of primary studies for PHIs must pay more atten-
tion to potential inclusion in a future SR by standardising
interventions and endpoints.
References
1. Petticrew M (2009) Systematic reviews in public health: old chestnuts
and new challenges. Bull World Health Organ 87(3):163–163
2. Pega F, Liu SY, Walter S et al. (2017) Unconditional cash transfers for
reducing poverty and vulnerabilities: effect on use of health services and
health outcomes in low- and middle-income countries. Cochrane Data-
base Syst Rev (11)
3. Pega F, Liu SY, Walter S et al. (2015) Unconditional cash transfers for
assistance in humanitarian disasters: effect on use of health services and
health outcomes in low- and middle-income countries. Cochrane Data-
base Syst Rev (9)
4. Heise TL, Katikireddi SV, Pega F et al. (2016) Taxation of sugar-sweetened
beverages for reducing their consumption and preventing obesity or other
adverse health outcomes: Protocols. Cochrane Database Syst Rev (8)
5. Lhachimi SK, Pega F, Heise TL et al. (2016) Taxation of the fat content of
foods for reducing their consumption and preventing obesity or other
adverse health outcomes (Protocol). Cochrane Database Syst Rev (10)
6. Pfinder M, Katikireddi SV, Pega F et al. (2016) Taxation of unprocessed
sugar or sugar-added foods for reducing their consumption and prevent-
ing obesity or other adverse health outcomes. Cochrane Database Syst
Rev (8)
7. Nussbaumer-Streit B, Klerings I, Wagner G et al. (2018) Abbreviated liter-
ature searches were viable alternatives to comprehensive searches: a
meta-epidemiological study. J Clin Epidemiol 102:1–11
8. Thomson H, Craig P, Hilton-Boon M et al. (2018) Applying the ROBINS-I
tool to natural experiments: an example from public health. Syst Rev
7(1):15
9. Lhachimi SK, Cole KJ, Nusselder WJ et al. (2012) Health impacts of
increasing alcohol prices in the European Union: A dynamic projection.
Prev Med 55(3):237–243
10. Schonbach JK, Thiele S, Lhachimi SK (2019) What are the potential pre-
ventive population-health effects of a tax on processed meat? A quantita-
tive health impact assessment for Germany. Prev Med 118:325–331
Journal of Health Monitoring 2020 5(S3) 16Journal of Health Monitoring Evidence-based Public Health for Public Health Action – Proceedings of an international workshop at the RKI PROCEEDINGS
Journal of Health Monitoring · 2020 5(S3)
DOI 10.25646/6505
Evidence-based public health (EBPH) health policy advising and
Robert Koch Institute, Berlin
information of the public
Manfred Wildner 1, 2
1
Public Health has been defined as ‘the science and art of public health action cycle, with various entry points for
avarian Health and Food Safety Authority
B
Department of Health preventing disease, prolonging life and promoting […] policy advising. Ideas: they have great influence, e.g. as
2
Ludwig Maximilian University of Munich health […] through organized community efforts’ [1]. Reflec- value systems in science, politics, practice or moreover as
Pettenkofer School of Public Health
tions on the art of bringing evidence into policies and final- political ideologies, or in the context of a stakeholder anal-
Corresponding author
ly practice are of equal importance as theoretical endeav- ysis of institutions, interests, ideas and their networks [9].
Prof Dr Manfred Wildner ours in this field, which are currently pointing out the Settings and context: they are important for defining the
Bavarian Health and Food Safety Authority
non-linearity of these processes [2, 3]. Generally, advising political playing field, the value system, the regional scope
Department of Health
Veterinärstraße 2 on the transfer of evidence in public health should consid- and administrative organizational level, the sensitivity for
85764 Oberschleißheim, Germany er the triangular relationship of the domains of science, context influences and last not least as reference for an
E-mail: Manfred.Wildner@lgl.bayern.de
practice and the political field with their differing core val- actors own place.
Submitted: 25.11.2019 ues [4]. This contribution hence argues for a conscious Information of the public is a necessary endeavour for
Accepted: 20.01.2020 awareness of words and framing, people involved, time at least three reasons. From a scientists’ perspective it is
Published: 04.06.2020
and timing, ideas and ideologies, settings and context and a responsibility resulting from public funding as well as an
Conflicts of interest their interplay, consistent with other reports [5, 6]. opportunity for advancing novel insights. From a political
The author declared no conflicts of interest. Words: they limit and predispose our thinking (‘linguis- perspective it is an important legitimation of decision mak-
Note
tic turn’) and are a frequent cause of subtle misunder- ing and an effective element for obtaining a mandate for
External contributions do not necessarily reflect the standings and ideological prejudices. People: institutions action. From a public health perspective it is ‘a fantastic
opinions of the Robert Koch Institute.
and their interests are critical, as is the ‘behavioral cohe- opportunity to get good accurate evidence-based science
siveness’ of actors/groups and the frequent ‘behavioural into the public domain at the very moment when the wider
incohesiveness’ of the media [7]. Time: awareness of the public really cares’, contributing to effective communica-
space time expectations of effects in public health, ‘flipped tion and management as part of e.g. risk or crisis manage-
evaluation’ of entrepreneurs, ‘reversed causality’ due to ment [10].
future expectations, the role of windows of opportunity The underlying vision is an interactive public health-trans-
[8] and the importance of short term, midterm and long lation network with elements such as projects, programs,
term planning horizons is crucial. Ideally, evidence partnerships, models of good practice and practice of good
This work is licensed under a appraisal, policy formulation, political priorisation and models, cooperative research and development, doctoral-,
Creative Commons Attribution 4.0 adaptation, practical implementation and scientific eval- postdoc- and midcareer-fellowships and cooperative insti-
International License. uation form a logical time sequence, derived from the tution building [11].
Journal of Health Monitoring 2020 5(S3) 17Journal of Health Monitoring Evidence-based Public Health for Public Health Action – Proceedings of an international workshop at the RKI PROCEEDINGS
References
1. Winslow CEA (1920) The Untilled Fields of Public Health. Science
51(1306):23–33
2. Smith KE, Katikireddi SV (2013) A glossary of theories for understanding
policymaking. J Epidemiol Community Health 67(2):198–202
3. Rutter H, Savona N, Glonti K et al. (2017) The need for a complex sys-
tems model of evidence for public health. Lancet 390(10112):2602–2604
4. Wildner M (2012) Prävention an den Schnittstellen zu Politik und Praxis.
Gesundheitswesen 74(4):229–233
5. Figueras J, Voipiu-Pulkki LM (2018) Twenty years of evidence into practice.
Eurohealth 24(2):2–3
6. Lessof S, Figueras J, McKee M et al. (2018) Twenty years of evidence into
practice: Refections on the observatory in 10 (key) lessons. Eurohealth
24(2):4–7
7. Frey FW (1985) The Problem of Actor Designation in Political Analysis.
Comparative Politics 17(2):127–152
8. Kingdon JW (1995) Agendas, alternatives, and public policies (2nd ed.).
Longman, New York
9. Shearer JC, Abelson J, Kouyate B et al. (2016) Why do policies change?
Institutions, interests, ideas and networks in three cases of policy reform.
Health Policy Plan 31(9):1200–1211
10. Fox F (2014) Engaging with the media. Lancet 383(Suppl 1):S6–7
11. Teichert U, Kaufhold C, Rissland J et al. (2016) Vorschlag für ein bundes-
weites Johann-Peter Frank Kooperationsmodell im Rahmen der nationa-
len Leopoldina-Initiative für Public Health und Global Health. Eine Stel-
lungnahme des Bundesverbandes der Ärztinnen und Ärzte des
Öffentlichen Gesundheitsdienstes (BVÖGD) e. V. Gesundheitswesen
78(7):473–476
Journal of Health Monitoring 2020 5(S3) 18Journal of Health Monitoring Evidence-based Public Health for Public Health Action – Proceedings of an international workshop at the RKI PROCEEDINGS
Journal of Health Monitoring · 2020 5(S3)
DOI 10.25646/6506
Experiences from the Department of Infectious Disease
Robert Koch Institute, Berlin
Epidemiology at Robert Koch Institute
Thomas Harder
At Robert Koch Institute’s Department of Infectious Dis- ing process. Since 2012, ten recommendations on new vac-
Robert Koch Institute, Berlin
Department of Infectious Disease Epidemiology ease Epidemiology, approaches for applying the principles cines or revisions of previous recommendations have been
of evidence-based public health (EBPH) have been particu- successfully developed using the SOP, accompanied by
Corresponding author larly successfully implemented in the Immunization Unit publication of background papers and of the respective sys-
Dr Thomas Harder
Robert Koch Institute
and in the Unit for Healthcare-associated Infections, Sur- tematic reviews in peer-reviewed journals [1–3].
Department of Infectious Disease Epidemiology veillance of Antibiotic Resistance and Consumption. Fur- At the Unit for Healthcare-associated Infections, Surveil-
Seestraße 10
thermore, both units participated in the international Pro- lance of Antibiotic Resistance and Consumption, the EBPH
13353 Berlin, Germany
E-mail: HarderT@rki.de ject on a Framework for Rating Evidence in Public Health approach has been used to perform systematic reviews on
(PRECEPT) which has been funded by the European Centre the burden of healthcare-associated infections [4]. As an
Submitted: 25.11.2019
for Disease Prevention and Control (ECDC). example, in an ECDC-funded project the long-term conse-
Accepted: 20.01.2020
Published: 04.06.2020 The Immunization Unit hosts the Executive Secretariat quences of health-care-acquired neonatal sepsis for the
of the German Standing Committee on Vaccination (STIKO). neurological development of preterm infants were assessed
Conflicts of interest
The author declared no conflicts of interest.
In 2011, STIKO has adopted a new Standard Operating Pro- by meta-analysis [5]. Furthermore, a systematic review
cedure (SOP) for the development of vaccination recom- addressed the prognostic value of neonatal surface screen-
mendations. Main aim was to provide a methodological ing for gram-negative bacteria for sepsis prediction [6].
framework to guide this process by applying the principles The latter review was part of the PRECEPT project.
of EBPH. The SOP comprises ten consecutive steps, start- PRECEPT defined a methodology for evaluating and grad-
ing with a prioritization process of relevant topics, and end- ing evidence in public health, with a particular focus on
ing with the publication of the recommendation and a infectious disease epidemiology, prevention and control,
related paper on the scientific background. Following the taking different domains and question types into consider-
principles of EBPH, systematic reviews and meta-analyses ation [7]. The methodology rates evidence in four domains:
are performed to address key issues. The methodology of disease burden, risk factors, diagnostics and intervention.
the Grading of Recommendations Assessment, Develop- The framework has four steps going from overarching ques-
ment and Evaluation (GRADE) working group is a core ele- tions to an evidence statement. In step 1, approaches for
ment of the SOP and is used to assess the quality of evi- identifying key areas and developing specific questions are
This work is licensed under a dence on vaccine efficacy/effectiveness and safety, including described. In step 2, methodological guidance for conduct-
Creative Commons Attribution 4.0 population effects. In addition, STIKO has started to use ing systematic reviews is provided. In step 3, a standardized
International License. Evidence-to-Decision Tables to support the decision-mak- evidence-grading scheme using the GRADE methodology
Journal of Health Monitoring 2020 5(S3) 19Journal of Health Monitoring Evidence-based Public Health for Public Health Action – Proceedings of an international workshop at the RKI PROCEEDINGS
is provided. Step 4 consists of preparing a narrative evi-
dence summary. The development of the approach was
accompanied by piloting studies as well as studies on
methodological aspects such as choice of risk of bias tools
and use of existing systematic reviews for development
of new recommendations [8, 9]. Dissemination of the
PRECEPT approach was supported by the development
of an e-learning tool hosted by ECDC.
References
1. Harder T, Wichmann O, Klug SJ et al. (2018) Efficacy, effectiveness and
safety of vaccination against human papillomavirus in males: a systematic
review. BMC Med 16(1):110
2. Falkenhorst G, Remschmidt C, Harder T et al. (2017) Effectiveness of the
23-Valent Pneumococcal Polysaccharide Vaccine (PPV23) against Pneu-
mococcal Disease in the Elderly: Systematic Review and Meta-Analysis.
PLoS One 12(1):e0169368
3. Delere Y, Wichmann O, Klug SJ et al. (2014) The efficacy and duration of
vaccine protection against human papillomavirus: a systematic review
and meta-analysis. Dtsch Arztebl Int 111(35–36):584–591
4. Cassini A, Plachouras D, Eckmanns T et al. (2016) Burden of Six Health-
care-Associated Infections on European Population Health: Estimating
Incidence-Based Disability-Adjusted Life Years through a Population
Prevalence-Based Modelling Study. PLoS Med 13(10):e1002150
5. Haller S, Deindl P, Cassini A et al. (2016) Neurological sequelae of
healthcare-associated sepsis in very-low-birthweight infants: Umbrella
review and evidence-based outcome tree. Euro Surveill 21(8):30143
6. Seidel J, Haller S, Eckmanns T et al. (2018) Routine screening for coloni-
zation by Gram-negative bacteria in neonates at intensive care units for
the prediction of sepsis: systematic review and meta-analysis. J Hosp
Infect 99(4):367–380
7. Harder T, Takla A, Eckmanns T et al. (2017) PRECEPT: an evidence
assessment framework for infectious disease epidemiology, prevention
and control. Euro Surveill 22(40)
8. Harder T, Remschmidt C, Haller S et al. (2016) Use of existing systematic
reviews for evidence assessments in infectious disease prevention: a
comparative case study. Syst Rev 5(1):171
9. Harder T, Takla A, Rehfuess E et al. (2014) Evidence-based decision-mak-
ing in infectious diseases epidemiology, prevention and control: match-
ing research questions to study designs and quality appraisal tools. BMC
Med Res Methodol 14:69
Journal of Health Monitoring 2020 5(S3) 20You can also read
