Audit Manual March 2016 - Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Audit Group - North West Clinical Senates
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Cheshire and Merseyside
Palliative and End of Life Care
Network Audit Group
Audit Group
Audit Manual
March 2016Cheshire and Merseyside Palliative and End of Life Care
Network Audit Group
Audit Manual 2016
CONTENTS
Acknowledgements 4
1. Introduction 5
1.1 Background. 5
1.2 Aims of the Audit Manual. 6
1.3 Overview of the Network Audit Programme. 6
1.4 Definitions of Clinical Audit. 6
1.5 Roles and Responsibilities for Members of the Guideline 8
Development Group.
1.6 The StR Audit Co-ordinator. 9
2. Key Steps in Delivering A Successful Audit 11
2.1 What Kind of Audit? 11
2.2 Planning the Audit. 12
2.3 Membership of the Guideline Development Group. 12
2.3.1 Defining the Scope of the Guideline and Audit. 12
2.3.2 The Systematic Literature Review. 12
2.3.3 The Invited Expert. 13
2.3.4 Managing the Process and Support. 13
2.4 Survey and Audit Form Design. 14
2.5 Data Collection. 16
2.6 Analysis of Data. 16
2.7 Producing Localised Reports. 17
3. The Audit Presentation 18
4. Developing the Standards and Guidelines 19
5. How to Access Support 20
6. Dissemination 22
7. The Standards and Guidelines Book and Website 23
8. References and Resources 24
2 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 2018CONTENTS (continued)
9. Appendices 25-41
Appendix 1 NICE Accreditation Process. 25-28
Appendix 2 Overview of Network Audit Structures. 29
Appendix 3 Job Description for StR Audit Co-ordinator 30-32
Appendix 4 Specialist Palliative Care Units / Teams participating in Regional 33-35
and Supra regional Audits
Appendix 5 Example of a Gantt Chart. 36
Appendix 6 One Page Protocol for Marie Curie Palliative Care Institute 37-38
Appendix 7 The Marie Curie Hospice Liverpool and Marie Curie Palliative 39
Care Institute Liverpool.
Appendix 8 Audit Topics to Date.( January 2016) 40-41
3 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 2018ACKNOWLEDGEMENTS
Clinical audit has the potential to make a significant impact on the quality of care that
patients and those important to them experience. This manual aims to encourage
healthcare professionals to get involved with audit and make a difference. I hope that
readers of this second edition of the Audit Manual will find it a useful and interesting
resource.
The audit programme and this manual would not exist without the input and support
of many colleagues. I would like to acknowledge the huge contributions from
members of the Audit Group over the past 18 years. They have helped the audit
process to develop and strengthen through their hard work and commitment.
Special thanks go to Dr Andrew Khodabukus, Dr Sarah Fradsham and
Dr Aileen Scott for their drive, enthusiasm and hard work in creating the first edition
of the audit manual. Thanks now also go to Dr Daniel Monnery for his help with this
second edition.
Thank you to all the members of the Audit Subgroup for their leadership of the audit
programme.
Thank you once again to Dr Helen Bonwick for proof reading the manuscript and
removing all of the grammatical errors.
Thank you to the Cheshire and Merseyside Strategic Clinical Network for their
financial support of the group which has helped us to develop and improve the
quality of our audit programme.
Finally a huge thank you to Kath Davies for her administrative support in producing
this new manual, setting up our website and coordinating the audit programme. We
wouldn’t manage without you.
Dr Alison Coackley
Chair of the Audit Subgroup
Cheshire and Merseyside Palliative and End of Life Care Network
March 2016
4 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 2018INTRODUCTION
Section 1
Welcome to the Cheshire and Merseyside Palliative and End of Life Care Network
Audit Manual 2nd edition 2016.
This manual has been developed to help you become familiar with the audit process
and hopefully answer some of the questions you may have. Further information is
available on the Network website. (Link) Alternatively please feel free to contact any
member of the Audit Subgroup for further guidance and support. Contact details can
be found on the audit page of the Network website.
1.1 BACKGROUND
The Cheshire and Merseyside Palliative and End of Life Care Network Audit Group
(CMPCNAG) was formed in 1995. One of the main objectives of the group is to use
clinical audit projects to develop standards and guidelines, which can be used to
support specialist palliative care professionals and empower generalists working in
other areas of health care.
The group consists of individuals involved in specialist palliative care across the
Network in community, hospital and hospice settings. The strength of the group lies
in the multidisciplinary representation and the willingness of individuals, units and
services to participate in the audit programme. Over the past twenty years the group
has continued to develop and now holds bimonthly audit meetings throughout the
year. Every spring the whole group plans the audit programme for the next 12 to 18
months, thus helping to maintain ownership of the audit process and enthusiasm for
the projects.
The number and type of audits undertaken each year has varied. Until 2014, five
audit projects were chosen, three of which were identified as regional audits and
based on a previously audited topic. One of these audits was often expanded to
include localities outside the Network as a ‘supra-regional’ audit. A further two new
audit topics were also chosen one of which was designated as non-symptom control.
Since 2014/2015 four regional audits are undertaken each year. This was in
recognition of the need to maintain quality with the more complex and intensive
guideline development process that the group adopted. Implementation of the
robust process will allow the group to apply for NICE accreditation in 2016. (see
Appendix 1)
Each audit meeting attracts approximately 60 health care professionals from across
the Network. The meeting begins with the presentation of a literature review of the
designated topic, followed by results from the audit. Proposed standards and
guidelines, or suggested amendments to existing guidelines, are then presented.
This is followed by an open discussion to further develop and refine the proposed
standards and guidelines. An invited expert is also asked to contribute to both the
presentation and discussion. The guidelines are developed following the process
outlined in the Guideline Development Manual.1[Link] The guidelines are not
standard protocols and are by no means prescriptive.
5 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 2018INTRODUCTION
Section 1
Regional audits complete the audit cycle by assessing the group’s performance
against previously developed standards. Dissemination of results from the audit
projects has been achieved by presentations and publications at national and
international level. Examples of publications can be found on the Network website.1
[Link]
Following the meeting, the draft standards and guidelines are sent to members of the
audit subgroup for comment and then to an external reviewer. At the end of the
process the standards and guidelines are finalized and published on the Network
website.[Link] A full description of the review process may be found in the Guideline
Development Manual.1 [Link]
In 2015 the group began a process of localized reporting for clinical audits and for
teams which had submitted a specific level of data. This is usually set as a minimum
of 10 clinical cases but will vary between the different audits. This initiative will
continue to develop over the next 18 months and will allow units to focus on quality
improvements to their delivery of care.
In 2013, following consultation with group members it was decided to apply for NICE
accreditation of the Standards and Guidelines. This process has included a rigorous
development of a new Guideline Development Manual to meet all of the 23 criteria
specified by NICE. We have been unable to submit an application until we have at
least two guidelines which had completed the entirety of the new process. We are
now in that position and the application for NICE accreditation will be submitted in
spring 2016 (see Appendix 1).
1.2 AIMS OF THE AUDIT MANUAL
This audit manual is designed to provide you with information about the audit
programme and how to participate in an audit project. It describes the different
phases of an audit project including planning, survey design, data collection,
analysis, presentation of results and dissemination. It is designed to be used in
conjunction with the Guideline Development Manual.1[Link]
1.3 OVERVIEW OF THE NETWORK AUDIT PROGRAMME
The Audit Subgroup oversees the audit programme for the Network and the
production of standards and guidelines. The Subgroup is responsible for ensuring
guidelines are produced according to the agreed process and disseminated in a
timely manner to appropriate stakeholders. This group has representatives from
each of the localities within the Network and holds four formal meetings per year.
The Chair of the Audit Group provides quarterly reports to the Network Steering
Group. Appendix 2 gives an overview of the Network structure. Terms of reference
and membership for the Audit Subgroup can be found on the Network website. [Link]
6 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 2018INTRODUCTION
Section 1
1.4 DEFINITIONS OF CLINICAL AUDIT
Clinical audit is a quality improvement process that seeks to improve patient care
and outcomes through the systematic review of care against explicit criteria and the
implementation of change. (see Figure 1) Aspects of the structure, processes and
outcomes of care are selected and systematically evaluated against explicit criteria.
Where indicated, changes are implemented at an individual team or service level
and further monitoring is used to confirm improvement in healthcare delivery. 3
Research, service evaluation and audit can sometime be confused. Table 1.1
highlights the differences in ethos, aims, components, data and organisation.
Table 1.1 Key Differences between Research, Service Evaluation and Audit
(adapted from Harris 4 )
Elements Audit Service Evaluation Research
Are we doing what What are we doing? What should we do?
we should be doing?
Ethos Are we doing what
we said we would be
doing?
Assesses current Describes current Identifies new
practice against a practice knowledge to
Aims
standard influence clinical
practice
Measures or Measures or Observational
describes activity describes activity but studies or
Components
against agreed not against a experiments testing
standards particular standard hypotheses
Examines standard Examines standard Often need to get
collected data (e.g. collected data – (e.g. new data
Data case note analysis) National Council for
Palliative Care
Minimum Data Set)
Proforma used to Proforma used to Study protocols.
assess standard assess standard
collected data collected data Must comply with
Good Clinical
NHS research ethics NHS research ethics Practice for
Organisation approval not approval not Research
needed. needed.
Almost invariably
Likely to need local Likely to need local needs NHS research
clinical governance clinical governance ethics approval
or audit approval or audit approval
7 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 2018.
INTRODUCTION
Section 1
Figure 1: The Audit Cycle
1.5 ROLES AND RESPONSIBILITIES FOR MEMBERS OF THE GUIDELINE
DEVELOPMENT GROUP
A number of professionals will volunteer to deliver the audit project and develop the
standards and guidelines for that topic. They will be known as the Guideline
Development Group. (GDG) The group will include a range of healthcare
professionals from specialist palliative care. Members of a GDG may not necessarily
work with other members on a day to day basis. Being involved in an audit is often a
good way of getting to know other colleagues from across the Network.
The group will include a clinical lead with overall responsibility for keeping the audit
project on track and ensuring that standards and guidelines are completed according
to the NICE process in a timely fashion. Historically this has usually been a
Consultant in Palliative Medicine but it is open to all professionals within the group
who have appropriate audit, leadership and supervision experience.
The group will include at least one specialty registrar and two registrars where
possible. Specialty registrars (StRs) are undertaking a four year training programme
in Palliative Medicine which will equip them to take on a Consultant role in the
specialty. All registrars have to complete audits as part of their training. If they are
nearing the end of their training they will be tasked with co-ordination of the GDG
and will do much of the organisation. If they are a more junior registrar they will have
a supporting role giving them opportunity to develop skills both in clinical audit and
project management. Individual StRs need to consider how they meet the curriculum
competencies required for audit. 5, 6 [Link]
8 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 2018INTRODUCTION
Section 1
It is equally important that specialty doctors working across the network have
adequate experience in clinical audit. They should be encouraged to participate in
the programme and where possible, be an active member of one of the GDGs.
Doctors hoping to submit a CESR application will need to provide extensive
evidence of participation in audit and quality improvement projects. It may be
appropriate for the specialty doctor to take the place of one of the registrars
depending on numbers and availability.
Membership of a GDG should include professionals with a variety of backgrounds
and experience as the project will benefit from diverse and shared input.
Participation in a GDG can support revalidation for doctors and nurses and provide
evidence for Agenda for Change/Knowledge and Skills Framework Appraisal (KSF)
for allied health professionals.
It is important that everyone in a GDG is prepared to contribute to the work as each
audit involves a significant amount of time and effort. At the first planning meeting it
is important to consider the roles and responsibilities of different members. Certain
aspects of the project may be unfamiliar to some members of the group depending
on previous experience. Participation should be seen as an opportunity to develop
skills, and help and support will be available from other members of the group.
It is important that where possible everyone attends the group meetings in order to
keep the project on track. Group members should do their best to arrange release
from the workplace. This can be difficult and should be discussed with their Line
Manager as soon as possible. It is also helpful if meetings are scheduled at the initial
planning discussion so that everyone can have dates, times and locations in their
diary. Where possible the location for the meetings should be rotated so that travel is
shared equally between members.
Table 1.2 provides an overview of key responsibilities for the GDG and an optimum
timescale for project management.
1.6 THE StR AUDIT CO-ORDINATOR
The Audit Co-ordinator is a senior StR within the training programme. The trainee
group nominates the most appropriate person to take on the role, usually for 12
months. The Audit Co-ordinator is a member of the Audit Subgroup and is
responsible for coordinating the bimonthly meetings and supporting other registrars
involved in audit projects. They may be a very useful source of advice and help.
Details of how to contact them are available on the website.2 [Link] A job description
for this role can be found in Appendix 3
You are encouraged to keep the Audit Co-ordinator up to date with the progress of
your audit to ensure you are meeting appropriate time frames and also to let them
know if you are encountering any specific problems with your audit.
9 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 2018INTRODUCTION
Section 1
Table 1.2 Roles and Responsibilities of the GDG
Responsibility Suggested Timescale
1. To follow the NICE agreed process for production of the Throughout the process
standards and guidelines using the information in the
Guideline Development Manual.
2. To define the clinical question(s) for the audit topic. At the beginning of the
process
3. To conduct a systematic literature review. Complete 4 weeks before
presentation
4. To design an audit tool/ data collection form. Complete, design and
submit for approval at least
4 weeks before start of data
collection
5. To communicate arrangements and guidance for data Discuss with Co-ordinator
collection with all localities, teams and units participating in soon after first GDG
the audit. meeting.
Approved materials to be
with Coordinator 2 weeks
before data collection starts
6. To keep the Audit Co-ordinator informed about the progress Throughout the process
of the audit and guideline production.
7. To collate the data returns and analyse the information. 4 weeks before
presentation
8. To invite an expert to the audit presentation. At start of process
9. To deliver an audit presentation to the Network Group. On set date. Copy of
presentation to be sent to
Co-ordinator 1 week in
advance of the meeting.
10. To develop updated or original standards and guidelines for 8 weeks after initial
that topic according to the process and present them to the presentation. Date set in
Network Group as timetabled. audit programme
11. To submit the final draft of the standards and guidelines to 4 months from the date of
the Audit Subgroup Chair ready to start the external review presentation
process
12. To take feedback from the external reviewer as part of the Complete whole process
guideline development process and adjust documentation within 6-8 months of
as appropriate. Produce finalized guidelines. presentation date
13. To ensure dissemination of the work e.g. abstracts, posters As soon as audit project
presentations, articles. This may be local, national or complete. Be aware of
international. submission deadlines for
conferences.
14. To contribute to the production of any supporting material Complete within 6-12
for the standards and guideline. months from the
presentation date
15. To assist with reviewing and updating the standards and As required
guidelines as appropriate
10 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 2018KEY STEPS TO DELIVERING A SUCCESSFUL
Section 2
AUDIT
2.1 WHAT KIND OF AUDIT?
The audit may be either local or regional.
Regional audits are designed to look at standards and guidelines which have
previously been developed. If it is a regional audit it will involve data collection from
all areas across the Network.
In each locality there are generally three care settings i.e. community, hospice, and
hospital. Where possible all three settings in each locality should be included in data
collection for a regional audit. The Audit Coordinator will have details for each of the
different areas in the Cheshire and Merseyside Network. A list is also available in
Appendix 4.
It may be appropriate to use a previously designed audit tool / proforma or the group
may want to develop a new tool. Examples used in previous audits can be found on
the website. [Link] It is essential that the group audit against the existing standards
and guidelines.
A local audit may look at either one or two localities. It may not necessarily include
all care settings although it is usually helpful to try and do so. This will be a new audit
so no standards or guidelines will exist and there will be no local audit tool /
proforma.
Table 2 highlights the key differences between a local and regional audit. Note: A
junior StR would not be expected to lead a regional audit .This would be the
responsibility of a more senior trainee. The less experienced StR would be expected
to play a supporting role allowing development of the necessary knowledge and
skills.
Sometimes partners from organizations outside the Cheshire and Merseyside
Cancer Network wish to participate in an audit. This then becomes a supra regional
audit. Details of teams and units who may wish to participate can be found in
Appendix 4.
The annual review meeting held in the spring determines the timetable and topics for
audit for the following year (see Guideline Development Manual 1).[Link]
Table 2 Key differences between a CMPCNAG local and regional audit
Local Audit Regional Audit
Existing standards and guidelines No Yes
Local Audit tool / form in existence No May be available
Extent of data collection May be one or two Will involve all localities
localities only across the Network
11 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 2018KEY STEPS TO DELIVERING A SUCCESSFUL
Section 2
AUDIT
2.2 PLANNING THE AUDIT
This is a crucial stage. Following the annual spring review meeting and allocation of
the Lead clinician(s) and StRs/ specialty doctors, a planning meeting should be
organised. This is usually the responsibility of the senior StR. It is strongly advised
that everyone in the group attends. The group will then be known as the Guideline
Development Group (GDG). The audit project forms part of the Guideline
Development process. Everyone in the group should read both this Audit Manual
and the Guideline Development Manual1 thoroughly.[Link] These two documents
provide important information about how the process should be undertaken and the
key steps along the way. In order for the final standards and guidelines to be NICE
accredited it is essential that they are produced by following the clearly outlined
process. There is specific paperwork that needs to be completed and declarations
regarding Conflicts of Interests for each member need to be submitted.1 [Link] Any
concerns should be communicated to the Chair of the Audit Subgroup as soon as
possible.
2.3 MEMBERSHIP OF THE GUIDELINE DEVELOPMENT GROUP
Firstly it is important that the group has representation from a range of settings and
professionals, and that the members are committed to the project. If there is a
problem with membership the lead clinician should contact the Chair of the Audit
Subgroup as soon as possible. It is also very important that patient, carer or public
involvement is considered at an early stage. Further guidance about this is found in
the Guideline Development Manual. 1[Link]
. 2.3.1 Defining the Scope of the Guideline and Audit
The group needs to define a clinical question(s). This process will have started
at the review meeting but now needs to be developed. There is more
information on how to do this in the Guideline Development Manual.1 [Link] if
your group has problems with this it is important you contact the Chair of the
Audit Subgroup who will be able to assist.
If this is a local (new) audit then you will need to determine your draft standards
prior to developing your data collection proforma. A regional re-audit will have
existing standards as a basis for the data collection.
Tips
Know what type of audit is expected so that you can plan accordingly i.e.
Local audits – one/two localities.
Regional – open to all localities in the Network.
Supra-regional – open to all localities in the Network and other invited localities.
12 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 2018KEY STEPS TO DELIVERING A SUCCESSFUL
Section 2
AUDIT
2.3.2 The Systematic Literature Review
A literature search is a key step in the audit process and the Guideline
Manual.[Link] gives extensive information on how to do this.1 It is important that
when planning your project you allow sufficient time for this stage. It should
involve the majority of the GDG membership.
There needs to be a relevant and concise summary of the literature review
presented at the audit meeting.
2.3.3 The Invited Expert
Audit projects and the presentation benefit from the input of an invited expert. It
is important to think about this at an early stage and approach the professional
who would be most appropriate to help with your topic.
2.3.4 Managing the Process and Accessing Support
Draw up a Gantt chart6 to plan the audit project. This helps you meet all the
required deadlines. Appendix 5 contains an example of an audit project Gantt
chart.
It may be that you would benefit from additional support for your audit. This
may take the form of general advice; help with the literature review, advice on
survey design or data analysis. Support and help may be accessed in several
ways. Please see section 6 for more details.
Figure 2 illustrates the important components of the planning phase.
13 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 2018KEY STEPS TO DELIVERING A SUCCESSFUL
Section 2
AUDIT
Figure 2 The Planning Phase
Membership
and allocated
roles
Timing of data Think about
collection additional
support/
resources
Think about Key elements
the invited in planning a Understanding
expert early successful of the process
audit
Form/
questionnaire Gantt Chart.
design Plan meetings
Define the
clinical
questions
2.4 SURVEY AND AUDIT FORM DESIGN
The audit form / survey is key for collecting the data and needs to be designed with
great care. If there are existing standards these should be used in the design. The
form should include some basic demographic questions. It is important that the
locality and the care setting are identified. It may also be important to know the role
of the professional responding to the survey.
GDGs may use an electronic method e.g. Survey Monkey or paper. The Audit
subgroup has access to an enhanced Survey Monkey account. This allows you to
design a questionnaire with more than 10 questions and use the upgraded facilities.
If you wish to use this account you should contact the Audit Subgroup Chair for log in
details.
14 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 2018KEY STEPS TO DELIVERING A SUCCESSFUL
Section 2
AUDIT
Questionnaires need to be simple, easy to read and unambiguous as data may be
collected by individuals not directly involved in the GDG. Clinical data collection
forms must be straightforward. Better response rates are usually obtained from
shorter forms. We would strongly advise that you avoid using paper forms as this
adds to the workload when inputting/analysing data. They should only be made
available in exceptional circumstances. If using paper the form should ideally be no
more than two sides of A4.
Examples of forms that have been designed for previous audits are available on the
Network website.[Link] As each set of standards and guidelines goes through the
NICE process the website will add the audit data collection form as supporting
evidence.
Decide who you will send the questionnaire out to, which settings and how many
responses you would want from each area. You will also need to determine how
many responses are required for a team or unit to receive a localized report (see
section 2.7 for further details).
To prevent the overlap of multiple data collection periods for the different audits you
should inform the StR Audit coordinator of the proposed time scale for data collection
at the earliest opportunity. They can then ensure this does not overlap with other
projects. On some occasions it may be necessary to alter timings slightly to avoid
any clashes.
Once the whole GDG has reviewed the form, it is recommended that you pilot the
form with someone outside the project group before submitting it for review by the
Chair of the Audit Subgroup (see below).
If it is a regional audit you will need to draft a covering letter to the Leads of all the
Integrated Clinical Networks and to the Audit subgroup member from each locality.
Both of these people are responsible for encouraging teams and units within their
locality to actively participate in the audit data collection and so they need to be well
informed about your project.
If it is a local audit there will be a need to be a letter to the Lead of the Integrated
Clinical Network and subgroup members for the areas involved in data collection.
This letter should give clear instructions about:-
The audit project e.g. title / topic/ clinical question(s)
The data collection period with the deadline for returning all forms
How many submissions you are expecting as a minimum and the number
required to receive a localized report
Who you want to complete the form i.e. which healthcare professionals
If using a paper form where these should be returned to
You should also include a contact name and email address in case there are
any questions or problems
15 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 2018KEY STEPS TO DELIVERING A SUCCESSFUL
Section 2
AUDIT
Once you have the final draft of both the form and the covering letter you should
send to the Chair of the Audit Subgroup and StR Audit Coordinator along with the
completed proposal form and all of the Conflict of Interest statements (see Guideline
Development Manual1). [Link]
Drafts should be sent at least 4 weeks before the start of the proposed data
collection period to give time for review and amendments.
If registering with the Marie Curie Palliative Care Institute Liverpool you will need to
complete the one page Institute protocol (See Appendix 6) and submit to the
Research/Development Administrator and the Research/ Development Lead for
review. Please see guidance in section 5 on which audits to register with the Institute
and how to do this
Top Tips
Plan effectively – a Gantt chart may help with this.
Questions included on the audit form should help answer the overall question “are we
doing what we should be doing?” Don’t collect too much data!
Pilot your form – preferably using someone not involved in its creation.
Send your form to the Audit Coordinator 4 weeks before your data collection period
to allow review and comments by the Audit Sub Group.
Factor in time to get any local Clinical Audit/Governance approval prior to data
collection.
Factor in time to get any local Clinical Audit/Governance approval prior to data 2.5
DATA COLLECTION
collection
When approval has been given by the Audit Subgroup Chair, send the approved
form or questionnaire link and the covering letter to the Audit Coordinator who will
then arrange for it to be circulated to the appropriate people.
Individual units/teams participating in an audit will sometimes need to get agreement
from their Trust/ Audit committee before they can go ahead. Different audit
departments will have differing stipulations about data collection. The lead StR, with
support from the lead clinician, should discuss with the individual units/teams about
how this issue is best managed. It is important that the GDG helps with advice on
completing the relevant local form(s) so that all the information is included. You need
to factor in the time required for this when planning your project.
16 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 2018KEY STEPS TO DELIVERING A SUCCESSFUL
Section 2
AUDIT
It is important to keep track of the number of responses received from each
individual unit /team. This is easily done by logging into the Survey Monkey account.
A reminder email should be sent halfway through the data collection period and
again two weeks from the final submission deadline. You should forward the draft
reminder emails to the StR Audit Co-ordinator for circulation.
2.6 ANALYSIS OF DATA
Try and avoid using paper data collection forms. They tend to simply increase the
workload. You should only issue paper forms to units/teams in exceptional
circumstances. Data from paper forms will need to be recorded using an appropriate
software package e.g. SPSS or Excel. Survey Monkey has its own analysis section,
but data may be downloaded and inputted into other software if needed.
The type of analysis to be used should be identified at an early stage as it influences
both the type and amount of data collected. Descriptive analysis is the most
commonly used approach.
There may be useful qualitative information collected and this should be analysed
thematically. Using comment boxes for free text on the form is one way of collecting
this kind of data.
The results should be summarized in the final audit presentation. It is important to
think about the key messages that the audience need to be aware of. Not all of the
graphs produced will need to be included in the final presentation although they may
well be of use in future abstracts, articles or presentations and will of course be
available in the final localized reports.
2.7 PRODUCING LOCALISED REPORTS
Individual units are increasingly being asked to demonstrate their results from a
regional audit. In 2015 we conducted a pilot of localized reporting with two of the
regional audits and this process will be further developed in 2016. A minimum
number of returns will be required. The exact figure will be different for each audit
and will be determined by the GDG. Further details of how this will be delivered will
be made available on the Network website and at the bimonthly meetings during the
year. GDGs will not need to produce the localized reports. This will be done by the
Audit Subgroup. However GDGs will need to organize their data collection so that it
is possible to produce meaningful reports. Advice on this can be obtained through
the StR Audit Co-ordinator.
17 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 2018THE AUDIT PRESENTATION
Section 3
The audit presentation meeting is held at the Marie Curie Hospice in Liverpool.
Details of the location can be found in Appendix 7.
The date for the presentation of the audit will have been decided at the preceding
spring review meeting. Ideally all members of the group should contribute to the
presentation so it is worth thinking about which areas of the work different members
would like to present. An electronic copy of the presentation and any files containing
hand-outs for the meeting should be sent to the StR Audit Coordinator seven days
before the audit meeting.
The presentation is usually 40-45 minutes followed by 30 minutes for discussion.
The presentation should include:-
Summary of the clinical questions.
Outline of the current standards and guidelines (if they exist).
Summary of the systematic literature review. There needs to be a relevant and
concise summary of the review presented at the meeting.
Description of the method(s) used.
Summary of patient and public involvement.
Audit results.
Proposed new standards and guidelines.
Comments/ overview from invited expert.
Following the presentation the GDG are invited to help with answering any questions
from the audience and contribute to the discussion.
At the end of the presentation and discussion the group will be asked to take all of
the comments away to produce a final draft of the standards and guidelines to
present at the next audit meeting i.e. in 2 months. If this is not possible the lead StR
should discuss with the Audit Co-ordinator about rescheduling. GDGs should note
that the revised standards and guidelines do not need to be completely
formatted in order to be presented. Full details about the development of the
standards and guidelines can be found in the Guideline Development Manual.[Link]1
Tips
When planning the questions for your audit form always think “how am I going to analyse this”
to help guide the kind of question you will use.
Results presented at the meeting should support your assessment in whether we are meeting
the standards or not.
Try and involve everyone in the presentation.
Make sure you send the presentation to the Audit Co-ordinator before the big day!
Try and finalise your standards and guidelines as soon after the presentation as you can whilst
the memory remains fresh.
18 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 2018DEVELOPING THE STANDARDS AND
Section 4
GUIDELINES
A clinical guideline has been defined by NICE as “recommendations on the
appropriate treatment and care of people with specific diseases and conditions within
the NHS. Clinical guidelines are based on the best available evidence. Guidelines
help professionals in their work, but they do not replace their knowledge and skills” 7.
Good clinical guidelines aim to improve the quality of healthcare. They can change
the process of healthcare and improve peoples’ chances of getting as well as
possible. Clinical guidelines can:-7
Provide recommendations for the treatment and care of people by health
professionals
Be used to develop standards to assess the clinical practice of individual health
professionals
Be used in the education and training of health professionals
Help patients make informed decisions
Improve communication between patient and health professional
The methodology for the production of clinical guidelines has evolved over the past
decade. The Guideline Development Manual located on the Network website 1 [Link]
details the production and review of the standards and guidelines in line with current
best methodologies and aims to meet the NICE criteria for guideline production.
Information in the manual includes:-
Defining the scope of the guideline
Establishing the evidence
Reviewing and grading of evidence
Development and grading of recommendations
Consultation and external review
Format and content
Dissemination and Implementation
Comprehensive list of resources and supporting documentation
Tips
Read the Network Guideline Development Manual 1 in conjunction with this guide. It
contains valuable information.
Electronic versions of the various forms are available on the Network website.
19 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 2018HOW TO ACCESS SUPPORT
Section 5
5.1 SUPPORT FOR REGISTRARS
For Specialty Registrars (StRs) leading audits whilst resident at the Marie Curie
Hospice and the Royal Liverpool and Broadgreen University Hospital (RLBUHT)
Palliative Care Team, the Marie Curie Palliative Care Institute( MCPCIL) will offer
support in the design, development and analysis phases of the audit project.
The StR must complete a one page protocol (See Appendix 6) and subsequently a
full protocol - indicating the support required and expected outputs (e.g. abstracts
and publications).
If a StR is leading an audit and would like support but working outside the Marie
Curie Hospice and the RLUBHT Palliative Care Team they should submit a one
page protocol to the MCPCIL R&D lead for consideration (See Appendix 6). If
accepted, a full protocol should be developed, identifying expected outputs (e.g.
abstracts and publications).
For a StR leading an audit and working in units other than the Marie Curie Hospice /
Palliative Care Team and the RLUBHT, and who does not wish to have support,
there is no obligation to register the project. Support, advice and signposting may be
accessed through the Audit Subgroup Chair and StR Audit Co-ordinator. See Figure
5.1 for an overview of the process.
5.2 ANNUAL TRAINING PROGRAMME
The Audit Subgroup has developed an annual training programme to help members
participate in the work and to deliver the regional audits and guidelines. All members
of the group are encouraged to attend the different sessions. StRs should do their
best to ensure they attend all of the sessions during their training (see Table 5.1).
There may be additional ad hoc sessions throughout the year including an
Introduction to the Audit programme delivered in September. Further details will be
made available on the Audit website. CPD accreditation points have been applied
for.
Table 5.1 Details of Annual Training Programme
May Essential Skills 1: Literature Searching
July Essential Skills 2: Sifting through the evidence - refining
search results
September Essential Skills 3: Grading the evidence - making
recommendations from the best available evidence
November How to audit effectively and analyse with Survey
Monkey/Localized reporting
Tips
If unsure – seek help early, there are many people that can either help or know someone who can.
20 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 201821 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 2018HOW TO ACCESS SUPPORT
Section 5
Figure 5.1 Support Offered by Marie Curie Palliative Care Institute Liverpool for Audit Projects
StR leading on Audit
StR based at either StR NOT based at either
MCHL or Palliative Care MCHL or Palliative Care
Team @ RLBUHT Team @ RLBUHT
Support Required Support NOT Required
Complete MCPCIL one page
protocol and full protocol indicating
support required and expected If accepted
outcomes. This should also be
copied to the Audit Subgroup Chair Submit MCPCIL one Book appointment No action needed,
page protocol to with ‘support and there is no
R&D lead for surgery’ to obtain expectation of
Support offered by MCPCIL: consideration advice on all aspects support from
Audit design of audit MCPCIL
Audit development
Audit Analysis
Poster productionDISSEMINATION
Section 6
Following the presentation remember to write and thank your invited expert for
his/her help and attendance. This is usually the responsibility of the lead StR in the
GDG.
Copies of the presentation will be made available on the Network website so that
other units may use as a teaching aid when disseminating the results from the audit
and implementing change. The website presentation will not include the actual audit
results. These can be accessed by contacting Kath Davies. The presentation on the
website will include the clinical question, methodology, literature review and the
draft standards and guidelines.
GDGs are strongly encouraged to disseminate the results of their work as widely as
possible. This may include abstracts/ posters for local, national and international
conferences. It may be appropriate to submit abstracts to non-palliative care
conferences. GDGs should also consider writing articles for both palliative and non-
palliative care journals.
Any posters or publications arising from the work should acknowledge the
contribution of the Cheshire and Merseyside Palliative and End of Life Care
Network Audit Group. For poster format, this is usually by including the Network
group logo which can be obtained from the StR Audit Coordinator.
Details of successful abstracts, presentations or articles should be sent to the Chair
of the Audit Subgroup and will be made available on the website.
Tips
Identify your conference early.
Write the abstract once data analysis is concluded – conference abstracts follow a
similar structure.
The Guideline Development Manual has suggestions of other ways to help dissemination
and implementation of your work.ACCESSING THE STANDARDS AND
Section 7
GUIDELINES
Every three years between 1998 and 2010 the group produced a book which
collated the standards and guidelines It was designed both for the purposes of
education and to facilitate the production of other standards and guidelines. It
aimed to encourage the setting and monitoring of standards and promote clinical
excellence in end of life care. It was intended as an information resource for
qualified medical staff and other health professionals caring for patients where
palliative care is appropriate. The content was applicable to patients with cancer
and other types of advanced disease. The guidelines are not standard protocols
and are not intended to be prescriptive.
The fourth edition of the book was produced in 2010. Each chapter had been
extensively reviewed by members of the group and experts from across the United
Kingdom. Six new chapters were included: drugs at the end of life, interventional
pain techniques, major haemorrhage, spiritual care, substance misuse and urinary
incontinence. Standards and guidelines which had not been re-audited since the
3rd edition were updated and reviewed. There are levels of evidence throughout the
guidelines and grades of recommendation for all of the standards in each of the 43
chapters.
The fourth edition was the last paper copy produced. At the moment there are no
plans to produce a further paper edition. However all of the standards and
guidelines are available on the audit website. [Link] Professionals are able to sign
up for email alerts.These will notify you when new guidelines are uploaded or
changes made to the website. [Link]
The website has four main areas:-
1. Introduction to the Audit Group.
2. Standards and Guidelines.
3. Details of annual programme and meetings.
4. Manuals for Audit and Guideline Development plus educational resources.
Appendix 8 contains a list of all the audit topics that have been undertaken by the
group.
Copies of the 2010 book can be obtained through the Marie Curie Palliative Care
Institute Liverpool website [Link]
Details of each of the annual audit programmes since 2010 can be found on the
network website.[Link]
24 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 2017REFERENCES & RESOURCES
Section 8
[Link]
REFERENCES
1. Cheshire and Merseyside Palliative and End of Life Care Network Audit
Group. Guideline Development Manual. March 2014. Available from:
www.cmscsenate.nhs.uk. [Last accessed 05 February 2016]. [Link]
2. Cheshire and Merseyside Palliative and End of Life Care Network Audit
Group. Website. Available from www.cmscsenate.nhs.uk [Last accessed 05
February 2016].[Link]
3. Health Quality Improvement Partnership Criteria and Indicators of Best
Practice in Clinical Audit 2012. Available from:
http://www.hqip.org.uk/public/cms/253/625/19/186/HQIP-
Criteria%20and%20indicators%20of%20best%20practice%20in%20clinical%2
0audit-March%202012.pdf?realName=JlO6ff.pdf [Last accessed 05 February
2016] [Link]
4. Harris D. Differentiating clinical audit from service evaluation and research.
Eur J Palliat Care 2013: 20(3):133-135.
5. Joint Royal Colleges of Physicians Training Board. Specialty Training
Curriculum for Palliative Medicine. August 2010. (Amendments August 2012).
Available from: www.jrcptb.org.uk/trainingandcert/ST3
SpR/Documents/2010Palliativemedicinecurriculum(AMENDMENTS2012).pdf
[Last accessed 5 February 2016].[Link]
6. Joint Royal Colleges of Physicians Training Board. 2010 Palliative Medicine
ARCP Decision Aid. Available from: www.jrcptb.org.uk/trainingandcert/ST3-
SpR/Documents/2010PalliativeMedicineARCPDecisionAid.pdf [Last accessed
05 February 2016].[Link]
7. Department of Health: National Clinical Audit Advisory Group Gantt Charts.
Available from:
www.improhealth.org/fileadmin/Documents/Improvement_Tools/Gantt_Chart.p
df [Last accessed 05 February 2016].[Link]
8. National Institute for Health and Care Excellence. Clinical guidelines. Available
from:www.nice.org.uk/CG Updated 28 October 2011. [Last accessed 05
February 2016].[Link]
25 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 2017REFERENCES & RESOURCES
Section 8
ADDITIONAL RESOURCES YOU MAY FIND USEFUL
Note. New resources are also available on the audit webpages and will be
updated regularly [Link]
Royal College of Paediatrics and Child Health. Clinical Audit and e learning.
Available from:
http://www.rcpch.ac.uk/training-examinations/education/clinical-audit-e-learning.
[Last accessed1 6th February 2016] [Link]
Clinical Audit Support Centre. www.clinicalauditsupport.com
[Last accessed 16 February 2016]. [Link]
NHS England . Clinical Audit, Available
from:https://www.england.nhs.uk/ourwork/qual-clin-lead/clinaudit/[Last accessed
16th February 2016] [Link]
Royal College of Physicians. Quality improvement for revalidation. Available from:
https://www.rcplondon.ac.uk/education-practice/advice/quality-improvement-
revalidation-p-cat-tool [Last accessed 16th February 2016] [Link]
Clinical Audit Teaching Toolkit. Available from:
http://www.hqip.org.uk/downloadable-learning-resources/
[Last accessed 16 February 2016]. [Link]
26 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 2017APPENDIX 1
NICE ACCREDITATION PROCESS
Accreditation helps health and social care professionals identify the most robustly
produced guidance available, enabling them to deliver high quality care. Since 2009
the National Institute for Health and Care Excellence have accredited 60 guidance
development processes.
What is accreditation?
The accreditation programme assesses the processes used to produce guidance
and advice. This will, in turn, help raise standards in guidance production.
What are the benefits of accreditation?
To help health and social care professionals identify the most trusted sources of
guidance developed using critically evaluated high quality processes.
To drive up the quality of information used by health and social care
professionals in decision making.
To improve patient outcomes through providing robust evidence for NICE
quality standards.
Why apply for accreditation?
To Increase the visibility of guidance and advice.
To attain accreditation for guidance, making it eligible to be considered for the
development of NICE quality standards.
To promote the guideline development processes as being robust and
transparent.
To use the NICE Accreditation Mark as an indication of high standard, good
quality information.
The following table outlines the 6 domains and criteria required for a guideline
development process to be NICE accredited. The process must meet all of the
criteria. Further information can be found on the NICE website
http://www.nice.org.uk/accreditation.
27 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 2017APPENDIX 1
ACCREDITATION DOMAINS AND CRITERIA
DOMAIN CRITERIA
1. Scope and purpose is concerned with These criteria consider whether the guidance producer has a
the overall aim of the guidance, the specific policy in place and adhered to that requires them to explicitly
health questions and the target population. detail:
1.1 The overall objective of the guidance.
1.2 The clinical, healthcare or social questions covered
by the guidance.
1.3 The population and/or target audience to whom the
guidance applies.
1.4 That the producer ensures guidance includes clear
recommendations in reference to specific clinical,
healthcare or social circumstances.
2. Stakeholder involvement focuses on These criteria consider whether the guidance producer has a
the extent to which the guidance represents policy in place and adhered to that means it includes:
the views of its intended users and those
2.1 Individuals from all relevant stakeholder groups including
affected by the guidance (patients and
patients groups in developing guidance.
service users).
2.2 Patient and service user representatives and seeks
patients views and preferences in developing
guidance.
2.3 Representative intended users in developing
guidance.
3. Rigour of development relates to the These criteria consider whether the guidance producer has a
process used to gather and synthesise clear policy in place and adhered to that:
information and the methods used to
3.1 Requires the guidance producer to use systematic
formulate recommendations and update
methods to search for evidence and provide details
them.
of the search strategy.
3.2 Requires the guidance producers to state the criteria
and reasons for inclusion or exclusion of evidence
identified by the evidence review.
3.3 Describes the strengths and limitations of the body
of evidence and acknowledges any areas of
uncertainty.
3.4 Describes the method used to arrive at
recommendations (for example, a voting system or
formal consensus techniques like Delphi
consensus).
28 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 2017APPENDIX 1
DOMAIN CRITERIA
3.5 Requires the guidance producers to consider the
health benefits, side effects and risks in formulating
recommendation.
3.6 Describes the processes of external peer review.
3.7 Describes the process of updating guidance and
maintaining and improving guidance quality.
4. Clarity and presentation deals with the These criteria consider whether the guidance producer ensures
language and format of the guidance. that:
4.1 The recommendations are specific, unambiguous
and clearly identifiable.
4.2 The different options for management of the condition
or options for intervention are clearly presented.
4.3 The date of search, the date of publication or last
update and the proposed date for review are clearly
stated.
4.4 The content and style of the guidance is suitable for
the specified target audience. If the public, patients
or service users are part of this audience, the
language should be appropriate.
5. Applicability deals with the likely These criteria consider whether the guidance producer routinely
organisational, behavioural and cost consider:
implications of applying the guidance.
5.1 Publishing support tools to aid implementation of
guidance.
5.2 Discussion of potential organisational and financial
barriers in applying its recommendations.
5.3 Review criteria for monitoring and/or audit purposes
within each product.
6. Editorial Independence is concerned These criteria consider whether the guidance producer:
with the independence of the
6.1 Ensures editorial independence from the funding body.
recommendations, acknowledgement of
6.2 Is transparent about the funding mechanisms for its
possible conflicts of interest, the credibility
guidance.
of the guidance in general and their
6.3 Records and states any potential conflicts of interest of
recommendations in particular.
individuals involved in developing the recommendations.
6.4 Takes account of any potential for bias in the conclusions
of recommendations of the guidance.
29 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 2017APPENDIX 2
OVERVIEW OF CHESHIRE AND MERSEYSIDE PALLIATIVE AND END OF LIFE CARE NETWORK STRUCTURE
30 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 2017APPENDIX 3
JOB DESCRIPTION FOR STR AUDIT CO-ORDINATOR ROLE
Outline of Role
There are four main aspects to the role:-
1. Co-ordination of the annual regional audit programme.
2. Organizing the bi-monthly regional clinical audit meetings.
3. Membership of the Audit Subgroup.
4. Liaison with specialty registrars.
Details of Responsibilities
1. Coordination of the annual regional audit programme
You will need to have a record of who is leading each Guideline Development
Group (GDG) and the membership of each. This information should be forwarded
to Kath Davies so the website can be updated. Traditionally this has been recorded
as ‘the matrix’, but with the website we have tried to improve the formatting. Lists
for 2014/15 onwards are on the memory stick.
You will also need to know where each GDG is up to in their process, and in
particular when they are planning to data collect. We would suggest emailing
each StR Lead once a month to get an update.
The StR leads should send their data collection tool to you along with a draft of their
covering letter. You should check both for any obvious errors or omissions. You
should make sure that all of the different units and teams are included.
Then send data collection form to the Chair of Audit Subgroup for approval.
Return the data collection form to the StR lead with clear guidance on any changes
required.
It finally comes back to you along with the covering letter. You then send out to the
ICN Lead and Audit Subgroup representative for each locality. The list of email
addresses is on the audit stick in the folder.
At the same time as submitting their data collection tool, the GDGs should also
submit their proposal form. This should be reviewed by you, then forwarded to the
Chair of the Audit Subgroup, who will then circulate to members of the subgroup for
comment and approval
31 Cheshire and Merseyside Palliative and End of Life Care Network Audit Group Manual
Production Date: March 2016 Review Date: March 2017You can also read
