To Annotate or Not to Annotate: Moving to paperless research ethics administration at UTAS
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
2/12/2013
To Annotate or Not to Annotate:
Moving to paperless research ethics
administration at UTAS
Paula Swatman, Katherine Shaw & Nicola Hodgman
Office of Research Services
University of Tasmania
Ethics Review Framework
2002 UTAS & DHHS establish (Australia first) state-wide human ethics review
body to streamline review of all research undertaken in Tasmania: HREC
Tasmania Network administered by UTAS. Comprises 2 HRECs:
Social Science HREC (SSHREC) – 18 members – reviewed 374 (full + min risk) new
applications 2012
Health and Medical HREC (H&M HREC) - 15 members – reviewed 207 (full + low risk)
new apps in 2012; and includes review by;
Sub-committee: Scientific Review Advisory Committee (SRAC) – 11 members
Animal Ethics Committee (UTAS only) – 13 members - reviewed 60 new apps
in 2012
Paperless ethics @ UTAS AEN 2013 2
12/12/2013
Financial benefits
Cost items
Costs 2011 2013
Postage $2,841 iPads $38,220
Keyboards &
Printing $26,689 covers $7,800
Copier lease $25,197
Tech support/
Stationery $4,180 Deployment $15,600
Labour $16,380
NB: one-off
Total $75,287 $61,620
payment
Avge of 18 (committee members + ex officio & admin) x 12 (meetings
p.a.) x 4 (committees in total) = 864 agendas p.a. (approx.)
Plus admin cost associated with compiling & producing agendas (approx.
2 days per month per committee) = 2 x 12 x 4 = 96 days per annum @
HEO4 = $16,380
Paperless ethics @ UTAS AEN 2013 3
Other Benefits
Efficiencies: time previously required for printing now used to benefit:-
Researchers: shorter time from submission to review
Committee members: additional days for review
Labour: a genuine saving of 96 days p.a.
Environmental: a physical saving of 126 reams paper p.a.
Space: project files for 950 active projects now available electronically!
Paperless ethics @ UTAS AEN 2013 4
22/12/2013
Process
UTAS management decision to discontinue paper supply of agenda materials
but commitment to supporting members who may (not) have existing hardware
/ resources to access e-Papers
Investigation of electronic hardware, software and delivery mechanisms
iPads + keyboard/cover with GoodReader or iAnnotate app to mark up agendas
Agendas pushed out via secure UTAS wiki page (one per committee) – email
notification to members when new papers available
Non-UTAS committee members given access to UTAS IT Services
Members given IT support to set up iPads and workshop to demonstrate
annotation software.
Paperless ethics @ UTAS AEN 2013 5
Is paperless ethics processing working?
A little over a year after paperless ethics processing was introduced at UTAS,
we felt it was time to see how well the new approach was working
As both the Chair and EO of the Social Science HREC are Information Systems
people, it wasn’t hard to decide who should do the review
So we designed a research project which would:
Gather consistent information from all present and recent members of all 4 ethics
committees – via an online survey
Gather rich information from all ethics committee EO’s – via interviews
Test findings against the most suitable theoretical IS acceptance model (UTAUT)
The project design was reviewed by 2 experienced IS academics from 2 other
universities – their suggested revisions have been implemented
Paperless ethics @ UTAS AEN 2013 6
32/12/2013
The UTAUT model
Unified Theory of Acceptance & Use of Technology (Venkatesh et al. 2003)
Paperless ethics @ UTAS AEN 2013 7
Approving the study
The hardest question we faced was: who should approve the ethics
application for this project?
It was clearly a low risk project – but …
Every ethics committee at UTAS would be affected by it
We eventually decided to complete a full ethics application and submit it to
the SSHREC
The project’s researchers recused themselves from the discussion
The Deputy Chair (who was about to retire) handled the review
Paperless ethics @ UTAS AEN 2013 8
42/12/2013
The Survey
The survey questionnaire was made available via Survey Monkey and members
received one reminder, three weeks after the initial mailout
31 ethics committee members responded - out of 71 present & past members
invited across 4 committees
Animal Ethics Committee – 6
Health & Medical HREC – 4
Scientific Research Advisory Committee – 4
Social Science HREC – 17
It’s probably fair to say the SSHREC’s being the committee to review the
application probably had some influence on this somewhat biased response
Paperless ethics @ UTAS AEN 2013 9
IT backgrounds and experience
We wanted to understand members’ technical background and level of
expertise – these graphs summarise the results
Paperless ethics @ UTAS AEN 2013 10
52/12/2013
Accessing meeting agendas via wiki
Members generally saw the wiki as easy-to-use and a good way of accessing
meeting agendas, e.g.
“It is easier than I expected, and there is always a student or someone else around the
Uni who can help. When documents came as attachments, I did not know how to get
them into the iAnnotate format - a nuisance”
Technical issues at the University level did cause some problems, e.g.
“Good except when UTAS IT randomly disallowed my access”
Members mostly found the wiki-based agenda delivery preferable to the previous
paper mountain, e.g.
“Apart from the environmental side, easier to handle, flick through and back on the
agenda. Less weight. No paper cuts, YES! easier to take anywhere to utilise free time to
review applications”
“It is more 'immediate', I download it when is practicable for me, it is not as heavy to
carry with me, which is also practical when do this from an interstate or overseas
location. As such, it is therefore very ‘time-friendly’
Paperless ethics @ UTAS AEN 2013 11
Hardware platform is not the crucial issue
Committees have varied in their use of hardware platforms: 2 are predominantly iPad users; 1 uses
laptops exclusively – and the 4th uses a mix of iPads and laptops
There was little criticism of the iPad tablet – most respondents seemed to find it intuitive and many
chose to use it for other applications as well
“The portability of the iPad in reviewing documents is important; this year I have twice read and commented
on the applications in airports! As committee members are volunteers who are fitting the application reading
into already busy lives, the portability is an important consideration”
“I think it is more confidential too as there is no chance of leaving the papers around on a desk and comments
can be made on the papers, erased, changed, notes put on them and these can be forwarded to the
Committee if I cannot be at the meeting”
Paper did rate one positive – when we asked members to compare the old and new approaches,
several respondents highlighted the ease of flicking back and forth in paper documents
“Only paper advantage is to easily compare (for example) a main application section with a consent form
much farther into the same application”
If we were doing this again, we would probably place less emphasis on one particular hardware
platform …
Paperless ethics @ UTAS AEN 2013 12
62/12/2013
Yes, the iPad is cool …
but meeting preparation is complex
Committee members are using their iPads for lots of activities – but they
are not the only choice for preparing meeting material …
Paperless ethics @ UTAS AEN 2013 13
Annotation software does matter: get
this right!
The one really big negative was our first annotation tool (Good Reader) which
almost nobody liked (me included!)
“It's important to have reliable mark-up software; I started by using GoodReader as this
was what was suggested in the iPad training provided for new committee members and
what we downloaded in the training session. It was fine for the first month, then in the
second month all my annotations suddenly moved to the wrong pages, which was very
annoying and time-consuming to fix. The chair of the committee said that GoodReader
was a 'flaky' piece of software and she suggested downloading and using iAnnotate,
which has proved problem free …”
Our subsequent switch to iAnnotate made a big difference to many members – but
not everyone was convinced of the benefits of annotation software
Pro: “Marking is easy, returning to bookmarks makes use of the iPad very easy during
the meetings. Navigation is easy.”
Con: “I felt uncomfortable using the 'mark up' features on PDF's so I always printed them
out anyway”
Paperless ethics @ UTAS AEN 2013 14
72/12/2013
Training – once really is never enough
2.5 Views on training were somewhat
2 mixed – this was a typical comment
How satisfied were you with the training you received?
1.5 “An interactive follow up after a
1
couple of meetings would have
made it easier. It is only once I
0.5
actually use an application that I
0 learn its value. I am still not sure I
-3 -2 -1
-0.5
0 1 2 3 4
am using it to its full potential,
but I manage”
-1
Bearing in mind that ethics
-1.5
committee members are likely to
-2
be 50+ and few are IT experts …
-2.5 training is key!
How easy do you find it to use the iPad to review ethics
applications?
Paperless ethics @ UTAS AEN 2013 15
The role of influence
We wanted to discover how Do you feel you have been influenced to adopt (or not to adopt)
important a role influence played the paperless environment – especially the iPad – by any of the
in adopting paperless processing 30
groups involved in research ethics at UTAS?
It seems clear that 25 Strongly agree
20 Agree
encouragement (from Chair and/or 15
I'm not sure
Disagree
other members) 10 Strongly disagree
Support from the EO’s and other 5
technical people 0
I use the iPad because The Chair of my Support from the ethics
other members of my Committee is very staff has encouraged
did play a role in assisting uptake Committee also use supportive of the use of me to overcome
one the wiki and iPad problems in using the
But it also seems clear this is not system wiki and iPad
the whole story …
Paperless ethics @ UTAS AEN 2013 16
82/12/2013
The view from inside …
Interviews with ethics support staff have identified some fascinating (and
unexpected) issues:
Medical research very complex ethics applications with multiple follow-up
activities – and these can be difficult to manage on an iPad
Many of these members have switched to laptops which allow multi-tasking and are more
flexible
Some EO’s found annotation printouts really useful when members could not
attend meetings
Others felt the lack of expertise in using these complex software packages meant that
printouts were almost impossible to use
IMHO: members spend less time flicking through paper and more time actually
interacting
Eye contact is greater and discussions more valuable – I’m a fan!
Paperless ethics @ UTAS AEN 2013 17
What would we do differently?
Hardware is not as central as we originally believed
Yes, iPads are cool – but they may not be the ideal solution for everyone
Software is much more important than hardware!
Training is critical – not just in terms of initial training, but follow-up sessions
as well
EO’s will be providing on-phone support for several months after the switch to
paperless processing – so make sure they are trained first!
And if you do need to switch packages, cost in additional training there too
With hindsight, we would organise training in 2 stages:
Stage 1: basic training on hard/software – split this by IT expertise
Stage 2: training in using the hard/software for ethics reviewing – split this by
committee
Paperless ethics @ UTAS AEN 2013 18
9You can also read
