User Perceptions and Experiences of a Handheld 12-Lead Electrocardiographic Device in a Clinical Setting: Usability Evaluation - JMIR Cardio
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
JMIR CARDIO Wong et al
Original Paper
User Perceptions and Experiences of a Handheld 12-Lead
Electrocardiographic Device in a Clinical Setting: Usability
Evaluation
Kam Cheong Wong1,2,3,4, BEng, MBBS, MSc; Aravinda Thiagalingam1,2,5, MBChB, PhD; Saurabh Kumar1,2,5, MBBS,
PhD; Simone Marschner1, BSc, MSc; Ritu Kunwar1, MBBS, MPH; Jannine Bailey3, BSc, PhD; Cindy Kok6, MBBS,
MPH, PhD; Tim Usherwood1,2,7, BSc, MBBS, MD; Clara K Chow1,2,5,7,8, MBBS, PhD
1
Westmead Applied Research Centre, Faculty of Medicine and Health, The University of Sydney, Westmead, Australia
2
Westmead Clinical School, Faculty of Medicine and Health, The University of Sydney, Westmead, Australia
3
Bathurst Rural Clinical School, School of Medicine, Western Sydney University, Bathurst, Australia
4
School of Rural Health, Faculty of Medicine and Health, The University of Sydney, Orange, Australia
5
Department of Cardiology, Westmead Hospital, Westmead, Australia
6
Faculty of Medicine and Health, The University of Sydney, Sydney, Australia
7
The George Institute for Global Health, Sydney, Australia
8
Charles Perkins Centre, The University of Sydney, Sydney, Australia
Corresponding Author:
Kam Cheong Wong, BEng, MBBS, MSc
Westmead Applied Research Centre
Faculty of Medicine and Health
The University of Sydney
Level 6, Block K, Entrance 10, Westmead Hospital
Hawkesbury Road
Westmead, 2145
Australia
Phone: 61 2 8890 3125
Email: kam.wong@sydney.edu.au
Abstract
Background: Cardiac arrhythmias are a leading cause of death. The mainstay method for diagnosing arrhythmias (eg, atrial
fibrillation) and cardiac conduction disorders (eg, prolonged corrected QT interval [QTc]) is by using 12-lead electrocardiography
(ECG). Handheld 12-lead ECG devices are emerging in the market. In tandem with emerging technology options, evaluations of
device usability should go beyond validation of the device in a controlled laboratory setting and assess user perceptions and
experiences, which are crucial for successful implementation in clinical practice.
Objective: This study aimed to evaluate clinician and patient perceptions and experiences, regarding the usability of a handheld
12-lead ECG device compared to a conventional 12-lead ECG machine, and generalizability of this user-centered approach.
Methods: International Organization for Standardization Guidelines on Usability and the Technology Acceptance Model were
integrated to form the framework for this study, which was conducted in outpatient clinics and cardiology wards at Westmead
Hospital, New South Wales, Australia. Each patient underwent 2 ECGs (1 by each device) in 2 postures (supine and standing)
acquired in random sequence. The times taken by clinicians to acquire the first ECG (efficiency) using the devices were analyzed
using linear regression. Electrocardiographic parameters (QT interval, QTc interval, heart rate, PR interval, QRS interval) and
participant satisfaction surveys were collected. Device reliability was assessed by evaluating the mean difference of QTc
measurements within ±15 ms, intraclass correlation coefficient, and level of agreement of the devices in detecting atrial fibrillation
and prolonged QTc. Clinicians’ perceptions and feedback were assessed with semistructured interviews based on the Technology
Acceptance Model.
Results: A total of 100 patients (age: mean 57.9 years, SD 15.2; sex: male: n=64, female n=36) and 11 clinicians (experience
acquiring ECGs daily or weekly 10/11, 91%) participated, and 783 ECGs were acquired. Mean differences in QTc measurements
of both handheld and conventional devices were within ±15 ms with high intraclass correlation coefficients (range 0.90-0.96),
https://cardio.jmir.org/2021/2/e21186 JMIR Cardio 2021 | vol. 5 | iss. 2 | e21186 | p. 1
(page number not for citation purposes)
XSL• FO
RenderX
JMIR CARDIO Wong et al
and the devices had a good level of agreement in diagnosing atrial fibrillation and prolonged QTc (κ=0.68-0.93). Regardless of
device, QTc measurements when patients were standing were longer duration than QTc measurements when patients were supine.
Clinicians’ ECG acquisition times improved with usage (P
JMIR CARDIO Wong et al
Figure 1. Usability evaluation framework.
The study was conducted in an outpatient cardiology clinic and Handheld and Conventional 12-Lead ECG Devices
inpatient cardiology ward at Westmead Hospital, New South A handheld 12-lead ECG device (Cardio 300, custo med GmbH
Wales, Australia. The hospital setting was selected because [20]; Australian Register of Therapeutic Goods number 302423)
patients with various arrhythmias and cardiac diseases present (Figure 2) was selected because of its small size (length by
to the hospital and require ECG assessment. Patients (age ≥18 width by thickness: 11.5 cm × 7.5 cm × 1.8 cm; weight: 430 g)
years) and their clinicians were recruited. Patients who were and ability to transmit ECG data via Bluetooth. It was compared
too ill or unable to provide consent were excluded. Users were with the routinely used conventional 12-lead ECG machine at
clinicians who applied the device to acquire ECGs and patients each clinic site—Mortara Eli 280 (Welch Allyn Inc) in the
who were connected to the device. outpatient cardiology clinic and Mac 5500 (General Electric)
in the inpatient cardiology ward.
Figure 2. Handheld electrocardiography device and graphical interface on a laptop computer.
https://cardio.jmir.org/2021/2/e21186 JMIR Cardio 2021 | vol. 5 | iss. 2 | e21186 | p. 3
(page number not for citation purposes)
XSL• FO
RenderX
JMIR CARDIO Wong et al
Reliability Survey Questionnaire and Semistructured Interview
Device reliability [21] is defined as its ability to reproduce Patient’s satisfaction while connected to the devices were
measurements (QT interval, QTc interval, heart rate, PR interval, surveyed using 5-point Likert scale (1, strongly disagree to 5,
and QRS interval) consistently within an acceptable limit in a strongly agree). Before using the handheld device, clinicians
clinical setting. We chose QTc interval as the primary were asked whether they had previously used a similar device
measurement because of its importance in assessing prolonged (yes or no); if yes, we asked the name of the device. Using
QTc—which is defined as an individual QTc measurement ≥460 5-point Likert scales, the clinicians rated importance (1, strongly
ms in females and an individual QTc measurement ≥450 ms in disagree to 5, strongly agree) and satisfaction (1, not satisfied
males [5]. There are guidelines for an acceptable limit of to 5, very satisfied) with respect to accuracy, quality of the ECG
variation in QTc measurements [22]. A clinically noteworthy trace, ease of use, and efficiency. We also asked clinicians
change in QTc from baseline is defined as 30 ms to 60 ms whether they found the handheld device easier to use than the
[22,23]. We defined acceptable limits of mean difference for routinely used conventional ECG machine, whether using the
QTc as within ±15 ms. Both devices produced QTc readings handheld device in their current workflow could increase their
using the Bazett formula [24]. Agreement between devices’ productivity, and assuming they had continual access to the
automatic ECG interpretation algorithms for diagnosing atrial handheld device, whether they intended to use it. In
fibrillation was also assessed. semistructured interviews, we asked clinicians if they found the
handheld device useful and to explain their response, if their
Efficiency needs were met when using the handheld device (probing
Efficiency was measured as the total time taken by the clinician questions: what were their needs and what could the device do
to place electrodes on a patient and acquire the first ECG trace. to better serve their needs), and if the handheld device could be
A research assistant measured the time taken by the clinician applied for clinician-led mass screening (probing question: how
to place electrodes while the patient was supine. The electrodes to make the device suitable for clinician-led mass screening?).
were left in place on the patient during device change over. The
time taken by the clinician to connect each ECG device to the Statistical Analysis
electrodes and acquire the first ECG trace was measured Demographic data are presented using descriptive statistics.
separately for each device. Reliability, in terms of agreement between devices in diagnosing
atrial fibrillation and prolonged QTc, was assessed using the κ
User Satisfaction statistic [21], for which κ=0.41 to κ=0.60 is generally considered
Clinician satisfaction was measured using 5-point Likert scales to demonstrate moderate agreement and κ>0.61 considered to
(for device accuracy, quality of ECG traces, ease of use, and demonstrate good agreement. Within- and between-device
efficiency). In addition, semistructured interviews were reliability in QTc measurements was assessed using the
conducted for feedback and to assess clinicians’ acceptance of intraclass correlation coefficient (ICC); ICC ≥0.7 demonstrates
the device; semistructured interview guides were based on the good reliability [21]. The within-device variability for QTc was
Technology Acceptance Model [25,26] (ie, ease of use, assessed by the difference in QTc in the first and second ECG
perceived usefulness, and intention to use the device). Patient acquired immediately one after another by the same device on
satisfaction with the device connected to them was assessed the same patient. The between-device variability was assessed
using a 5-point Likert scale. by the difference in QTc in the first ECG produced by the 2
devices. The between-device variability over a range of QTc
Sample Size
intervals was examined using Bland-Altman plot [28]. The
The required sample size of patients was calculated (Sealed within-device and between-device variabilities in QTc compared
Envelope, Sealed Envelope Ltd) based on the primary objective with the predefined acceptable limits of ±15 ms were examined
to evaluate device reliability using within- and between-device by plotting the mean of the differences in QTc and their 95%
variabilities in producing QTc measurements within the confidence intervals in forest plots. Similarly, within-device
predefined acceptable limit of ±15 ms. With a power of 80% and between-device variability in other key ECG parameters
and α=5%, the required sample size was 96, which we rounded (QT interval, heart rate, PR interval, and QRS interval) were
up to 100. examined using forest plots.
Data Collection Procedures The differences in clinicians’ ECG acquisition times using the
A research assistant provided in-person training to clinicians to devices were assessed using a scatter plot. A logarithmic
demonstrate how to use the handheld device to acquire ECGs. transformation was used for the frequency of usage because the
In addition, clinicians could opt to watch a short introductory time difference due to a unit of increment of usage from first
video about how to use the handheld device. For each patient, to second usage was not proportional to a unit of increment in
ECGs were acquired twice while supine and while standing subsequent usages (ie, the change in time difference levelled
using each device. ECG acquisition order was randomized with off as the frequency of usage increased). These time differences
block sizes of 2 and 4 using SAS (version 9.4; SAS Institute). were analyzed using linear regression analysis. The impact of
The randomized sequences, participants’ demographic data and the randomized sequence of using the devices on the ECG
ECG parameters (QT, QTc, heart rate, PR and QRS) were acquisition times was analyzed using a 2-tailed t test.
recorded in REDCap [27].” Quantitative data were analyzed using SPSS statistical software
(version 25; IBM Corp), except linear regression analysis, which
https://cardio.jmir.org/2021/2/e21186 JMIR Cardio 2021 | vol. 5 | iss. 2 | e21186 | p. 4
(page number not for citation purposes)
XSL• FO
RenderX
JMIR CARDIO Wong et al
was performed using R software (R Foundation for Statistical Ethics
Computing). Normality of distribution was assessed with a The study was approved by Western Sydney Local Health
Shapiro-Wilk test [29]. Nonparametric testing (Wilcoxon signed District Human Research Ethics Committee (ethics approval
ranked test using the Hodges-Lehman method to compute 95% number 5929).
CI of the median difference) was used for nonnormal
distributions. A P value
JMIR CARDIO Wong et al
Table 1. Characteristics of the patients.
Characteristic Total (n=100)
Sex, n (%)
Male 64 (64)
Female 36 (36)
Age (years)
Mean (SD) 57.9 (15.2)
Range 18-88
Median (IQR) 61.0 (20.0)
a
Preexisting morbidity , n (%)
Heart diseases
Ischemic heart disease 27 (27)
Cardiomyopathy 7 (7)
Valvular disease 6 (6)
Heart blocks 6 (6)
Pacemaker 6 (6)
Arrhythmia
Atrial fibrillation 14 (14)
Paroxysmal atrial fibrillation 3 (3)
Atrial flutter 3 (3)
Supraventricular tachycardia 1 (1)
Hypertension 63 (63)
Hypercholesterolemia 42 (42)
Diabetes 27 (27)
Medicationsa , n (%)
Antihypertensive medication 53 (53)
Anticoagulant/antiplatelet 47 (47)
Lipid lowering medication 43 (43)
Oral hypoglycemic 18 (18)
Diuretics 15 (15)
Antiarrhythmic medication 9 (9)
Insulin 3 (3)
a
The total exceeds 100% because many patients had more than 1 morbidity or took more than 1 medication.
A total of 11 clinicians (nursing staff: n=10; clinical trial to use the handheld device from the research assistant while the
coordinator: n=1) participated. Prior ECG experience was high, remaining 2 clinicians (clinicians 1 and 6) opted to watch a
with 10 of the clinicians routinely acquiring ECGs daily or short video demonstration. Clinicians 1, 2, 3, and 6 were nurses
weekly and 1 clinician routinely acquiring ECGs fortnightly. working in cardiology outpatient clinics, and they acquired
Among the clinicians, 8 were from outpatient cardiology clinics, ECGs daily and recruited 18, 16, 19, and 20 patient participants,
and 3 were from inpatient cardiology wards (Table 2). Most respectively.
clinicians (9/11, 82%) opted to receive a demonstration of how
https://cardio.jmir.org/2021/2/e21186 JMIR Cardio 2021 | vol. 5 | iss. 2 | e21186 | p. 6
(page number not for citation purposes)
XSL• FO
RenderX
JMIR CARDIO Wong et al
Table 2. Characteristics of the clinicians.
Characteristic Total (n=11)
All, n
Outpatient 8
Inpatient 3
Role, n (%)
Nurse 7 (63.6)
Nurse educator 3 (27.3)
Clinical trial coordinator 1 (9.1)
Gender, n (%)
Male 4 (36)
Female 7 (64)
a
How often do you acquire an ECG ? n (%)
Daily 6 (55)
Weekly 4 (36)
Fortnightly 1 (9)
Monthly 0 (0)
Have you used the 12-lead handheld ECG device before? n (%)
Yes 0 (0)
No. If no, have you used similar device before? n (%) 11 (100)
Yesb 1 (9)
No 10 (91)
Number of patients recruited per clinician
Mean 9
Median 6
Range 1-20
a
ECG: electrocardiography.
b
Clinician 10 had used wireless electrocardiography before but not the handheld device used in this study.
A total of 783 ECGs were collected (Figure 3). The randomized being connected first for 52 patients and the conventional ECG
sequence for ECG device order resulted in the handheld device machine being connected first for 48 patients.
https://cardio.jmir.org/2021/2/e21186 JMIR Cardio 2021 | vol. 5 | iss. 2 | e21186 | p. 7
(page number not for citation purposes)
XSL• FO
RenderXJMIR CARDIO Wong et al
Figure 3. Patient randomization and number of electrocardiograms (ECG) acquired. ECG traces could not be acquired due to loss of connection for
(a) 1 patient, (b) 1 patient, (c) 3 patients, (d) 7 patients, (e) 2 patients, and (f) 3 patients.
Both devices had good reliability in producing heart rate, PR
Reliability interval, and QRS interval measurements while patients were
The within- and between-device mean differences for QT and supine and standing. The within- and between-device mean
QTc while patients were supine and standing were consistently differences in heart rate, PR interval, and QRS interval
within ±15ms for both handheld and conventional devices measurements were ±5 bpm, ±10 ms, and ±10 ms (Figure 5).
(Figure 4).
Figure 4. Within- and between-device variability of QT interval and corrected QT interval by patient posture. Mean differences and 95% confidence
intervals. Dashed red lines indicate predefined acceptable limits.
https://cardio.jmir.org/2021/2/e21186 JMIR Cardio 2021 | vol. 5 | iss. 2 | e21186 | p. 8
(page number not for citation purposes)
XSL• FO
RenderXJMIR CARDIO Wong et al
Figure 5. Within- and between-device variability of heart rate, PR interval, and QRS interval by patient posture. Mean differences and 95% confidence
intervals. Dashed red lines indicate predefined acceptable limits.
Variability of the differences in QTc measurements between standing (median 446.0 ms, IQR 50.0 ms) and supine QTc
the handheld device and conventional ECG machines were measurements (median 420.0 ms, IQR 48.0 ms) was significant
randomly distributed (Figure 6). For conventional ECG, the (PJMIR CARDIO Wong et al
Table 3. Within- and between-device agreement in diagnosing atrial fibrillation and prolonged QTc. Reliability of QTc measurements.
Variablea Within-device κ or ICCb,c (95% CI) Between-device κ or ICCc (95% CI)
Handheld device Conventional machine
Atrial fibrillation
Patients were supine 0.82 (0.58-1.00) 0.79 (0.50-1.00) 0.90 (0.71-1.00)
Patients were standing 0.68 (0.40-0.97) 0.84 (0.62-1.00) 0.93 (0.78-1.00)
Prolonged QTcd
Patients were supine 0.84 (0.68-0.99) 0.75 (0.55-0.94) 0.92 (0.81-1.00)
Patients were standing 0.71 (0.54-0.88) 0.77 (0.61-0.93) 0.69 (0.51-0.86)
QTc measurements
Patients were supine 0.96 (0.93-0.97)c 0.92 (0.88-0.95)c 0.92 (0.88-0.95)c
Patients were standing 0.90 (0.84-0.94)c 0.95 (0.92-0.97)c 0.94 (0.90-0.96)c
a
Clinicians mistakenly reprinted the second conventional ECG from the first ECG in the first 20 patients, and 8 other patients declined repeating ECG
acquisition after several attempts. These 28 patients were excluded from this analysis resulting in a sample size of 72.
b
ICC: intraclass correlation coefficient.
c
These values are ICCs.
d
QTc: corrected QT interval.
conventional ECG machine was 39.5 seconds (95% CI
Time Efficiency 27.0-51.0). These times excluded the time taken to prepare the
The mean time taken to place electrodes on patients (regardless patient (eg, the time taken by the patients to undress themselves
of ECG device) was 42.2 seconds. The mean times taken by for the procedures). The randomized sequence of applying the
clinicians to acquire the first ECG using the handheld device devices had insignificant effect on the difference in ECG
and conventional ECG machine were 144.6 seconds and 103.5 acquisition time (conventional: P=.51; handheld: P=.97). ECG
seconds, respectively. On average, the total times taken by acquisition times improved with the number of time clinicians
clinicians to acquire an ECG using the handheld device and used the devices (PJMIR CARDIO Wong et al
User Satisfaction and Acceptance satisfaction after are shown in Table 4 and Figure 8. Patients’
Clinicians’ expectations of the handheld device before and experiences are also shown in Table 4.
Table 4. Clinician and patient satisfaction.
Survey item Rating, mean (SD)
Clinicians’ expectations before using the handheld device and their satisfaction after using it
Accuracy
Beforea 5.0 (0.0)
Afterb 4.0 (0.8)
Quality of ECGc trace
Beforea 5.0 (0.0)
Afterb 4.0 (0.8)
Ease of use
Beforea 4.7 (0.5)
Afterb 3.9 (0.9)
Efficiency
Beforea 4.9 (0.3)
Afterb 3.5 (1.0)
Clinicians’ response after using the handheld device
Compared to conventional ECG, I found the handheld device easier to used 3.4 (0.8)
Using the handheld device in my job increased my productivityd 3.1 (0.5)
Assuming I had continual access to the handheld device, I intended to use itd 3.6 (0.5)
Patients’ experience with the handheld device compared to the conventional ECG machine
Patient felt comfortable while connected to the device and lying down
The handheld deviced 3.2 (0.6)
Conventional ECGd 3.0 (0.3)
Patient felt comfortable while connected to the device and standing up
The handheld deviced 3.2 (0.5)
Conventional ECGd 3.1 (0.4)
a
5-point Likert scale ranging from 1 (not important) to 5 (very important).
b
5-point Likert scale ranging from 1 (not satisfied) to 5 (very satisfied).
c
ECG: electrocardiography.
d
5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree).
https://cardio.jmir.org/2021/2/e21186 JMIR Cardio 2021 | vol. 5 | iss. 2 | e21186 | p. 11
(page number not for citation purposes)
XSL• FO
RenderXJMIR CARDIO Wong et al
Figure 8. Radar chart of clinician satisfaction after using the handheld device. ECG: electrocardiogram.
between while doing ECGs, especially for standing
Thematic Analysis of Interviews With Clinicians one. [Clinician 3, cardiology outpatient clinic where
Overview patients had ECG acquired before seeing cardiologist]
Most (10/11) clinicians attended a 1-to-1 interview. One It has less interference mainly while doing standing
clinician was on leave and could not attend the interview. When ECGs compared to conventional device. [Clinician
asked about their needs, satisfaction, and useful features of the 8, cardiology inpatient ward]
handheld device, clinicians frequently mentioned the following The aspects that I liked about this device is it’s clear,
features: quality of the ECG traces, ease of use, efficiency, less interference with cardiac monitors, small,
accuracy, small size, and portability. light-weight, easy to carry and chargeable. [Clinician
11, cardiology inpatient ward]
Main Themes
Clinicians specified portability as a desired feature of the
When asked about their needs regarding an ECG device,
handheld ECG device. The wireless transmission of ECG trace
clinicians expressed that they wanted a device that was easy to
from the device to a laptop computer was an added advantage;
use, efficient (took short time to acquire an ECG), accurate, and
however, the need to carry a laptop computer to connect with
produced good-quality ECG traces. Clinicians reported
the ECG device could be a drawback.
satisfaction with the accuracy and quality of the ECG traces
produced by the handheld device, however, were equivocal I do find it useful as far as portability wise it is good.
about the efficiency due to the extra time they took to acquire However, it takes more time compared to the old
an ECG. device. Few aspects of the device, I like are it's handy,
easy to carry, save lots of space and ECG are saved
My needs are to do ECG quick and efficient as
within the laptop so less chance of loss of ECG. Also,
possible, and better quality ECG. Quality-wise it is
no damage to papers and no extra cost for papers.
good, but I am slightly satisfied with the efficiency of
[Clinician 1]
the device. [Clinician 1]
I like the most about this device is it is portable, small
I am satisfied with the quality and accuracy of the
and quite fast picking up ECG sometimes. But, most
device. I also find it easy to use. But usually, it takes
of the times we need to wait a few more seconds to
more time than the old device. [Clinician 6]
get a satisfactory ECG. The thing I don't like about
My needs are to acquire accurate and better quality this device is it is attached to the laptop. The device
ECGs. I am satisfied with accuracy, efficiency and is smaller and easy to carry everywhere, but along
quality of the device. [Clinician 8] with this, we should also carry a laptop everywhere.
Feedback about interference (loss of connection between the I feel for our clinic (Rapid Access Cardiac Clinic)
device and computer) varied between clinicians in cardiology paper is more efficient. [Clinician 6]
inpatient wards and outpatient clinics. Clinicians in inpatient Most clinicians (9/11) agreed that the handheld device was
wards reported less interference in the handheld device than suitable for clinician-led mass screening; 1 clinician was unsure,
that in a conventional ECG machine, however, clinicians in and 1 clinician stated “yes” with suggested enhancements to
outpatient clinics found the opposite. the device. Suggestions for enhancements included improving
There is a lot of interference with the connection of efficiency in using the device to acquire ECG by increasing
the device. The connection gets lost multiple times in user training,
https://cardio.jmir.org/2021/2/e21186 JMIR Cardio 2021 | vol. 5 | iss. 2 | e21186 | p. 12
(page number not for citation purposes)
XSL• FO
RenderXJMIR CARDIO Wong et al
As it is small in size and not bulky and easy to carry patients in supine and standing positions and concluded that the
everywhere, it can be used for mass screening. But, ECG results in supine and standing were comparable. This
firstly, clinicians who will be using this device for comparability may allow ECG recording in busy clinical setting
mass screening purpose should be properly and to be more cost-effective—ECGs could be acquired while
adequately trained. [Clinician 3] patients are standing. Despite high reliability, we should note
and improving wireless transmission of ECG trace to a computer that QTc measurements (regardless of device) while patients
(increasing the range of wireless transmission to allow patient were standing compared to those when supine were longer,
isolation for infection control and remote assessment of which was consistent with the literature [13,14]. Compared with
patients). QTc measurement when supine, QTc measurement while
standing was more accurate in distinguishing patients with long
Long range would help for infection control use. ie QT syndrome from individuals without long QT syndrome [14].
receiver on patient and device outside the room for Lengthened QTc while standing could assist clinicians to
an isolated patient. [Clinician 10] diagnose long QT syndrome. Researchers should evaluate the
It is a small device so we can easily carry this device effect of change in body position when comparing device
everywhere, even in remote areas. So, I think it is reliability.
more convenient for mass screening in rural settings
Clinician perceptions of efficiency were affected by their
where it is difficult to carry the old device. [Clinician
familiarity with the device. It was expected that users would
2]
take a longer time using the newly introduced handheld device
Discussion to acquire an ECG than when using a familiar conventional
ECG machine because users lacked familiarity with the new
Principal Findings device. It is worthwhile to note that the time measured in this
study excluded the time to prepare the patients (eg, time for
The results suggested that the handheld device had high
patients to remove their top clothing and get ready for the
reliability in producing key ECG parameters and had good levels
procedure). The clinicians’ ECG acquisition times were less
of agreement with the conventional ECG machines in diagnosing
than the 10.6 minutes reported by Somerville and colleagues
prolonged QTc and atrial fibrillation using the device automatic
[31] (which included the time taken for preparing the patients)
algorithm. The clinicians’ efficiency in using the devices
using conventional 12-lead ECG in general practice. The
improved with usage, which was demonstrated by ECG
clinicians in our study were mainly nursing staff working in the
acquisition times. This user-centered approach helped us identify
cardiology clinic and ward, and they acquired ECGs daily or
remediable action to improve user efficiency with training.
weekly. The clinicians’ familiarity with acquiring ECG could
Highly desirable device features, such as portability (small size
differ in comparison to those in general practice setting, and
and lightweight) and wireless ECG transmission (enhancement
this factor should be taken into consideration when formulating
in the wireless range of ECG transmission from the device to
a training program.
computer) allow clinician-led mass screening and remote
assessment of patients to be feasible. However, the mass Clinician experience in using the device was contextual. In the
screening should be clinician-led because users require the skill inpatient cardiology ward, clinicians reported that there were
to apply electrodes correctly on the body. less artefacts on the ECGs because of less interference between
the handheld device and the other monitors to which patients
The mixed methods approach allowed us to explore diverse
were connected, but clinicians in the outpatient cardiology clinic
perspectives in the usability of the handheld device. Quantitative
reported that there were several incidences of lost Bluetooth
evaluation of clinicians’ ECG acquisition times provided an
connection between the handheld device and laptop computer
objective measurement of time efficiency and characterized the
resulting in multiple attempts to reacquire ECGs. Thus, the
trend of improvement in efficiency with the number of usage
experiences and resulting perceptions on ease of use of the
(Figure 7). In this study, we found that differences in ECG
device varied depending on the environment in which the device
acquisition times between the devices approached 0 after the
was used. Patient satisfaction was mainly focused on their
users used the devices 18 times. Quantitative measurements of
comfort when connected to the devices. Most of the patients
ECG acquisition times revealed the learning curves of different
felt comfortable during ECG acquisition with both handheld
users. With qualitative evaluation of user perceptions and
and conventional devices.
experiences in using the device, we identified their training
needs, desired device features, and suggestions to make the Strengths and Limitations
device suitable for clinician-led mass screening and remote
The strengths of this study included the application of
assessment of patients. This mixed methods approach addressed
quantitative and qualitative methods to evaluate devices using
gaps in the common approaches to medical device evaluation,
a usability evaluation framework that integrated the International
which lack evaluation of user perceptions, experiences, and
Organization for Standardization Guidelines on Usability [18]
efficiency [16].
and the Technology Acceptance Model [25,26], and the use of
The devices had high reliability in producing key ECG forest plots to examine within- and between-device variabilities
parameters while patients were supine and standing. This was to complement the use of other quantitative indices (ICC and
consistent with findings in previous research. Madias and κ). However, because of time constraints, we did not evaluate
colleagues [30] evaluated a standard 12-lead ECGs recorded in the variability of ECGs while patients were sitting, we did not
https://cardio.jmir.org/2021/2/e21186 JMIR Cardio 2021 | vol. 5 | iss. 2 | e21186 | p. 13
(page number not for citation purposes)
XSL• FO
RenderXJMIR CARDIO Wong et al
explore in-depth views of patients, and most ECGs were make the device a potential tool for clinician-led mass screening
acquired by 4 clinicians. and remote assessment of patients. Patient body position should
be included in the evaluation of device reliability because QTc
Conclusions lengthening secondary to standing offers diagnostic information.
The handheld 12-lead ECG device was comparable to routinely The user-centered evaluation framework utilized in this study
used conventional 12-lead ECG machine in its reliability and could be applied to evaluate and better understand the
usability. The device’s small size, light weight, and wireless acceptability and usability of new medical devices.
ECG transmission coupled with improved efficiency via training
Acknowledgments
We would like to acknowledge and thank the clinicians and participants of the study. We would also like to thank Mr. Lucas Bao
Thu Nguyen for his assistance in producing the forest plots. The study was funded by the University of Sydney Industry Engagement
Seed Fund, custo med GmbH, and Cardio-Jenic Pty Ltd. The funders were not involved in study design, analysis, and interpretation
of results.
Authors' Contributions
KCW, CKC, SK, and AT initiated conceptualization of the study. KCW and SM established the statistical analysis plan including
randomization sequence and sample size computation. KCW drafted the research protocol, which was reviewed and approved
by CKC, SK, AT, and SM. KCW and RK performed data collection and data management. RK assisted in transcribing interview
transcripts. KCW and JB performed thematic analysis on interviews. KCW performed statistical analysis, and SM reviewed the
analysis. KCW drafted the manuscript, which was reviewed, revised, and approved by all authors.
Conflicts of Interest
None declared.
References
1. Australia's leading causes of death, 2015. Australian Bureau of Statistics. URL: https://www.abs.gov.au/ausstats/abs@.nsf/
Lookup/by%20Subject/3303.0~2015~Main%20Features~Australia's%20leading%20causes%20of%20death,%202015~3
[accessed 2021-08-09]
2. Srinivasan N, Schilling RJ. Sudden cardiac death and arrhythmias. Arrhythm Electrophysiol Rev 2018 Jun;7(2):111-117
[FREE Full text] [doi: 10.15420/aer.2018:15:2] [Medline: 29967683]
3. Wong KC, Kok C, Marschner S, Usherwood T, Chow CK. Screening for atrial fibrillation and other arrhythmias in primary
care. BMC Fam Pract 2020 May 06;21(1):79 [FREE Full text] [doi: 10.1186/s12875-020-01151-8] [Medline: 32375662]
4. Roth GA, Mensah GA, Johnson CO, Addolorato G, Ammirati E, Baddour LM, GBD-NHLBI-JACC Global Burden of
Cardiovascular Diseases Writing Group. Global burden of cardiovascular diseases and risk factors, 1990-2019: update from
the GBD 2019 study. J Am Coll Cardiol 2020 Dec 22;76(25):2982-3021 [FREE Full text] [doi: 10.1016/j.jacc.2020.11.010]
[Medline: 33309175]
5. Zhang N, Gong M, Tse G, Zhang Z, Meng L, Yan BP, et al. Prolonged corrected QT interval in predicting atrial fibrillation:
a systematic review and meta-analysis. Pacing Clin Electrophysiol 2018 Mar 16;41(3):321-327. [doi: 10.1111/pace.13292]
[Medline: 29380395]
6. Kirchhof P. The future of atrial fibrillation management: integrated care and stratified therapy. Lancet 2017 Oct
21;390(10105):1873-1887. [doi: 10.1016/S0140-6736(17)31072-3] [Medline: 28460828]
7. Schwartz PJ, Ackerman MJ, Antzelevitch C, Bezzina CR, Borggrefe M, Cuneo BF, et al. Inherited cardiac arrhythmias.
Nat Rev Dis Primers 2020 Jul 16;6(1):58 [FREE Full text] [doi: 10.1038/s41572-020-0188-7] [Medline: 32678103]
8. Waddell-Smith KE, Skinner JR, members of the CSANZ Genetics Council Writing Group. Update on the diagnosis and
management of familial long QT syndrome. Heart Lung Circ 2016 Aug;25(8):769-776. [doi: 10.1016/j.hlc.2016.01.020]
[Medline: 27262388]
9. Walker A, Muhlestein J. Smartphone electrocardiogram monitoring: current perspectives. Adv Health Care Technol 2018
Jun;4:15-24. [doi: 10.2147/AHCT.S138445]
10. Prgomet M, Georgiou A, Westbrook JI. The impact of mobile handheld technology on hospital physicians' work practices
and patient care: a systematic review. J Am Med Inform Assoc 2009;16(6):792-801 [FREE Full text] [doi:
10.1197/jamia.M3215] [Medline: 19717793]
11. Nguyen HH, Silva JN. Use of smartphone technology in cardiology. Trends Cardiovasc Med 2016 May;26(4):376-386.
[doi: 10.1016/j.tcm.2015.11.002] [Medline: 26686266]
12. Strik M, Caillol T, Ramirez FD, Abu-Alrub S, Marchand H, Welte N, et al. Validating QT-interval measurement using the
Apple Watch ECG to enable remote monitoring during the COVID-19 pandemic. Circulation 2020 Jul 28;142(4):416-418
[FREE Full text] [doi: 10.1161/CIRCULATIONAHA.120.048253] [Medline: 32478565]
https://cardio.jmir.org/2021/2/e21186 JMIR Cardio 2021 | vol. 5 | iss. 2 | e21186 | p. 14
(page number not for citation purposes)
XSL• FO
RenderXJMIR CARDIO Wong et al
13. Adler A, van der Werf C, Postema PG, Rosso R, Bhuiyan ZA, Kalman JM, et al. The phenomenon of "QT stunning": the
abnormal QT prolongation provoked by standing persists even as the heart rate returns to normal in patients with long QT
syndrome. Heart Rhythm 2012 Jun;9(6):901-908 [FREE Full text] [doi: 10.1016/j.hrthm.2012.01.026] [Medline: 22300664]
14. Viskin S, Postema PG, Bhuiyan ZA, Rosso R, Kalman JM, Vohra JK, et al. The response of the QT interval to the brief
tachycardia provoked by standing: a bedside test for diagnosing long QT syndrome. J Am Coll Cardiol 2010 May
04;55(18):1955-1961 [FREE Full text] [doi: 10.1016/j.jacc.2009.12.015] [Medline: 20116193]
15. Nadal C, Sas C, Doherty G. Technology acceptance in mobile health: scoping review of definitions, models, and measurement.
J Med Internet Res 2020 Jul 06;22(7):e17256 [FREE Full text] [doi: 10.2196/17256] [Medline: 32628122]
16. Park J, Kim E, Shin K. Developing an evaluation framework for selecting optimal medical devices. J Open Innov Technol
Mark Complex 2019 Aug 28;5(3):64. [doi: 10.3390/joitmc5030064]
17. Shorten A, Smith J. Mixed methods research: expanding the evidence base. Evid Based Nurs 2017 Jul 14;20(3):74-75.
[doi: 10.1136/eb-2017-102699] [Medline: 28615184]
18. International Organization for Standardization. BS EN ISO 9241-11:2018 Ergonomics of human-system interaction.
Usability: Definitions and concepts. The British Standards Institution. 2018. URL: https://landingpage.bsigroup.com/
LandingPage/Standard?UPI=000000000030319071 [accessed 2021-08-13]
19. Bevan N, Carter J, Harker S. ISO 9241-11 revised: what have we learnt about usability since 1998? In: Kurosu M, editor.
Human-Computer Interaction: Design and Evaluation Lecture Notes in Computer Science, vol 916. Cham: Springer;
2015:143-151.
20. Hardware, description of device for custo cardio 300. Custo Med GmbH. 2018. URL: https://www.customed.de/
DownloadDoc?v=1.1.95 [accessed 2021-08-13]
21. Streiner DL, Norman GR, Cairney J. Reliability. In: Streiner DL, Norman GR, Cairney J, editors. Health Measurement
Scales: A Practical Guide to their Development and Use. Oxford: Oxford University Press; 2008.
22. Darpo B, Nebout T, Sager PT. Clinical evaluation of QT/QTc prolongation and proarrhythmic potential for nonantiarrhythmic
drugs: the International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for
Human Use E14 guideline. J Clin Pharmacol 2006 May;46(5):498-507. [doi: 10.1177/0091270006286436] [Medline:
16638733]
23. Mert KU, Turk UO, Gorenek B. A statistical perspective to the QTc interval in the treatment of COVID-19. J Electrocardiol
2021 May;66:48 [FREE Full text] [doi: 10.1016/j.jelectrocard.2021.03.002] [Medline: 33770647]
24. Dahlberg P, Diamant UB, Gilljam T, Rydberg A, Bergfeldt L. QT correction using Bazett's formula remains preferable in
long QT syndrome type 1 and 2. Ann Noninvasive Electrocardiol 2021 Jan;26(1):e12804 [FREE Full text] [doi:
10.1111/anec.12804] [Medline: 33070409]
25. Gagnon MP, Orruño E, Asua J, Abdeljelil AB, Emparanza J. Using a modified technology acceptance model to evaluate
healthcare professionals' adoption of a new telemonitoring system. Telemed J E Health 2012 Jan;18(1):54-59 [FREE Full
text] [doi: 10.1089/tmj.2011.0066] [Medline: 22082108]
26. Rahimi B, Nadri H, Lotfnezhad Afshar H, Timpka T. A systematic review of the technology acceptance model in health
informatics. Appl Clin Inform 2018 Jul 15;9(3):604-634 [FREE Full text] [doi: 10.1055/s-0038-1668091] [Medline:
30112741]
27. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a
metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed
Inform 2009 Apr;42(2):377-381 [FREE Full text] [doi: 10.1016/j.jbi.2008.08.010] [Medline: 18929686]
28. Bland J, Altman D. Comparing methods of measurement: why plotting difference against standard method is misleading.
The Lancet 1995 Oct;346(8982):1085-1087. [doi: 10.1016/s0140-6736(95)91748-9]
29. Yap BW, Sim CH. Comparisons of various types of normality tests. J Stat Comput Simul 2011 Dec;81(12):2141-2155.
[doi: 10.1080/00949655.2010.520163]
30. Madias JE. Comparability of the standing and supine standard electrocardiograms and standing sitting and supine stress
electrocardiograms. J Electrocardiol 2006 Apr;39(2):142-149. [doi: 10.1016/j.jelectrocard.2005.07.006] [Medline: 16580409]
31. Somerville S, Somerville J, Croft P, Lewis M. Atrial fibrillation: a comparison of methods to identify cases in general
practice. Br J Gen Pract 2000 Sep;50(458):727-729 [FREE Full text] [Medline: 11050790]
Abbreviations
ECG: electrocardiography
ICC: intraclass correlation coefficient
QTc: corrected QT interval
https://cardio.jmir.org/2021/2/e21186 JMIR Cardio 2021 | vol. 5 | iss. 2 | e21186 | p. 15
(page number not for citation purposes)
XSL• FO
RenderXJMIR CARDIO Wong et al
Edited by G Eysenbach; submitted 19.01.21; peer-reviewed by HC Lin, W Sun; comments to author 17.03.21; revised version received
23.03.21; accepted 27.07.21; published 26.08.21
Please cite as:
Wong KC, Thiagalingam A, Kumar S, Marschner S, Kunwar R, Bailey J, Kok C, Usherwood T, Chow CK
User Perceptions and Experiences of a Handheld 12-Lead Electrocardiographic Device in a Clinical Setting: Usability Evaluation
JMIR Cardio 2021;5(2):e21186
URL: https://cardio.jmir.org/2021/2/e21186
doi: 10.2196/21186
PMID:
©Kam Cheong Wong, Aravinda Thiagalingam, Saurabh Kumar, Simone Marschner, Ritu Kunwar, Jannine Bailey, Cindy Kok,
Tim Usherwood, Clara K Chow. Originally published in JMIR Cardio (https://cardio.jmir.org), 26.08.2021. This is an open-access
article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/),
which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR
Cardio, is properly cited. The complete bibliographic information, a link to the original publication on https://cardio.jmir.org, as
well as this copyright and license information must be included.
https://cardio.jmir.org/2021/2/e21186 JMIR Cardio 2021 | vol. 5 | iss. 2 | e21186 | p. 16
(page number not for citation purposes)
XSL• FO
RenderXYou can also read
