Simplicity of Use - The Sight OLO Hematology Analyzer
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Simplicity of Use
The Sight OLO® Hematology Analyzer
Findings from an observational study and usability questionnaire
At Sight, we keep things simple. From the AI-driven design of our blood test technology to the ease of operating our
machines and our quick turnover time, we streamline the typical manual blood staining workflow. With our self-contained
OLO hematology analyzer and its disposable cartridge, fast and accurate Complete Blood Count (CBC) results are
minutes away, resulting in rapid diagnosis leading to treatment of health concerns (Figure 1). But how easy is our
sample preparation process and technology to use? In February of 2019, we tested our training and the OLO system
(device and test kit) by obtaining feedback directly from new users. Through a multi-step observational study and
accompanying questionnaire, we set out to answer whether or not the OLO works according to our intended
design — simple to operate with no critical operation or manipulation needed from the user.
Figure 1. OLO Self-Contained Hematology Analyzer
The Observational Study
Our Goals
• To establish whether new OLO users understand how to use the device correctly after receiving the standard
training provided during regular commercial use.
• To assess whether OLO users find the OLO system simple to operate.
• To determine whether the OLO design is compatible with the skills of intended users and not vulnerable to the
introduction of risks arising from user error.
S.D. Sight Diagnostics LTD. 23 Menachem Begin Rd Tel Aviv 6618356 Israel | www.sightdx.com
CMR0230 rev 1.0 Intended for global audience outside of the USA | All rights reserved | 2021 Copyright Sight Diagnostics®Study Design*
Population Demographics
16 Participants
All participants
were between 18
and 32 years old.
Participants Job Titles
Paramedic Medic
Nurse Ward Medical Secretary
Lab Assistant Doctor
Doctor's Assistant Medical Student
Figure 2. Infographic of the study’s population demographics
Glossary
The user interface workflow in demo mode is identical
to the workflow in standard operating mode. However,
the device does not perform a scan of the sample but
Demo Mode
instead simulates a short scan (less than a minute) and
provides the user with quick, preset results shortly after
loading the cartridge into the device.
The workflow and user experience are identical in QC
and standard operating mode. However, in QC mode, the
user doesn’t need to specify demographic and age group
and doesn’t need to work with a whole blood sample
QC (Quality Control) Scanning Mode but rather with a QC blood sample. QC mode must be
selected from the QC menu when scanning a QC blood
sample (provided in a blood tube) to avoid rejection
by the system (which recognizes when a QC sample is
analyzed in the standard operating mode).
S.D. Sight Diagnostics LTD. 23 Menachem Begin Rd Tel Aviv 6618356 Israel | www.sightdx.com
CMR0230 rev 1.0 Intended for global audience outside of the USA | All rights reserved | 2021 Copyright Sight Diagnostics®An operational difficulty case occurs when a user
appears to struggle to perform a task. Indications of
such a struggle might be: Multiple attempts to perform
the task; anecdotal comments about the task difficulty;
Operational Difficulty (OD)
requests for assistance with the task; facial expressions
and vocalizations suggesting frustration or confusion
(subject to confirmation); and higher than usual task
performance times.
A close call is a case in which a user almost commits an
error but self-corrects in time to avoid making the error.
The term also describes a case in which a user commits
an error but detects it shortly after and recovers before
Close Call (CC)
the error becomes consequential. For example, a user
might forget to dispose of the used cartridge after the
scan but then realizes the mistake and performs the
step correctly.
A use error is a case in which a user performs a task in
an incorrect manner that will not lead to the intended
outcome. Use errors are not necessarily the user’s fault;
they can also be shortcomings in the device’s user interface
design. Depending on the particular use error’s detectability
Use Error (UE)
and where users direct their attention when the use error
occurs, users may or may not be aware that a user error
occurred. Examples of use errors include mistakes in
sample preparation or entering the wrong number into a
data entry field.
Three Phases
The study’s three phases occurred from 2019-01-30 until 2019-02-06:
1. Participant enrollment and training:
One day before the evaluation, the 16 participants — representing a range of potential OLO users — were trained
in smaller groups on the expected tasks of the study. During these training sessions, the participants received the
same level of training and documentation as actual OLO users would during commercial use. All sessions ended
with each participant performing a blood tube sample preparation on the OLO device to assess the effectiveness
of the training for that user.
2. Assessment of study tasks:
On the day of evaluation, the study facilitators observed as the participants performed the study tasks, including:
• Blood tube handling
• Cartridge preparation
• Scanning and result interpretation
Users performed each task twice. The first round was on a device configured in demo mode using a quality control
sample (i.e., preserved blood) placed in a K2EDTA 3CC tube (to simulate a standard whole blood sample). The second
round was performed in QC scanning mode, using a standard QC sample. This round evaluated the participant’s
ability to complete a successful scan without rejection by OLO's failsafe system, which arises from user error.
S.D. Sight Diagnostics LTD. 23 Menachem Begin Rd Tel Aviv 6618356 Israel | www.sightdx.com
CMR0230 rev 1.0 Intended for global audience outside of the USA | All rights reserved | 2021 Copyright Sight Diagnostics®The study facilitators observed all the task steps and recorded the outcomes. The study ended with the 16 participants
completing debrief interviews and providing subjective feedback via a questionnaire* regarding any usability issues
encountered and their overall experience with the device.
*This was part of a more extensive questionnaire-based study (conducted with 27 participants) to assess the OLO’s simplicity of use;
see more below.
3. Results and data analysis:
Post task completion, the data were analyzed accordingly:
• The numbers of each type of observed OD, CC, and UE were tallied.
• The rate of task completion was calculated.
• Based on each participant’s input and observations made by the study team, the potential root causes of all
ODs, CCs, and UEs were identified.
• All observed ODs, CCs, and UEs were reviewed and evaluated by our cross-functional team for their potential to
cause erroneous results.
• The current mitigations to the hazards from the OLO risk analysis that can cause risk to the patient, environment,
or operator were evaluated. The acceptability of each risk was determined through the risk analysis procedure
(evaluating potential harm, probability of occurrence, and detectability).
• Any patterns and frequencies in the ODs, CCs, and UEs that suggest user interface.
*The validation test methodology was consistent with the guidance provided in: “Guidance for Industry and FDA Staff: Applying Human
Factors and Usability Engineering to Medical Devices, February 2016”.
Acceptance Criteria
Our assessment criteria to determine whether or not the OLO is safe and easy to use:
• The successful execution of each task by at least 15 of the 16 participants who attempted that task.
• A determination that no use errors, task failures, or user interface shortcomings encountered will lead to an
unacceptable level of harm to the patient, environment, or operator.
• An overall 90% rating of 4 or more to all 1 - 5 scale questions and an average rating of 4 or more per question in
each section of the accompanying questionnaire.
• A positive subjective reply from the users in response to whether the device is simple to use.
*The validation test methodology was consistent with the guidance provided in: “Guidance for Industry and FDA
Staff: Applying Human Factors and Usability Engineering to Medical Devices, February 2016”.
S.D. Sight Diagnostics LTD. 23 Menachem Begin Rd Tel Aviv 6618356 Israel | www.sightdx.com
CMR0230 rev 1.0 Intended for global audience outside of the USA | All rights reserved | 2021 Copyright Sight Diagnostics®The Results
Identified Use Cases
A total of six ODs, one CC, and two UEs were identified:
UE #1: Wrong patient gender OD #1: Had difficulty scanning the QR
selected. code, but repeated once and succeeded.
UE #2: Did not notice that OD #2: Two users had difficulty properly
there was an Interpretive opening the test kit package; while
Program message indicating opening the test kit package using
some results were flagged. scissors, accidentally cut the edge of
OD
the Microsafe microcapillary. One user
CC completed the task successfully and the
other user threw the test kit away and
UE
used a new one.
CC#1: Did not notice an
OD #3: Had difficulty filing the Microsafe
Interpretive Program
microcapillary with enough blood because
message indicating some
too little blood was dispensed from the
results were flagged and
blood tube onto the parafilm.
immediately self-corrected.
OD #4: Two users had difficulty dispensing
enough blood from the blood tube
dispenser.
OD #5: Had difficulty closing the mixing
bottle with the dropper cap. Threw it
away and took a new test kit to start over.
OD #6: Had difficulty understanding the
Interpretive Program message indicating
some results were flagged.
Figure 3. Infographic showing each type of observed OD, CC, and UE
Finding Highlights
• In this observational study, 16 participants, representing a wide variety of potential operators, easily, safely, and
effectively performed all critical tasks for the intended use of the OLO device when provided with the level of
training and documentation provided in standard commercial use.
• Only one uncompleted task was related to the user. Thus the “at least 15 participants should successfully complete
each task” assessment criteria was met.
• The OLO is compatible with the users’ skills. Evaluation of the observed ODs, CCs, and UEs that occurred during
the study revealed that none posed more than minimal risk to a patient. And mitigations preventing significant
risk from occurring were all found adequate and effective.
S.D. Sight Diagnostics LTD. 23 Menachem Begin Rd Tel Aviv 6618356 Israel | www.sightdx.com
CMR0230 rev 1.0 Intended for global audience outside of the USA | All rights reserved | 2021 Copyright Sight Diagnostics®Post Study Questionnaire
Upon completing the observational study, the participants filled out a questionnaire to assess the OLO’s ease of use.
All 16 participants expressed a favorable opinion when asked whether the device is simple to use. This survey was
part of a more extensive questionnaire-based study conducted with 27 participants with various levels of experience
using the OLO (from new users to users working with the device for eight months). A brief overview of the questionnaire
design and results:
• The quantitative and qualitative survey questions addressed different aspects and potential use errors when
working with the OLO device and test kit, including the sample preparation (including the finger prick and blood
tube usage), scan, results, rejects, QC, user interface, and instructions for use.
• Two descriptive questions regarding user interface and experience created space for open-ended and unaided
feedback directly from the users.
• Overall all tasks were rated 4.6/5 or higher on average and ≥93% of respondents rated each task as “4” or higher.
This usability questionnaire study concluded that the OLO system is simple to use across different levels of operators
and that the on-screen instructions provided to perform the sample preparation and scan are easy to follow and clear.
Thus, we detected no patterns that would suggest any user interface, instructions for use, or training shortcomings.
Final Thoughts
We put the design and simplicity of our OLO hematology analyzer to the test, and the usability study confirms that new
(and more experienced) users find the OLO and training as we designed it to be —simple and easy to use.
OLO is CE marked and indicated for use in the EU and other territories for point of care settings.
The device is also FDA 510(k) cleared for use in moderately complex settings in the United States.
For full indications for use and safety information, please refer to the Quality and Compliance page at www.sightdx.com.
For more information,
contact your local sales representative at:
United Kingdom: sales-uk@sightdx.com
International: sales-ww@sightdx.com
S.D. Sight Diagnostics LTD. 23 Menachem Begin Rd Tel Aviv 6618356 Israel | www.sightdx.com
CMR0230 rev 1.0 Intended for global audience outside of the USA | All rights reserved | 2021 Copyright Sight Diagnostics®You can also read
