Enhancing the fidelity of simulated-use environments in human factors studies during a pandemic
Agilis resumed in-person human factors (HF) studies in June 2020. While the world’s understanding of the SARS-CoV-2 virus has evolved over the past two years, so has the approach to conducting in-person HF studies in the medical device and pharmaceutical industries. Facemasks, physical distancing, plexiglass dividers, sterilization protocols, COVID-19 testing, and other preventative measures have now become “normal” when executing in-person HF studies. While certain points of view may argue that these protective measures have created a larger gap between simulated-use and actual use environments than existed pre-pandemic, perhaps there are ways to spin these protective measures to enhance fidelity of current simulated-use environments. In this newsletter, we provide an Agilis’ guide to conducting in-person HF studies during the ongoing COVID-19 pandemic.
Purpose of Human Factors Studies:
Due to evolving regulations by the U.S. Food and Drug Administration (FDA) and European Medicines Agency (EMA), human factors (HF) studies are becoming more common during development of medical devices and pharmaceutical products. Appropriate HF processes, of which HF studies are a component, can maximize the likelihood that new medical devices will be safe and effective for the intended users, uses, and use environments [1]. In late stages of product development, HF validation/summative studies can provide evidence that implemented use-related risk control measures are mitigating use-related risk as low as possible.
What is Necessary for a Human Factors Study to Fulfill its Purpose?
To answer this question, close examination of the following statement is advised, “Appropriate HF processes can maximize the likelihood that new medical devices will be safe and effective for the intended users, uses, and use environments.” Intended users, uses, and use environments has been underlined because it is critical to simulate these aspects of a medical product when conducting HF studies. Otherwise, data generated from an HF study is not fulfilling its purpose described above. Intended use is defined by the manufacturer based on the purpose of the medical product. The intended use of a product cannot be thoroughly evaluated in an HF study if intended users and use environments are not accurately represented. Intended users and use environments can be identified through multiple avenues such as research on predicate products, ethnographic research, task analysis, and others.
What is a Simulated-Use Environment?
The FDA and EMA understand that most HF validation testing will be conducted under conditions of simulated use [1, 2]. The expectation of a simulated-use HF studies is that test conditions should be sufficiently realistic so that the results of the testing are generalizable to actual use [1]. During the COVID-19 pandemic, many manufacturers and HF service providers have conducted remote HF testing via video conference. During remote studies, conducted by Agilis, troubleshooting was required of software and hardware by a study participant and/or study execution team taking up to 20 minutes at times impacting study schedule. Additionally, Agilis’ experience in these studies found that participant interactions with the product under evaluation may have been impacted by the participant feeling they have to stay in the video frame of the videoconference. Overall, Agilis recommends that in-person HF studies be conducted whenever possible because it allows for more natural use of the product being evaluated and more efficient data collection. So, how do we prevent safety measures such as facemasks, physical distancing, and plexiglass dividers from confounding our ability to generalize simulated-use HF study results to actual use? Agilis’ main advice is to address the safety measures head-on with study participants from recruitment through study execution.
Agilis’ Guide to In-Person HF Studies during an Ongoing COVID-19 Pandemic:
During Recruitment of Study Participants
Whether it is through the manufacturer, a recruiting agency, or a HF service provider, study participants are typically recruited to participate in HF studies using a participant screener. Within the participant screener, questions will be asked to determine if a person qualifies to participate in the HF study as a representative user, and additionally, it is when the participant is asked if they want to participate. This is an excellent time to inform participants of the COVID-19 precautions that will be taken during the HF study. This can help confirm if a participant will comply with and be understanding of the COVID-19 precautions, ensuring the study will be safe for the study participant and study execution team. Additionally, providing this information during recruitment will help set expectations for the participant, thus helping them be more focused on evaluating the product being tested in the HF study when they arrive for their study session rather than being distracted by COVID-19 precautions being taken.
During HF Study Conduct
When a participant arrives for their session as part of an HF study, it is important the study participant feels comfortable in the simulated-use environment as this will promote natural use of the product being evaluated. With the addition of COVID-19 safety precautions, Agilis has found that addressing why these safety precautions are being employed helps participants accept the precautions instead of being distracted by them. For example, during an introduction to a study session, instead of a study moderator saying, “We have a plexiglass shield between us and we are practicing social distancing to keep us safe from COVID-19,” we recommend saying, “We have a plexiglass shield between us and will be practicing social distancing to both keep us safe and give you space to use the product just as you would at home (or work).” Addressing the COVID-19 safety precautions in this manner ensures participants that safety is a priority, but also frames the COVID-19 safety precautions to enhance fidelity of the simulated-use environment.
Take Home Message
As the COVID-19 pandemic continues to impact our daily lives, it is still the expectation from U.S. and global regulators that medical devices and pharmaceutical products undergo robust HF processes during product development, including evaluation via HF studies. To continue to have HF study results obtained in simulated-use conditions with COVID-19 safety precautions be generalizable to actual use conditions, Agilis recommends that study participants be made aware of COVID-19 safety precautions prior to and during a HF study. This will maximize the likelihood of participants not being distracted by COVID-19 safety precautions and focusing on evaluating the product in the HF study.
References
[1] FDA guidance entitled, Applying Human Factors and Usability Engineering to Medical Devices that was published by CDRH, FDA on February 3, 2016.
[2] AAMI/IEC TIR62366-2:2016, Medical devices – Part 2: Guidance on the application of usability engineering to medical devices.
About the Author:
Jamie Jackson, MS
James (Jamie) Jackson is a Senior Human Factors Engineer Consultant with Agilis Consulting Group, LLC. Jamie's background is in human factors for combination products, specializing in use-related risk assessments, risk management strategies, integration of human factors data with design controls, and combination product design & usability testing for clinical and commercial use. Jamie's undergraduate is in biomedical engineering from Rensselaer Polytechnic Institute (summa cum laude) and his master’s is in human factors from Embry Riddle.
AAMI Book
Applied Human Factors in Medical Device Design