Successful Human Factors Programs: Key Considerations from a Regulator’s Perspective
This article provides two regulatory leaders insight on human factors engineering considerations for helping manufacturers optimize user interfaces and successfully meet regulatory expectations to bring safe, effective and usable medical devices to market. It was developed from an interview with Shannon Hoste, team lead of the Center for Devices and Radiological Health’s (CDRH’s) human factors premarket evaluation team and Michael Ryan, acting deputy director in the division of anesthesiology, general hospital, respiratory, infection control, and dental devices.
Human Factors Engineering (HFE) is a proven science within the US Food and Drug Administration (FDA) and other international regulatory bodies, especially in the premarket submission process. Accordingly, manufacturers are implementing better HFE practices within the overall design process to produce safer, more effective and usable medical devices and combination products for end users. The final HFE guidance was published in February 2016 five years after the draft guidance was issued by the Center for Devices and Radiological Health (CDRH).1 Concurrently, CDRH published a draft guidance that lists 16 device types that FDA expects to see human factors data for in the premarket submission.2 This guidance also provides considerations for determining other unlisted device types for which FDA requests human factors premarket submission data. US and international HFE standards also are more harmonized.
“We don’t view human factors as just another checkbox,” said Michael Ryan, acting deputy director in the Division of Anesthesiology, General Hospital, Respiratory, Infection Control,
and Dental Devices. “It is an important consideration in device development.”
Consistent with Ryan’s point, Shannon Hoste, cognitive systems engineer and team lead for Human Factors at CDRH, said that the international standard AAMI/ANSI/IEC 62366-1:2015 “further aligned with our perspective in that it is focused on safety, in hazard-related use scenarios.”3
According to Hoste, members of the CDRH premarket review team have settled into their respective roles, such as disseminating CDRH’s HFE expectations, providing reviews of validation protocols and reviewing final Human Factors Engineering/Usability Engineering (HFE/UE) reports. The new team, which recently expanded from three to five reviewers, brings a wealth of practical experience and expertise to FDA. They continuously participate in various public forums to disseminate and contribute information related to the final guidance and CDRH human factors expectations.
However, even with FDA HFE guidance documents, manufacturers may still have questions about best practices and what FDA and other notified bodies will accept as part of their submission.
The Evolving Regulatory HFE Perspective
The final guidance document, “Applying Human Factors and Usability Engineering to Medical Devices – Guidance for Industry and Food and Drug Administration Staff,” was developed by CDRH to support manufacturers in improving the design of devices, reducing risk and minimizing potential use errors and resulting harm.4 By following the CDRH human factors guidance, it is more likely that new medical devices will be safe and effective for the intended users, uses, and use environments.
For more than six years, FDA has expected manufacturers to apply good HFE to medical device design and to provide appropriate human factors evidence as part of the premarket submission when use- related risks exist.
“With the final guidance, we made some updates for clarity’s sake, based on feedback, but the overall process that was identified in there didn’t change overly dramatically,” said Hoste.
The Quality System Regulation requires that manufacturers establish and maintain design controls for medical devices (21 CFR 820.30 Design Controls).5-6 CDRH’s final guidance recommends that manufacturers consider human factors testing for medical devices as a part of a robust design control subsystem, and provides recommended HFE processes to help manufacturers optimize the design of user interfaces and produce the evidence necessary for premarket submissions. FDA’s recommendations focus on products where an analysis of risk indicates users risk harm when they are failing to perform tasks, or performing them incorrectly.
While extremely useful for introducing potential methods and tools, it is important to
recognize that the guidance does not provide a “recipe” type approach to a successful human factors submission.
“Both the guidance and 62366-1 are process documents, which outline an overall [process for] how you should consider use-related risks and how you can evaluate them and provide data to show if they’ve been eliminated or how they feed into the overall risk benefit decision of the product itself,” said Hoste. “Therefore, for any given development effort, application of sound HFE considerations, and the science behind this field, are still required.”
Key Considerations for Meeting CDRH Expectations
Submit Validation Protocols Describing Both Scope and Methods for FDA Review
FDA encourages manufacturers to submit a draft of the HFE validation testing protocol prior to conducting the study to ensure that the planned methods are acceptable.
“We always say that it’s worth investing that time and that energy upfront as opposed to getting into a situation where we’re late in the review clock, late in the product
development cycle, and there are open safety questions that necessitate additional testing,” said Ryan. “The premarket mechanism is the Q-sub process described in FDA guidance document “Requests for Feedback on Medical Device Submissions: the Pre-Submission Program and Meetings with FDA Staff.”7
To provide the most useful feedback for a HFE validation protocol, FDA requests information defining the scope of the study as well as the specific methods being used to gather the body of evidence to support the human factors submission. It is crucial to include background information as well as the specifics of the protocol and proposed data analysis (See Appendix A).8
“Seeing that preliminary analysis and contextual information helps us to identify that the use-related risk assessment is complete. Reviewing the use-related risk assessment helps us to determine that validation is complete,” explained Hoste.
For Hoste, content related to the validation study scope includes information on intended users, uses, use environments and training methods. It also consists of a description of the user interface, known use problems and preliminary analyses and evaluations, including the use-related risk analysis. All of this information feeds into identifying the critical tasks or the use-related tasks that may lead to serious harm. “The next step is to design sound data collection methods to collect evidence to show that the device can be used by the intended users without serious use errors or problems, for the intended uses and under the expected use conditions,” she said. “This is the overall focus of the HFE validation study.”9
When we’re interacting with sponsors, whether it’s reviewing a protocol or… a human
factor report, we are looking for how that evidence was built and that it is complete,” explained Hoste. “For a pre-submission protocol review, it is good to have all of that information as well as the protocol so that we can provide feedback on scope and method.”
The review ultimately helps manufacturers know if their protocol is sufficient for gathering acceptable human factors evidence in support of the final submission. According to Hoste, some questions that may be unaddressed if manufacturers do not seek an FDA protocol review include:
• critical tasks that are not being considered, but should be considered
• potential use scenarios of concern that are not being considered
• if the study is structured to provide representative use data that is generalizable to
Recognize the Intent of CDRH HFE Guidance
Manufacturers may ask why a pre-submission protocol review is necessary when the
final Human Factors Guidance offers important considerations and sets out steps for developing the final report. According to Hoste, it is important to recognize the guidance is a process document that identifies how to consider use-related risks, but it does not outline
what risks exist with a device or how to evaluate them, design the protocol, collect the data, or appropriately analyze the data. Essentially, the guidance builds on the scientific knowledge base of HFE.
The international standard IEC62366-1 is a similar process document.10 While these process documents spell out what is required from a conceptual basis, they do not explain how to derive the quality or completeness of HFE data. A pre-submission protocol review can assist manufacturers in determining whether appropriate user groups are included, adequate characteristics of the use environment are simulated, and representative training and periods of learning decay are included. Also, scenarios and knowledge task comprehension questions are appropriate, and critical tasks must be adequately categorized in a protocol that will gather sufficient human factors evidence for review by FDA.
Minimize Business Related Risks
After the validation study report is completed, it is often impossible to address FDA’s protocol concerns without conducting a supplemental study or redoing the validation study altogether. For example, FDA may have concerns with the study methodology if it is not representative of actual use, such as providing too much information in the moderator script, asking participants questions at a time that would bias subsequent data or leaving out the collection of subjective data. By waiting until the final submission to present the scope and methods along with the human factors validation data, manufacturers risk delaying product launch if FDA has protocol questions or concerns. Instead, manufacturers
can have questions answered during pre-submission reviews when the protocol can be adjusted proactively to answer FDA’s questions, which ultimately saves manufacturers both time and money. An analogy is to decide whether the bandage should be “ripped off” now or later. It is likely that the level of pain will be the same at both points in time.
But, in this context, the earlier manufacturers can determine and address any FDA feedback, making it more likely project budgets and timelines will be met. And, for some companies, meeting these requirements could be deciding factors in bringing a good product to market.
“From a business risk perspective, making sure that information is gathered and presented in a way to support the submission, [if done during the] pre-submission process, would be all the better to eliminate potential questions or reduce potential questions in that final submission,” explained Hoste.
Suggestions for HFE Submissions
It is important for medical device manufacturers to incorporate HFE throughout the device development process. When HFE is integrated as part of design controls, manufacturers
more efficiently develop devices that are safe and effective from a use perspective.11
“When this is done as part of the product development process, it creates a fairly clear body of evidence to support a submission,” said Hoste.
This applies to change efforts as well, for example, if there is a change in technology, ascertain whether the user interface change is safe and effective by following the human factors process to collect data to validate safe and effective use related to the change. Table A lists several suggestions from CDRH for HFE submissions.
HFE/UE reports are one part of the overall evidence reviewed in FDA submissions to determine the safety and effectiveness of devices. CDRH human factors premarket review team will identify any concerns, such as missing critical tasks or use scenarios as well as the quality of data analysis, in order to weigh the human factors risks before making a decision to put a device into the market. CDRH’s review of human factors scope, methods and data occurs during the final submission process.
“We’re asking for that information (HFE report) so that we can have sufficient data to evaluate and support the regulatory decision. If there are questions around use-related safety and effectiveness and the human factors analysis does not provide sufficient support for those items, then that can play into the decision,” said Hoste.
If the submission is missing use-related risk analysis or if the team is unsure if the scope of the validation was complete, and this is needed as part of a risk/benefit discussion, the team can ask the manufacturer for those data. If a study is not planned or conducted in an effective manner, the team will work with the manufacturer to better collect and evaluate
the data in a manner that supports the submission.
“In some cases, FDA’s final decision can come down to human factors being a sticking point, but we’re usually able to find a way to work with sponsors,” added Ryan.
Successful HFE Submissions
If HFE is implemented throughout the development of medical devices, manufacturers can optimize their devices by focusing on the end users, use environments, and user tasks. Doing so also helps manufacturers address use-related safety and effectiveness early in the development cycle, which means more cost effective HFE validation testing, more efficient timelines and lower overall budgets related to HFE. While there may not necessarily be one solution for mitigating use-related risks, there are specific HFE processes and tools that can be applied to most effectively optimize user interface designs. Lack of sufficient HFE evidence for safe and effective use will ultimately trigger FDA to request further information. This causes delays in the overall device submission, and in some instances, questions from FDA may require manufacturers to redesign their devices or studies to collect the necessary data to successfully answer FDA’s questions. Any additional and unplanned effort at the end of the submission process could result in significant delays in getting the device to market, which affects manufacturers and end users. A sound HFE
process can minimize these potential risks.
One solution is working with trained human factors professionals who understand both the science of HFE as well as the human factors regulatory submission process. Working with these professionals at the onset of a project to determine a strategy, along with implementing specific methods instead of trying to address inadequacies after a submission is questioned, yields the most efficient and cost-effective results. Talking with FDA to understand and address their concerns, as well as implementing CDRH’s suggestions described in this article, is a helpful first step.
• preliminary analyses
• formative evaluations
• clear identification of device users and assumptions
• adequate understanding of use scenarios
• integration of use-related risk analysis and risk management
• conducting HFE validation as part of design validation (if indicated)
Table A. Suggestions for HFE Submissions from CDRH.12-14
Manufacturers seeking to design better products and streamline the human factors part of the regulatory process can partner with an HFE organization or develop resources internally to leverage professionals that have mastered the science of human factors. An added benefit is these organizations and skilled professionals know how to “tweak” processes based on the current FDA thinking to implement sound methods. By incorporating good HFE into the design and development process, including implementing FDA HFE guidance related to development of safe and effective medical devices, quality medical devices can be manufactured more quickly. The devices also will reach and benefit end users sooner while minimizing post-market use-related issues.
Applying Human Factors and Usability Engineering to Medical Devices. Guidance for Industry and Food and Drug Administration Staff. US Department of Health and Human Services Food and Drug Administration
website. https://www.fda.gov/downloads/MedicalDevices/…/UCM259760.pdf . Accessed 7 December 2017.
List of Highest Priority Devices for Human Factors Review. Draft Guidance for Industry and Food and Drug Administration Staff. US Department of Health and Human Services Food and Drug Administration
gen/documents/document/ucm484097.pdf . Accessed 7 December 2017.
ANSI/AAMI/IEC 62366-1:2015. Medical Devices Part 1: Application of Usability
Engineering to Medical Devices. Association for the Advancement of Medical Instrumentation website. http://my.aami.org/store/detail.aspx?id=6236601-PDF . Accessed 7 December 2017.
Op cit. 1.
Code of Federal Regulations. Title 21 – Food and Drugs. US Government Publishing
Office website. https://www.gpo.gov/fdsys/pkg/CFR-2012-title21-vol8/xml/CFR-
2012-title21-vol8-sec820-30.xml . Accessed 7 December 2017.
Quality System (QS) Regulation/Medical Device Good Manufacturing Practices. US
Food and Drug Administration
website. https://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/Post marketRequirements/QualitySystemsRegulations/ucm2005736.htm . Accessed 7 December 2017.
Requests for Feedback on Medical Device Submissions: the Pre-Submission Program and Meetings with Food and Drug Administration Staff. Guidance for Industry and Food and Drug Administration Staff. US Department of Health and Human Services Food and Drug Administration
dance/guidancedocuments/ucm311176.pdf . Accessed 7 December 2017.
Op cit. 1.
Op cit. 3.
Hayhurst, C. “Better Together: the Vital Role of Patients and end Users in Developing
Safe and Effective Medical Devices.” Biomed Instrum Technol. 2017;51(4), 290-299.
Op cit. 3.
Mobile Medical Applications. US Food and Drug Administration
website. https://www.fda.gov/MedicalDevices/DigitalHealth/MobileMedicalApplic ations/ucm255978.htm . Accessed 7 December 2017.
Digital Health. US Food and Drug Administration
website. https://www.fda.gov/MedicalDevices/DigitalHealth/MobileMedicalApplic ations/ucm255978.htm. Accessed 7 December 2017 .
About the Author
Melissa R. Lemke holds a master of science in biomedical engineering and is the Director of Human Factors Engineering with Agilis Consulting Group, a full-service human factors firm focused exclusively on medical device human factors engineering, instructional labeling and training, human factors regulatory guidance, and post-market surveillance. She has more than 14 years of experience designing and evaluating medical devices, including for lay users, healthcare professionals, and individuals with functional limitations. Lemke is a member of the Human Factors and Ergonomics Society (HFES) as well as the Association for the Advancement of Medical Instrumentation (AAMI), AAMI
Human Factors Engineering Committee, AAMI Home Care and EMS Environments Committee, contributing author to ISO/AAMI HE-75 (2009), and AAMI faculty for the course Applying Human Factors to Improve Instructional Materials as Part of the User Interface. She may be contacted at firstname.lastname@example.org.
Cite as: Lemke, M.R. “Successful Human Factors Programs. Key Considerations from a Regulator’s Perspective.” Regulatory Focus. December. Regulatory Affairs Professionals Society.