Does your human factors study need to be submitted to an Institutional Review Board?
Does your human factors study need to be submitted to an Institutional Review Board (IRB)?
Navigating requirements for usability studies
It is commonly debated whether usability studies are required to go through Institutional Review Board (IRB) review. Despite the general knowledge of the purpose of IRBs, inconsistent practice exists across institutions conducting human factors research. Human factors, specifically, has long existed in a grey area when it comes to IRB requirements and oversight, given the perceived lack of clarity in federal regulations governing research. As usability studies are often performed by private businesses that do not produce research for publication, many practitioners assume their research is exempt from IRB oversight.
In many cases, submitting human factors research to an IRB can be seen as an obstacle; an extraneous effort that requires both time and money. In other cases, confusion or perceived lack of clarity results in questions and assumptions that lead human factors practitioners down a path of opting out of submitting their research for IRB review. Many human factors practitioners are faced with common questions, such as:
Is IRB review needed for usability studies?
I believe my study is ‘low risk’ to study participants, does that mean I can forego IRB review?
Deciding if usability studies need oversight by an IRB can be a source of confusion or delay, but this does not necessarily need to be the case.
What is an IRB?
IRBs are independent committees that have been established as a result of federal regulations to protect the rights and welfare of human subjects participating in research activities. IRBs serve an important role in the protection of the rights and welfare of human research subjects.
IRBs are composed of at least five members of varying backgrounds, at least one member who is not affiliated with the institution, and one member who is not a scientist. All of these members convene to evaluate research plans and protocols to ensure study teams have considered all risks and protections for their study participants.
IRBs ensure that physical, psychological, and social risks to research subjects are minimized as far as possible. IRBs also evaluate the risks associated with the research to ensure they are commensurate with the importance of the research and the knowledge to be gained. During review, the committee also evaluates whether the study protocol takes measures to ensure research subjects receive accurate and complete information, at a level they can understand, about the nature of the research, how their personal information will be handled, any associated risks, and their rights as research subjects. Finally, study plans are reviewed to assess compliance with the standards set forth in the Code of Federal Regulations (CFR) by the U.S. Department of Health and Human Services (HHS).
When do you have to submit your human factors research to an IRB?
All research involving subjects in the United States is required to go through mandatory IRB review. The U.S. HHS states that research involving human subjects must receive IRB approval in accordance with CFR policy: Title 45 (Public Welfare), Part 46 (Protection of Human Subjects), often referred to by its shortened title: 45 CFR 46. The HHS states that an investigation is subject to the regulation if it meets the definition of Human Subjects Research. The simplify these terms, HHS has specifically defined what qualifies as research as well as what qualifies as a human subject. 45 CFR 46 specifically defines research and human subjects as the following:
Research is a systematic investigation, including research development, testing, and evaluation, designed to develop or contribute to generalizable knowledge.
Human Subjects are living individuals about whom an investigator conducting research:
(1) obtains information through intervention or interaction with the individual, and uses, studies, or analyzes the information, or
(2) obtains, uses, studies, analyzes, or generates identifiable private information.
Since usability tests by definition are both research and include human subjects, they are subject to the 45 CFR 46 regulation, and should be reviewed by an IRB prior to beginning research activities. This includes all research studies conducted at any point in a product lifecycle, not just the usability testing protocols that are submitted to an agency for review.
Remember, the purpose of an IRB is to protect human subjects participating in all research. According to the Food and Drug Administration (FDA) Guidance Applying Human Factors and Usability Engineering to Medical Devices[1], any product that has undergone usability testing at any point in product development should include a summary of preliminary analyses and evaluations that were used to inform the validation protocol in the Human Factors Engineering / Usability Engineering (HFE/UE) Report. Therefore, any usability studies that are performed and included in a submission to the FDA are subject to audit for any reason, including to evaluate human subjects protection measures.
The FDA also considers usability testing data for safety reporting in late pre-market and post-market applications that can be published in publicly accessible databases for review. As a result, safety data generated from usability studies should adhere to requirements of other human subjects studies that also produce product safety data (e.g., clinical studies).
What if the study is low risk to participants?
Even if a usability study is considered to be low-risk by the study team and likely to not need IRB oversight, then it is the responsibility of the study team to present the usability test plan to the IRB and allow the committee to determine that status. It is possible that a research study is low-risk to a participant and does not require IRB oversight, but an independent IRB committee should be making that determination based on the protocol and risks associated with the study, not those sponsoring or performing the research. There are cases where seemingly benign activities present more risk to certain research participants. IRBs can help study teams navigate those risks for usability testing.
Additional drivers for an IRB
With the constant global evolution of regulatory and social landscape, it is advisable to submit your test plan complete with a data plan for evaluation. The test plan should identify how your team will collect and protect the personal information of research participants so that it can be evaluated by a team familiar with the latest knowledge of the evolving requirements for human subjects and usability data.
Other considerations
While IRBs are primarily designed for the protection of human subjects, they also serve as protection for the research team conducting the usability testing. If something goes wrong during testing, having IRB approval for your study protocol would be the first line of defense for the study team and study sponsor. Having independent IRB approval demonstrates that the research team and sponsor have:
taken precautions to carefully consider risk and protections for human subjects
allowed an independent entity to ensure that the research team is properly trained and qualified to ethically perform research with human subjects
Final thoughts
While some researchers may view IRBs as an obstacle to research, independent ethical reviewers such as IRBs protect the interests of both research participants and research teams. Given how government regulations define research, it is both required and within the best interest of all parties to submit research for IRB evaluation before research activities begin.
Sources:
[1] U.S. Department of Health and Human Services. Food and Drug Administration. Center for Devices and Radiological Health. Applying Human Factors and Usability Engineering to Medical Devices Guidance for Industry and Food and Drug Administration Staff, 2016.
About the Author:
Ray Talbott, MS
Ray brings over a decade of experience in executing Human Factors activities for medical devices and combination products. Ray has extensive Human Subjects research experience across industry and academia from executing clinical trials to usability testing for clinical and commercial use. Ray is experienced in human factors and regulatory strategy, integrating human factors activities into design controls, and risk management.