Home based Human Factors evaluations

Home based Human Factors evaluations

How can we best develop medical devices for use at home? Demonstration of safe and effective use of medical devices is generally obtained via human factors formative evaluations and validation studies. These human factors studies often take place in simulated use environments that replicate use conditions and environments that users are expected to experience. For home use devices, including combination devices, over the counter (OTC) products, and many others, it is expected that users will engage with their devices in their homes and other non-clinical environments. Use environment profiles are intended to document the typical environmental characteristics of non-clinical environments, but as human factors practitioners, how do we know what the actual conditions of home environments are for any particular user group? From a device development perspective, numerous challenges exist with identifying non-clinical use environment conditions. FDA acknowledges this fact and has identified 3 challenges related to home use of medical devices: caregiver knowledge, environmental unpredictability, and device usability [1]. With simulated use studies, environmental unpredictability can be difficult to replicate, yet we know that it is a major challenge for home use medical devices!

We must heavily assume that home environment conditions are effectively identical across all individuals in a user group, and the reality is that those home environment conditions are likely not the same. The assumption that home environment conditions can be represented via simulated use is an assumption that is generally accepted by regulatory agencies for the purposes of human factors studies, but for sponsors involved in the development of new products or with an interest in post-market use problems, it can be very valuable to discard this assumption and go directly to the home of users in order to truly evaluate the expected use environment.

Let’s discuss some of the differences between home use and simulated use environments and how those differences could get in the way of successful product development or usability evaluations.

Difference #1: White Coat Syndrome

White coat syndrome is general anxiety experienced by patients in clinical settings that results in higher than normal blood pressure. However, this anxiety experienced in clinical settings often occurs in simulated use human factors studies as well. Participants often comment on feeling nervous while being observed using a medical device in simulated use testing. This feeling is understandable since participants have likely never used medical devices while sitting in a research facility, being filmed, being observed from behind a one-way mirror, and sitting with a moderator while it feels as if their every move is scrutinized. Again, this effect is an accepted limitation of simulated use studies just as it is an accepted limitation in any human subjects research that is conducted in a research environment.

 When participants are observed using medical devices at home, however, at least some of these factors are removed. The participants are in a comfortable and very familiar environment where they may use a medical device every single day. Also, it is not possible to have observers behind a one-way mirror, so the number of observers in a home use study is typically lower. The increased comfort of a participant is likely to reveal true representative use and reduce white coat syndrome. In their own home environment, medical device users are able to demonstrate their device use at their own pace, with their own preferences for device use, and within a familiar environment For example, in a study observing participants using their own prescribed injection devices, information was gained on the location in the home where the injection is performed, actual time taken to prepare and perform the injection, actual lighting conditions, and deviations from product instructions that users had adapted based on their own use preferences.

Difference #2: Performance Evaluation Approaches

During formative evaluations or validation studies, user performance is evaluated as it links to critical use tasks which relate to the risk profile of a device. These performance tasks are closely observed during a human factors study and the user’s performance is categorized to evaluate their success in performing the task or the presence of any use errors or difficulties. This detailed performance evaluation is crucial in connecting representative use performance to risk in order to demonstrate safe and effective use of a device.

Earlier in product development, possibly during contextual inquiry, home use observation study performance evaluation may have a different goal. There, observation during home use, not representative use in a simulated environment, is likely the goal, so scoring of participant performance is less valuable than gathering holistic information about device use. Where in the home is the device used? What are the lighting, noise, interruptions and distractions in that room? Are there other people or pets in the home while the device is being used and does this effect cleanliness of the device use space? Does anyone else participate in the device use to assist the user? How closely does the device use adhere to the instructions and what adaptations has the user made on their own?

During early product development, information from home use could be invaluable in optimizing device design. For example, if users consistently indicate they use a device in their bedroom because that is where they have the most privacy, then it cannot be assumed that overhead lighting will be as bright as is expected in a kitchen or bathroom. If users consistently receive assistance from a family member to use their device, then this information must be considered in the instructions or the overall design must be reevaluated to potentially optimize it for a patient to be able to use without assistance. If a patient demonstrates adaptation to use the device that deviates from the instructions, then it is possible that the instructions need to be updated since the patient may not be able to effectively use the device by following them. 

Difference #3: What Cannot be Simulated

The FDA [1] outlines specific potential hazards for home use medical devices including pets, unsanitary conditions, children at play, plumbing, temperature extremes, dust, fire hazards, tripping hazards, poor lighting, and background noise. While it may not be possible to optimize device design to overcome all of these potential hazards, having a more representative picture of actual use of a medical device in a user’s home may provide vital perspectives on design mitigations that should be prioritized. For example, for a medical device that is often used by people between the ages of 25-50 where it is likely that children still live in the home, it may be valuable to observe during home use to gain a perspective on how often children or pets are present during device use. If in an observation study of 20 users, children are present in 12 of the homes, it could be valuable to ensure that any buttons or switches in a device have a cover over them or require significant pressure to release in order to avoid accidental engagement by a child playing near the device.

Home use study case examples

AdobeStock_269468743.jpeg
  • In a home use study conducted by Agilis, 3 of 7 users had their pets present in the room with them while they were using a self-injection device. An additional 2 of the 7 users had pets in the home who they specifically moved into another room for the purpose of the study when they typically would not move their pet to another room. In that case, over 70% of users had pets in the home who were typically present and possibly interacting with the users while they performed their injections.

  • In the same home use study conducted by Agilis, 2 of 7 users performed their injection in very low lighting conditions, one in her kitchen and one sitting in a large chair in her living room. 2 of 7 users performed their injection sitting on the couch and using the coffee table to hold injection supplies. 3 of 7 users performed their injection sitting at a kitchen or dining room table with varying lighting conditions and one user performed an injection while standing in his living room and placing his injection supplies on a shelf. None of these actual use scenarios would be directly reflected in a typical simulated use study performed in a controlled research facility.

When participants in the same study gave feedback on their preferences for injection devices with packaging of different sizes, users commonly commented about the size of the packaging that would fit in their refrigerator. Several participants indicated they simply did not have refrigerators large enough to accommodate more than 1-2 devices on hand at a time which could be a problem if their prescription included receiving several devices at one time. While it was not the practice of the users Agilis observed and spoke with, it is possible that users could forgo refrigeration of their device if the packaging was so large that their refrigerator could not accommodate multiple devices. Once again, this information was obtained during home use observation and interviews of patients. While it may not directly impact device usability as tested through performance observation and scoring in a formative evaluation or validation study, feedback on device packaging size could have serious implications on the efficacy of medication if it was not properly stored as a result of packaging size.

If your organization is interested in approaching early product development by implementing home use studies of currently available established devices, please consider that the implementation of home use studies is unique and requires expertise in both human factors study design and in-home study design. When employed early in product development, home use studies may be invaluable to optimizing device design. Additionally, expertise in evaluating users with limitations, including environmental limitations seen during home use, is especially important for collecting data that can be translated to usable information to optimize medical device usability.

References:

[1] FDA. Home Use Devices. (2018). Available at: https://www.fda.gov/medical-devices/home-health-and-consumer-devices/home-use-devices.

Image Source: https://blog.athelas.com

 

4 Tips to Creating More Effective Instructional Materials

4 Tips to Creating More Effective Instructional Materials