ISO 14971 & Medical Device Risk Management
Highlights:
Risk Management is becoming an growing theme within the medical device community.
Ideas on how to bring usability information into a risk management file considering severities and probabilities for the medical device and software device industries.
Memorable Quotes:
“I envision risk management as kind of the backbone to product development and product life cycle.” - Shannon Hoste.
“FDA is very interested in incorporating 14971 now in the new QMSR” - Edwin Bills
“There's no such thing as a risk free medical device. You're always going to have some level of risk, and you just need to know what that is.” - Edwin Bills
Transcript:
Denise - 00:00:04: Hi everyone, and welcome to The Factor, a Global Medical Device Podcast Series sponsored by Agilis by Kymanox. I'm your host for today's session. My name is Denise Wagner. I'm Senior Director of HF Human Factors Engineering and Usability Engineering at Agilis Consulting Group. I've been in medical device for about 15 years and before that worked in Aerospace and Academia. One of the things I'm truly passionate about is risk management as it pertains to any product development process, but especially medical devices. I'm active on several standards committees and near and dear to my heart is the Usability Committee. So let's just jump in and let me introduce our guests today.
We have Edwin Bills and Shannon Hoste. Edwin Bills teaches the AAMI Risk Management course and is involved in the standards development for ISO 14971. During his 31 year career in medical devices, Ed has held a number of quality and regulatory affairs positions. He is an ASQ Fellow and is ASQ certified as a Quality Engineer, a Quality Auditor and as Manager of Quality and Organization Improvement.
Many of you know Shannon Hoste. She is our president of Agilis by Kymanox. She's also an Assistant Professor in the Quality Science Education Program at Pathway for Patient Health and is active also on several Standards Committees and Conference Committees for medical devices as well as combination products. So today we're going to be discussing the current developments around risk management for medical devices. So thank you both for joining me today.
Shannon - 00:01:53: Thanks.
Edwin - 00:01:54: Thank you.
Denise - 00:01:55: First off, Edwin, how are you today?
Edwin Bills - 00:01:58: I am doing just fine. Spent last week with Shannon at MedCon in Columbus, Ohio. And one of the things that I thought was kind of the underground theme of the whole program was risk management. Every one of the sessions I attended, risk management came up. So it's a theme across the medical device world, at least right now.
Shannon - 00:02:23: I was excited to see it. I envision risk management as kind of the backbone to product development and product life cycle. And I was excited to see at that conference, it was truly woven in through all of the discussions as that backbone. It's that foundation of decisions that we're making and so forth.
Edwin - 00:02:46: Yes, and the new stuff that's there, risk management popped up, there as well. The one thing I think, Shannon, that I'm most concerned about right now is we've got all these standards and documents that say you should do a risk analysis, but none of them say that those need to all go. Back to the risk management file because we have to consider the overall residual risk of a device, and that includes AI, cybersecurity, biocompatibility, electrical, all the things that are out there. And they all talk about creating risk analysis, but none of them say, well then you need to take that back to 14971, even though just about everything now has made that a normative reference except for ISO 13485.
Shannon - 00:03:40: True.
Edwin - 00:03:41: So risk management is a common theme across the industry now, I think.
Denise - 00:03:46: It is, and I agree with you that one of the issues, and maybe we can kind of unravel this in a little bit is like you were saying, there are just so many standards out there, and there isn't anything that's kind of bringing it together, particularly for complex devices. So a lot of manufacturers are just kind of handling the risk analyses separately but not looking at the overall risk of the system. So I agree that's something that's going to need special attention. Any key learnings from either of you from the conference?
Edwin - 00:04:19: Well, I think one of the things that I picked up is the FDA is very interested in incorporating 14971 now in the new QMSR, and they seem to spend a lot of time talking about 14971 and risk. And of course in 13485, the only place that there's not even a requirement, there's a recommendation in I think it's 7.1, it says in quality planning. See ISO 14971 for more information. Well, it's not really pointed to specifically except there, and it's kind of interesting because a lot of the definitions in 13485 came from 14971:2007, because that was the addition that was current at that time. And of course it came out in 2016 and then the new 14971 came out in 2019. So that's the reason for the differences.
Shannon - 00:05:31: I learned something interesting. It's a historical element, but I found it interesting because I had not thought about it before. Kim Trautman, was presenting on some of the history behind the quality system regulations, and she had discussed that when that was written would have been early mid 90s. It was referring to a risk management standard. 14971 wasn't around then. I was referring to a risk management standard that called that risk process risk analysis. So the terminology used in the preamble all talks about, quote unquote, risk analysis, which now we all know is a piece of risk management, but not all of risk management. And lots of times it was brought up that that referral reference to risk analysis shouldn't be misinterpreted as the FDA is only asking for that specific portion of risk management, but what they're really asking for is the full risk management. And it was just a terminology difference between what was in the standards at that time to now. I found that interesting. I hadn't thought about that.
Edwin - 00:06:34: Yes, if you look at ComEd 83 in the preamble, it tells you you have to do risk analysis, risk control, and then post market risk. It tells you that in the comment, because that was what risk analysis meant at the time. And that came from a European standard, EN 1441, which was withdrawn when 14971 was. I know all the history because I started in risk management in 2000 and got to hear all the how we got here, because when the standard was released in 2000, they were pointing back to the first actually, the first release of 14971 was done in 1998. And the time it came out is when they realized they needed to change everything from a standard with multiple parts to a single standard. They were going to do a one, two, three and said, well, people will be complaining about buying all those documents and maybe only buy the one that concerns them and all that. So we need to pull it all together so I can digress on history for a long time if you want, but I don't know if that's the point of today's.
Denise - 00:07:52: Yeah, I think the historical perspective that both you and Shannon have provided, though, kind of indicates why there may be some confusion. Right. So it's very difficult when the timing of standards, regulations and guidance are kind of staggered, and then as they're written, they're referring to perhaps old versions, and in the meantime, new versions come out, and so it's very difficult to keep everything consistent. But tell me about some of the trends. What are you seeing and trends pertaining to the overarching risk management process?
Edwin - 00:08:28: Well, there's a number of things going on. First of all is we're kind of in the quiet period. There's a five year time period on a standard from when it's released to when they go back and look at it again. So we're in that time from 2019 to December of 2024, which is not all that long away, when they'll have the look back and say, okay, what have we learned over this period of time and what do we need to change? Now? Historically, every time 14971 has been reviewed, everybody says, don't change the process. The process is good, but we need more information on how to implement it. So continually, our guidance is expanding. The current 24971, for instance, is 85 pages of guidance document information, and that doesn't include the annexes A, B and C in 14971, which is more guidance. So there's a lot of information out there to help people, but people don't buy 24971. For one thing, we advertise the heck out of it when I go to conferences and when the other members of the standards committee always say, buy that document because there's so much good information in there. And it was all revised and updated during the last revision of the standard. So it's all new information. And there's some really important things that are in there that people don't look at. Things like how to use international standards in doing your risk management process, because you can skip a lot of the steps because the standard is having you do those things that are also in 14971. So you don't have to do it twice. You just do it once following like, let's say 6061 or say 62366. Those standards tell you what to do well, you just need to take the results and put it in the risk management file. Then when you get to overall residual risk evaluation, you have all of those risks to include so that you're making a true estimate of the overall residual risk of the device. So there's a lot going on behind the scenes that is still part of people don't understand. They still think it's putting documents in the file. Not true. What I always start my classes with is the patient. That's our focus. Now, what if you were the first person that was going to get this device that you just designed? How would you do risk management? Would you do it different than what you're doing right now in your development of a product? And that's the way you should do it with a focus on the patient and making sure that you're making a device that is safe as possible. Now, one thing we mentioned in the risk management in the Preamble is there's no such thing as a risk free medical device. You're always going to have some level of risk, and you just need to know what that is. And you need to know what you need to tell the users, the medical professionals and the patient about what risks are still present so they can make a decision on is this the right device to use or not for my condition.
Denise - 00:12:05: Right.
Edwin - 00:12:06: So there's a whole lot of things that people aren't pulling into the picture. They're still focusing on. I've got to have these documents in the file.
Shannon - 00:12:16: Checkbox.
Denise - 00:12:17: Right.
Edwin Bills - 00:12:17: Yes.
Denise - 00:12:18: Shannon, how about you? What's kind of a trend and the application of risk management that you're seeing? Maybe a trend and a tip.
Shannon - 00:12:27: Well, I was thinking, as you were talking, some of the things you mentioned, and you mentioned it earlier, and then I think it came back up again in what you were just discussing, the area specific standards. So human factors 62366 software and other process standards that define a risk based process to essentially design safety and effectiveness into your product. Right. Each of those processes discusses how to evaluate and we'll talk about usability, since we're here use related risk, right. 62366, it breaks down how to do that process and how to identify, based on potential severity of harm, what you want to take a look at, and it drives what human factors activities you'll be doing to evaluate that further. There's still the question then of how do you take that data and roll it into your larger risk management file. So based on human factors, for example, I'm going to be looking at the things that are the highest severity harms. I'm not going to be thinking about probability. The severity is telling me what to focus in on. All of that information, I gather, is now informing my risk management file. And I can start to understand with that and all of the other areas of potential hazards and hazardous situations. How does that all speak to overall risk and residual risk? I think that is some of the challenges you were referring to in the integration?
Edwin Bills - 00:13:57: It certainly is. There's a couple of things I picked up there. One is you use the word probability. Probability is in the standard because we couldn't use the word likelihood. Likelihood is not translatable into other languages as directly as probability. In a lot of languages probability includes likelihood but in English we tend to think of we got to have a number when we say probability and that is not the requirement of 14971. In fact, if you use numerical probability then you have to have statistical validity on the data that you use to come up with that, especially early in design. In fact, probably until you get to at least design validation or usability validation or product use, you're not going to have valid data to support those probability estimates. So we always recommend that when you start the design that you use qualitative probabilities so that you're getting something that is reasonably good as you can get you can get ranges but you can't get the exact value. And that's I think something that's important is we don't start with a probability that is quantitative, we start with qualitative and then as soon as we have that statistically valid data, we have good confidence levels, then we can switch over and we will maintain that through the lifecycle of the device.
Shannon - 00:15:40: Yeah. And I think the challenge with human factors and use related risk, I'll call it that, is that it's identifying potential causes to hazards and you're evaluating those and the hazards that can become hazardous situations and you're evaluating that based on severity. But when I pull that into my larger risk file, I may have an understanding of probability of hazardous situation to harm. So I might be able at that point to start thinking about what is my level of residual risk. Sure. But I see a lot of companies struggling with that concept of they either try to build probability into their human factors activities, which is problematic, or they kind of throw their hands up of, how do I bring this Usability information into my risk management file when it's all these severities and these probabilities of one and so it kicks everything high? It's a similar dilemma that the software risk management folks have.
Denise - 00:16:36: The same thing with software, right. They don't use probabilities either.
Edwin Bills - 00:16:40: One thing that is timely is today on the regulatory focus, part of wraps question and answer session is there was a person that wanted to know how to develop effectiveness of labeling because all of the risk control measures you're required to provide the effectiveness, implementation and effectiveness you're required to verify that and they want to know how to do that. And I pointed to the fact that labeling is covered in 62366 and that's where your validation or verification of that activity comes up in the Usability validation area. So if you want to measure the effectiveness of labeling, you do it through Usability.
Shannon - 00:17:35: Yes, absolutely.
Denise - 00:17:45: Thank you both, for the wealth of knowledge and advice that you've provided our listeners and listeners. Make sure to subscribe. We will catch everyone on the next of The Factor. Thank you.
Stay tuned for Part 2: We discuss the complexities of medical devices including AI. Can risk management keep up?
Like this episode?
