Part 2: ISO 14971 & Medical Device Risk Management

Highlights:

• Review the importance of traceability, proper terminology, and the early integration of risk analysis as a design tool.

• Can risk management keep up with the evolving medical tech landscape?

Memorable Quotes:

• “I think we have a problem in risk management is I think we have a language problem. A lot of the terms are somewhat common language, hazard, hazard situation, harm, risk. Risk is something we use all the time and all of those have very specific meanings in the standard. Probability is another one that they have very specific meaning in the standard or within the file.” - Shannon Hoste.

• “Risk analysis is a design tool that you do to identify design requirements, which then feed into features.” - Denise Wagner

• “Probability of a hazard occurring is not the same as the probability of harm.” - Edwin Bills

Show Links:

• Part 1: Watch here

• Documenting medical device risk management through the risk traceability summary

• Implementation of risk management principles and activities within a Quality Management System

• Medical devices — Guidance on the application of ISO 14971

Transcript:

Denise  - 00:00:04: Hi everyone, and welcome to The Factor, A global medical device podcast series powered by Agilis by Kymanox. I'm your host, Denise Wagner, Senior Director of Human Factors and Usability Engineering at Agilis by Kymanox.

Shannon Hoste and Edwin Bills are back for part two of our series on risk management. In part one, we chatted about the overarching risk management process for medical device development. If you haven't listened to part one, do that first, then come back to this one.

(Part 1: Watch here)

Today we continue the conversation, digging into the complexities of medical devices, including AI and other MedTech. Can risk management keep up with the evolving medical tech landscape?

Part 2: What about complex systems? So medical technology is becoming more and more complex. In the beginning, as you were saying, now we even have a risk management standard for AI. So really, that's the way medical technology is going. It's software enabled, it's complex. Are there any plans for 14971 to speak to the complexity of medical device systems and how you break down the risk management for the system, down to the subsystems, down to the component level, and then inductively carry those risk back up to the system level.

Edwin - 00:01:38: We haven't up to this time, looked at it from that direction, but what we do require is traceability within the risk management process. So you need to trace from the hazard to the risk analysis, to the risk evaluation, to the implementation and effectiveness of the risk control and the residual risk. And I wrote an article in, was it the Horizons publication of AAMI in 2015, which had, and everybody hates I know, Excel, but it had a spreadsheet that broke out all the different things that need to go in there, like biocompatibility, usability, the electrical safety, all the things in categories. And what we did is we developed a spreadsheet that had each category shown, and you brought your items over from whatever your analysis document did. And the one thing I did that was different from there's a GHTF-risk management guidance that was released in 2005 and it was based on the 2000 version of a standard. It's still pretty good today. But what I did is I broke out and said, okay, where'd the hazard come from? What document and what line item in that document can I go back to and find that information? So as I move forward, I can go back and look at what's the information that supported that all the way through. Because when we release the device to manufacturing and we sell it, someday, probably within 20 minutes after it's put into production, somebody's going to come up with a design change to reduce cost or something. So I need to be able to go back and look and see how that would impact the risk of that product. And that traceability function of the standard is how I can tie all that information together and go back. So I've got all the usability, all the electrical, all the biocompatibility, all of those there and the supporting documents. In fact, one of my clients when we first introduced that would probably been in 2009 or 2010, used that for a notified body audit when they said we want to look at your risk management system. They laid that traceability document down. The guy looked at and said, okay, let's go to the next subject. Because they saw everything there was all tied together. There's documentation connected and obviously you've done what the standard says. 

Denise – 00:04:20: Perfect. 

Edwin – 00:04:21: So that's an approach that worked for them.

Shannon - 00:04:24: We should provide the links for some of those documents because some of those are public documents and we can provide the links in this at the bottom, underneath this podcast. I was thinking about that and those are great references. And what I like about that is it provides you have the top level traceability for the product and then everything is feeding into it. It gets back to that kind of the backbone concept that I mentioned earlier. So I can feed in my understanding around use related items and I can feed in my understanding around software and so forth. And I think part of the devil in the details with that is the complexity of that is going to mirror the complexity of your product. So if I'm talking about surgical robotic system, it's going to be pretty complex and that traceability is going to have many references and tracing. And what I see as a problem with that and what I think we have a problem in risk management is I think we have a language problem. A lot of the terms are somewhat common language, hazard, hazard situation, harm, risk. Risk is something we use all the time and all of those have very specific meanings in the standard. Probability is another one that they have very specific meaning in the standard or within the file. And it's really easy for those to get out of sync, especially when you're talking about a large system. And so what I'm calling a hazard in one document might actually be a harm in another document if you're not keeping it clean. I think that's a challenge. The other challenge I have is probability without a context, a probability of what is really difficult to factor in. So if I'm looking at a failure mode probability of this product fracturing that's the probability of that failure mode. That's not the probability of then that failure mode becomes a hazard. I've got the probability of that hazard and then the probability of that hazard becoming a hazardous situation and a harm. And so I see a lot when I look into complex systems that folks are kind of running around in circles because they don't speak to each other well, the documents don't speak to each other well. Have you seen that as well?

Edwin  - 00:06:43: Definitely. And that's why actually the hazards of situation and the harm. And that whole concept, the P1, P2 concept, was put into the guidance because of those questions that were going on after the release of the first edition. We developed that as an educational concept. The idea was not to establish a requirement that you have to do P1 and P2. Because people read that and said, I have to do that. No, that was to explain why the probability of a hazard occurring is not the same as the probability of harm. 

Shannon – 00:07:20: They're different. 

Edwin – 00:07:22: And you have to get through the hazardous situation exposure. And the old shark example that we used way back in the development of that concept helps to explain that. Well, the first place, they're not reading the guidance. They're not reading and understanding the guidance. And people just look at the standard. They don't even look at the definitions, and they jump right into, I got to do this stuff for the file. I got to create this risk analysis, put it in the file. I've got to have these risk controls and all this, and that has to be in the file. They're all worried about documentation and not worried about the outcome. What we're going to do here to improve the safety of the product?

Shannon - 00:08:14: It gets me so frustrated because there's so much value in creating those documents the way they're intended, because they should drive your design decisions and your focus and your budget and all of your activities. It should be a driver of those, not. 

Denise - 00:08:29: Yeah. I think that's something that has been a frustration for me, is that often times engineers see the risk documentation as something you do after the fact, rather than risk analysis is a design tool that you do to identify design requirements, which then feed into features. And so then doing it early and often is what is going to lead to designing out the risk. Right. Rather than go through the whole process and then try to I've seen it also as a laundry list of taking credit for the risk controls that you built in, rather than preliminary risk analyses, identifying where you need risk control. Right. So it's not being used iteratively as part of the product development process. Often maybe I shouldn't say often, but in some instances, I think people see design control and risk as very separate things, not complementary.

Edwin - 00:09:43: In ISO 13485, they helped us, and the requirement is 7.3.3C. Is that the outputs of risk management are the inputs? Are the design inputs?

Denise - 00:09:56: Yes.

Edwin - 00:09:57: That means you have to have gone through risk analysis and done that before design input.

Denise - 00:10:02: Absolutely. And that's that traceability that gets really complex for complicated systems. Right?

Edwin - 00:10:09: Yeah. And one of the things is you can't do FMEA and meet that requirement. FMEA only?

Denise - 00:10:16: Yes.

Edwin - 00:10:17: FMEA requires you have design output before you can perform it. You've got to have the design available to look at the components. If you didn't have the components and know what they are. You can't do FMEA well, it comes too late in the process, but it provides a check for you to see that you haven't missed anything by looking at the failure modes, but the cost and the time that you're going to have. If you wait until design output before you do risk analysis, your product time to market and cost is going to be much higher. So you need to be working if you're doing clinicals. For instance, there's a requirement in the 14155 that you have to have risk analysis before you perform the clinical trial. That's where we start way back there, gathering information and folding it in. So some of those risks that show up in your traceability summary could come from clinical trials. And there was a segment I put in there for that too. But the idea is preliminary hazard analysis is a great tool to use because you can look at the standards. For instance, if I know I'm making an electrical device, I can go to 60601 and I can pull all the hazards that are already identified there and put them in my risk analysis, and I can skip right ahead to the risk control measure. I don't need to do the estimates of probability and severity. I could jump ahead because it's already identified a risk control measure, and then it already has identified a verification of effectiveness test. And if you do those according to the standard, you don't have to do anything else. You just put that in your risk management file, in your traceability summary and show where that came from, point back to 60601-8.3 or whatever it is, and that's where this came from. I've reduced the amount of effort that I have to do because somebody has already done that in a standard that is one of those standards that is accepted by the industry, by the FDA, too.

Shannon- 00:12:42: You can do that day one of the project. 

Denise – 00:12:44: Yes

Edwin – 00:12:45: Yes, you can do it day one.

Shannon – 00:12:47: It's going to be electrical. Okay, now I have.

Shannon - 00:12:50: I know that and it's going to be in contact with the patient biocompatibility.

Denise - 00:12:57: And so Shannon, when you said before you start the project, so what I like to tell people are these are activities, your preliminary hazard analysis that can happen before design control. This can be part of your playing in the sandbox, your feasibility assessment, which helps you in fact, scope your development process.

Shannon - 00:13:18: How much better would your project be scoped?

Denise - 00:13:20: Yeah, if you do preliminary hazard analysis before phase one of design control and risk management, you can scope design control and risk management and your product development process and really identify what is your time to market. So often that is not leveraged in the way that it can actually scope the entire development project.

Edwin - 00:13:43: And in combination products, we have an AAMI guidance document there, TIR105, that talks about the fact that we've got drug risks and device risks and then we put the drug and device together. Now we have interaction risk between what could be the chemical attack on the syringe that's made out of plastic that might introduce a new risk. So you have to do the individual components, the constituents of the product and put those risks in and then the interaction risks and put those in as well. So there's things beyond just the individual elements and what happens when you combine that all together in a system. We were talking about systems earlier and there's a system safety engineering area of expertise that has been around a while and got some great things to include. But we have to then again look at the terminology which is different there than it is here and all those things to try and understand. Okay, what is the real picture for a medical device or a combination product? System safety is a great concept because it goes back and does what I said earlier. It's being the patient and looking back at the device from that direction and seeing what you see as a whole system, you don't see the individual parts. So that's an important concept that we need to consider when we're building our risk management system in our different companies.

Shannon- 00:15:32: I agree. The other challenge with combination products, you have the interactions and you also have a lot of times with the device. You don't necessarily have an intended use or intended purpose until you couple it with a drug for a combination product. And so now I may have one device that's used with different drugs and so therefore it has different indications, intentions, purpose, and so your patient perspective can change what might be okay for one patient group might be problematic for another based on the device function or drug reactions. And there's many factors there. It's challenging enough working in combination products, getting medical device systems and pharmaceutical systems to speak to each other, just quality systems, but then to get risk management files to mesh well can be another challenge.

Edwin - 00:16:26: Well, there's two TIRs, AAMI for combination products because combination products in the US is different from other countries in the world. Susan Neadle just finished the Combination Products Handbook which was pre-order opened last Tuesday, and I happen to collaborate on the chapter on risk management there. So that was 70 pages. That's going to be a good read when People 

Shannon – 00:16:53: Usability. 

Edwin – 00:16:54: Yes, it's all in there, but the idea again is trying to understand the whole area that you're working in and what I need to do to meet those requirements. Looking from a big picture perspective and then focusing down. TIR48 is in final review process right now. We've got a review that I need to get done this month which talks about the different GMPs and how you use them for a combination product. And if you're coming from the drug side and incorporating the device, you follow the drug GMPs, but also these parts of the device GMPs. And now we're having to revise the device GMPs with the new emphasis of 13485 over the old 820. It's still going to be numbered 820, but it's going to be a new 820. We've got to look back and see, okay, what's changed there? But we already have that published, so that'll be in the next revision, I guess because we're too far along. We're ahead. Like all standards development, you're always out of sync. Okay, we know the new 820 is coming probably in December, it looks like, from what we heard in the session last week. Is it's currently on the FDA's schedule for release in December, although that's subject to change in June. They're going to reevaluate and decide, but that's coming. And then the transition period will be at least one year. But there were a lot of comments in the initial release that wanted two and three years. I don't know why anybody would need three years, but they're 95% the same 820, and 13485 almost the same. So why should you take a long time to anyway, so that is what I'm talking about, about the out of sync thing. We're going to release the guidance document on how to do the GMPs when the GMP is going to change. Within a few months after that release, probably, we'll see what the comments are in this current round of reviews. Maybe somebody will say we need to include the 1345 instead of 820, or the it'll be 820. But that's what's confusing. There's an 820, 1996. Now there'll be an 820, whatever year it is.

Denise - 00:19:51: Yeah. So one thing I'm thinking, and Shannon, I love your idea. We will tag some of the standards, guidance, and regulations that you and Ed are pointing out in the podcast, along with the podcast, so people can refer to them. One thing that I think is overwhelming for manufacturers that we discussed two weeks ago in the AAMI Usability committee meeting is that there are so many of them. There's a proliferation of standards and guidance and regulations, and how do manufacturers use these together? So one action item that came out is designing an infographic of what are all the standards, guidance, and regulations and when do you use them? And I think the Overarching Design Control, Risk Management Usability, we really need something that is like an information graphic that just lays it all out. And what are all of these documents and how do they play together? So maybe future work, another committee?

Edwin - 00:21:00: Well, I heard about something last week that jumps way beyond that. Somebody's developing an AI application.

Denise - 00:21:09: Absolutely.

Edwin - 00:21:09: You put in what the device is you're trying to develop, and it will give you all the standards. I better apply. 

Denise - 00:21:15: Yes, I've heard that. That's a great application of AI, but I think it's time to start wrapping up. And I thank you both for just all of the wisdom and knowledge that you've imparted during this podcast. I think it's going to be excellent for listeners to hear and figure out how to apply. Right. Any any closing words on, you know, if if you were to give a manufacturer one tip on risk management, Ed, I'll let you go first and then Shannon, what would that just one tip be?

Edwin - 00:21:59: Well, I think manufacturers need to get involved in risk management, and the Standards Committee, especially 14971, currently is inactive, but it's Joint Working Group number one of  ISO/TR 24971. When the vote comes up, which, like I said, our five year period is up for vote in December next year, December of 2024. And internationally, the ISO and IEC National Committees will ask for input on should this be reaffirmed or revised. And you have to submit what you would see as what needs to be revised and how would you revise it. And that opportunity is out there for input. So, as you see, what are you having trouble with? Write that down and get it to your national committees. Was it 62? Is the IEC committee SC 62? And then on the ISO side, it's ISO/TC 210. We will be evaluating those. I think we had 60 comments the last time that were consolidated by ISO and IEC and all the national committees. Everything was pulled together into 60 categories that we assessed and decided what we needed to do, and all of them were more guidance. We did have one requirement from ISO is that we fixed the problem that had everybody looking at the 2007 edition of 14971 as all requirements and not separating out the guidance. So that's why we moved so much into that 24971 document. And so that's separate there that's guidance. Although Annexes A, B and C in 14971 are also guidance, the first one is the rationale for all the requirements. So if you want to know why we got that darn requirement in there, go read the rationale and it will tell you why that's there. The next one is B, which has the big rats nest flowchart that shows how it's all interconnected and that's what risk management really looks like, rather than that nice, neat straight line flowchart in figure one, go to Annex B and see how it really works. And then Annex C with some basic requirements, and we talk a little bit about differences between hazard and hazardous situation. And it explains give some examples in there, some very good examples to help you clear that up. And then if you go into 24971, I know I wrote an article once on what's the difference between the policy, the process and the acceptability criteria. All those things are not understood well. I wrote an article that was in Med Device Online probably early 2020 when that came out because we knew it was going to be a problematic thing. But we had an annex in there that addressed that, Annex C addressed that. So lot of good stuff out there if people would go look for it and read it and then ask questions if you don't understand. I make myself open on LinkedIn and on the Reps forum, and I address every question that I see that pops up related to risk management. So we're trying to educate and do forums and podcasts and all kinds of things to help out.

Shannon - 00:26:15: Thank you for that.

Denise - 00:26:15: Yeah, thank you. Shannon, what is your one tip? 

Shannon - 00:26:20: If you’re working in a system and you don't have an efficient way to develop and maintain your risk management files? And what I mean by that is understanding your product risk, but understanding it from the standpoint of every activity you do along the way to product realization should be coming back to that risk management file for you to make decisions. If you don't have an efficient way to do that, find one. Figure it out. There are scalable. That process should match the complexity of the products that you make. Figure out a way to make that efficient because it is only going to become more and more important with FDA regulations, with IVDR, with MDR. It's only going to become more important that you have a clean, clear risk trace, and it's essentially a design story behind how you're assessing that's.

Denise- 00:27:20: All right. That was Shannon Hoste and Edwin Bills. Thank you so much for coming on the show. Thank you so much for listening or watching this episode. Please subscribe or follow this podcast in whatever app you're using right now. Or follow Agilis by Kymanox on LinkedIn for all updates. This episode is edited and produced by Earfluence. I'm Denise Wagner, and we'll see you again on The Factor.

Stay tuned for our next episode with Susan Neadle as we discuss her book on Combination Products.

Like this episode?