Let's Discuss: How Should the Device Approval Process Ensure Software Safety?

By Jamie Hartford
Published: September 28, 2011

Over at our sister publication EE Times, Dave Kleidermacher, of Green Hills Software, sounds off about the controversial IOM report that recommended FDA scrap the 510(k) process. Kleidermacher agrees with the report's conclusion, and he's a more frank in his criticism of the 510(k) process than the IOM:

. . . [T]he medical industry's certification process for life-critical devices, such as ventilators and pacemakers, is a joke because accreditors are not required to perform any kind of analysis of the device's software/firmware. A 510(k) FDA approval is essentially a rubber stamp: if the manufacturer can show that a product has essentially the same function as another device already on the market, then it simply doesn't matter if every single byte of code in the new product has been rewritten, relies on SOUP (software of uncertain provenance), or does not follow a quality development process. The government must rely on vendor self-management."

Tell Us What You Think

We want to know what you thought about this feature. Let us know by adding a comment.

ADD A COMMENT >

Kleidermacher cites a tragic example from the 1980s. People died as the result of software flaws in the Therac-25 computerized x-ray system. By contrast, he argues, the aviation industry—upon which the FAA has imposed a certification process with tough requirements for avionics systems—has never seen a jet fail as a direct result of flawed software.

Kleidermacher also counters the argument that more rigorous safety controls will curb innovation and reduce profits. He says tougher standards will knock out suppliers that don't have the chops, opening up more opportunities for those that focus on quality. He nixes the idea that "trends towards open source and powerful multimedia" can't be reconciled with safety, too.

I think he's right. It's obvious that the 510(k) process is broken, but what should its replacement require to ensure the safety of device software? Here's your chance to sound off. Please share your recommendations in the comments below.

—Jamie Hartford

Med blog Tags:
»

Ensuring Software Safety ...

Submitted by george@samaras-... on October 4, 2011 - 2:40pm.

It is very easy to make the FDA everybody’s “whipping boy”. Yes, the clinical voice (the end-user) is important, but the vast majority of clinical voices behave as if they could care less.
You want a standard UI? Have the AMA or ANA issue the specification and don’t buy anything that does not comply. Setting a UI standard is neither the FDA's nor the developer's responsibility.
“System use errors”, as opposed to “individual useR errors”, arise not only in development, but also in deployment. While device manufacturers and regulators are by no means “innocents”, once you bought it, it is your problem to deploy it and maintain/upgrade it correctly. Faulty deployment not only can create new system use errors, but can compound those created by the developers.
Until the adversarial relationship among developers, deployers, and regulators wanes, I think we will continue to enjoy the status quo.
Full Disclosure: I worked for the FDA and consult for industry.
GM Samaras, Pueblo, Colorado

It's about more than just FDA and Manufacturers

Submitted by rick schrenker on September 30, 2011 - 9:32am.

I'm glad somebody finally brought the FAA into the discussion. As I understand it, FAA regulates both providers and users. Ditto NRC. FDA does touch the user side a bit, but not very much, and the current uproar (or maybe just a tempest in a teapot) over the new MDDS regulations gives a glimpse into how the user side reacts to such an imposition.

Yet every day, in hospitals and other caregiving sites, technology comes, and technology goes. Drop by an ICU bedside and check out the menagerie of UIs a clinician has to make sense out of. Consider the complexity behind each, keeping in mind they were developed and approved for the market one by one, and that they are implicitly integrated into systems with devices that were not, and for all practical intents and purposes could not have been, designed to take them, and interacting with them, into consideration. Today the implicit integration of the past is being replaced with explicit integration via IT networks, both proprietary and general purpose (see, for instance, IEC-80001).

Consider now the use of medical technology in the home or any setting where users may be cognitively impaired users as well as emotionally and physically stressed caregivers, including family and friends.

It's a shame Andy Rooney is going away...

This very week a new phone showed up on my desk. It has a 3x4 screen with softkeys, one labelled CFwdALL. I had to look at a 9x12 instruction card to see what it meant. I still don't know how to use it, but I'm sure I can figure it out when I need to. Hopefully it won't be during an emergency... Sometimes when I pick up the handset and there's a dial tone. Sometimes there isn't, but that's OK because all I need to do is click the NewCall softkey to get one. Hopefully I figured it out right, and there's no other scenario I'm overlooking that could matter, e.g., in an emergency.

The phone even has a Barge softkey, although it's in a menu I haven't found yet (I wouldn't even know about it without the 9x12). But at least the name for the key is intuitive. There's also a RmLstC softkey somewhere; it appears to be an antidote to Barge. Maybe. Hopefully I can figure it out when I need to.

A million lines of code? Quite frankly, from this side of the fence I couldn't give less of a damn if I tried. My first radio had its number of transistors emblazened across it. My first uP had a whopping 1k ROM and 4k RAM. In the latter two cases, those facts were worth knowing in conversations with fellow nerds and geeks but nowhere else. The same goes for LOC, whether you guys get it or not. I care about reliability, usability, availability, supportabilty, interoperability, and a slew of other qualities. More to the point, I care about them in the context of my organization's points of care and the systems in place to support it. Give me an assurance case that addresses these and other qualities. Show me explicitly how LOC matters in the context of my concerns and what you've done to manage threats to my organization's ability to provide care. And stop feeding my colleagues and me the line that what you do to achieve these qualities is proprietary. Heaven help us both if we have to use that excuse at a Congressional hearing someday.

So yeah, the current process has weaknesses. But if you think you can fix them without bringing the clinical voice to the table, good luck with that.