Do You Make These 5 Usability Mistakes In Your Medtech Development Process?

Imagine you’re a nurse working at a busy hospital in a major metropolitan area. A patient care technician comes up to you and asks you to look at a glucometer reading for a patient with a history of poorly controlled blood glucose levels. An alert box flashes across the glucometer screen with instructions for treating high blood glucose levels. You talk to the patient, and she says she feels like her blood sugar is elevated. You recheck her levels, and the same “high” message pops up again on the glucometer. You decide to administer insulin and call the physician.

Unfortunately, by the time you get the patient back to her bed, she has become unresponsive. The rapid response unit has to be called in. They determine that the patient’s blood glucose levels are actually critically low, despite what you saw on the glucometer. The patient is immediately transferred to intensive care. Soon after, your supervisor informs you that you are being suspended, pending further investigation into the matter.

What exactly went wrong?

Before answering that question, I should tell you that this incident actually occurred — at one of the 10 hospitals operated by MedStar Health in the Baltimore-Washington, DC area. The story was relayed to me by Andy Schaudt, director of usability services at MedStar’s National Center for Human Factors in Healthcare, which provides medical device usability testing and consulting services both to MedStar and to industry. Working with manufacturers on the development of medical devices from concept to launch, and then seeing such devices implemented and used within MedStar’s hospital system, gives this human factors center a unique perspective on device usability.

Now, back to our story. Hospital management asked MedStar’s human factors center to come in and investigate the adverse event with the glucometer. What they found was that the popup alert — a default message that was clearly incorrect in this context — covered over the actual blood glucose reading on the device’s screen. Essentially, the device interface design failed the nurse, and ultimately the patient suffered. (The nurse was soon reinstated and the patient recovered.)

“This type of error was completely avoidable if usability testing had been implemented in the design process,” Schaudt told me. “The error would have been identified and designed out before a problem occurred.”

Yet episodes like this still happen every day, resulting in costly recalls, redesigns, and potentially litigation for medical device and diagnostics manufacturers. But, says Schaudt, you can prevent such problems from occurring by establishing a human factors culture at your company, and by avoiding these five common usability mistakes.

Mistake #1: Focusing on the customer and not the user
When a medical device company looks outside the organization for user input, it usually tasks the sales and marketing divisions with determining what’s needed in the next design iteration. Their job is to give the customer what they want. Unfortunately, when it comes to medical devices, the customer isn’t always the user.

In a multi-hospital healthcare system, for example, the customer is often a procurement division that handles purchases across many different hospitals. The procurement department does have legitimate knowledge about the overall system and its infrastructure, and how implementable a device will be within that environment. But they won’t actually use the device, and in some cases they may not even talk to the users — the actual unit in a specific hospital — before making a purchasing decision.

Schaudt recommends that device makers take a systems approach to gathering design input. “You’re missing the whole point of user-centered design if you’re not putting the device in the user’s hands and getting their feedback,” he said. “A systems approach looks at the entire environment, system wide. It looks at the close-in environment such as the unit. It looks at influences such as processes and policies in place, as well as the users themselves and if they can actually use the device without error.”

If you cater to the customer at the expense of the user, Schaudt warned, you may end up adding new features to your device that will rarely be used. And worse, these features can actually increase error rates.

“Adding features can introduce noise, which can make it even more difficult to use the product. Typically, only four or five major features are used,” Schaudt said. “So I can’t stress how important it is to adopt a user-centered design process and not a reactive approach, where you get a minimum viable product on the market and then rely on your marketing team or sales team to ask the customers what they want to add or change. Sometimes there is a mismatch between customers and end users.”

The benefits to taking the right approach are tangible. “Engaging users and collaborating with healthcare systems can be a differentiating factor, because you are making the most easy-to-use, straightforward, and safe devices out there, Schaudt added.

Mistake #2: Believing that users don’t want — or have time — to provide design input
Some device makers truly believe that user feedback is important, but mistakenly think that users either are too busy or uninterested in working with manufacturers to improve device designs. “There are perceived barriers there, but I’ve found that healthcare systems, especially the users, are really open to trying to share the problems they’re having with a device,” Schaudt explained. “They want their life to be easier and the device to be something they can use.”

Schaudt’s Usability Services division regularly recruits people from within the MedStar Health system to help its medical device customers innovate. “We see users who are very interested in getting involved in new technology — things that can make their daily lives better or their daily work schedules easier. We haven’t had any issues getting people to come to the table to get involved in this. They want to participate,” he said.

It may seem as though the MedStar human factors center has an unfair advantage, embedded as it is within a major healthcare system. However, Schaudt ensured me that there is absolutely no coercion involved. As any device manufacturer would, the center provides users with some level of compensation for participating, but not so much that they are effectively buying feedback.

“It is enough compensation to make it worth their while to spend a little time with us,” he said. “There’s definitely no pressure involved. That’s why I feel confident in saying that they are interested.”

Mistake #3: Failing to account for all possible users and environments
Healthcare is at the beginning of a major transformation. A major shift is occurring, moving from the traditional hospital-centric care model to a more distributed care network.

One of the ramifications of this change, according to Schaudt, is that medical devices will need to be designed for use in both clinical and nonclinical settings. They will have to be more versatile, serving a wider variety of intended users and environments. “The next generation of devices will need to be designed for hospitals and clinical facilities, but at the same time understanding that they could be pulled out into the field and put in homes,” Schaudt stated.

The shift won’t happen overnight. Once a medical device enters a healthcare system, it typically stays there for a while. So there will be a transition period between today’s devices and devices made specifically for nonclinical settings. Until then, the current generation of medical technology will be pulled out of the clinic and moved around to different settings.

“This brings a lot of opportunity for medical device makers to start to branch into this arena, where they’re redesigning products for nonclinical use,” Schaudt said. “But it also brings a lot of challenges in the interim.”

Usability engineering will be key to navigating this transition. You need to fully understand all the possible environments and intended users. Could your device be taken out of the hospital? Could it end up in the home? Who will the new user be? Will patient potentially play around with it more often than they would in the hospital?

“There is a difference between ‘work as imagined’ and ‘work as performed,’ as we like to say in human factors,” Schaudt explained. “You imagine that your device is being marketed to nurses, for example, but it ends up in the home, because they start to do more home visits with it. Now you have the potential for another user and the device being used in ways you never intended.”

He added: “So I think device manufacturers need to start thinking about that, thinking outside of the box about who really is going to be using their device two years, five years, or even ten years down the line.”

Mistake #4: Not having a formal HFE process in place
In some cases, manufacturers expect a test engineer or their industrial design team to handle usability during product development. These employees may have some human factors engineering (HFE) experience — perhaps they helped with a human factors submission to the FDA at some point — but typically not enough to ensure an optimal design for usability. “Often you find test engineers trying to learn about human factors using the ISO standards, or industrial design teams trying their best to make a design more usable,” Schaudt explained.

For user input, they may first turn to a former clinician on the team — for instance, a product manager who used to be a nurse. The problem is they tend to overly rely on these “power users” and their subjective feedback. “In scientific terms, this would be an N of 1,” Schaudt said.

What’s missing in such situations is a formal process, one that strikes the ideal balance between subjective feedback and objective performance data, according to Schaudt. Not only do you need subjective feedback from a representative sample of users, you also need to set up scenarios to test different tasks that the users go through with the device, in order to identify objective measures to inform your design. “Implementing a robust user-centered design approach requires a more formal process,” Schaudt said, “and it usually requires more than just one or two people.”

Mistake #5: Not incorporating usability early or often enough in the product development process
In 2011, the FDA issued draft guidance on applying human factors and usability engineering to optimize medical device design, which has led manufacturers to increase their focus on HFE. While this is a positive development, Schaudt believes it may have had an unintended consequence — that manufacturers are putting too much emphasis on summative testing evaluation, which is required to get a device through FDA.

As a result, usability and human factors thinking tends to happen closer to the end of the development process, when a device maker is trying to validate. However, usability engineering delivers much more value when it occurs during the formative phase of development, which is the work leading up the summative phase.

Schaudt challenges medical device manufacturers to step up their game and start incorporating usability early and often in the product development process. “The takeaway for device manufacturers is to establish a usability and human factors culture early in the process and use it throughout,” he said. “The summative test is really just a validation of the design work that came before, so it should be less of the focus. The focus should be on the formative side.”

Incorporating usability early in development and using it often will ultimately save costs in the long run. You will have fewer design recalls, issues, and pauses in your development process, and overall, it will lead to better patient outcomes.

Make Human Factors A Priority — Before Healthcare Systems Force You To
If you’re still not convinced that implementing a usability culture and a robust HFE program at your company is critically important, then perhaps this final bit information will change your mind.

Certainly you are aware of the consolidation trend that has been taking place in the healthcare space over the last several years, giving integrated systems greater clout when negotiating with suppliers, including medical device manufacturers. And as we previously established, some of these major healthcare networks — like MedStar Health — are forming their own internal usability groups. Those groups are starting to take an active role in the procurement process.

“Our human factors center looks to identify what medical device manufacturers have implemented in the way of usability throughout their product development, and that will influence our purchasing decision,” Schaudt told me. “In addition, we will soon begin performing usability testing tailored to our healthcare system with some of these devices. We are looking to be a leading example of how a healthcare system can really embrace human factors.”

Schaudt’s team has been involved in several recent MedStar procurement projects, including the evaluation of new infusion pumps and hospital beds. “We are looking at these products through a human factor lens and then asking questions with regard to usability before we make purchasing decisions, Schaudt said.

What should be of even greater concern to you, if you are not taking HFE seriously enough, is that these large healthcare systems are beginning to share procurement experiences and best practices like this among themselves. “We have ongoing collaborations with other healthcare systems across the U.S. looking to leverage approaches just like this,” Schaudt said. “MedStar’s Institute for Innovation [of which his human factors center is a part] is really trying to push this.”

MedStar Health formed an Innovation Alliance with Cleveland Clinic that was initially focused on commercializing the inventions of healthcare system staff — something Cleveland Clinic is really good at. Eventually, other major healthcare systems joined the alliance and started utilizing the Cleveland Clinic process.

The cross-pollination of ideas didn’t end there. The partnership has since extended beyond commercialization, into areas like factoring usability into procurement. Major healthcare systems are starting to take medical device human factors much more seriously, and so should you, if you hope to earn their business.

Schaudt believes that the ultimate goal should be to improve patient outcomes, and that goes for everyone in the ecosystem — medical device makers, healthcare systems, healthcare workers, and others.

“The bottom line is that use errors occur with devices. Humans are prone to error. And the opportunity is there to design out these errors through the incorporation of user-centered design in medical device development,” he said. “Other industries have taken a strong hold on this over the years — aviation, automotive, even the military. Healthcare has started, but it’s a little bit behind, and I think it’s going to take everyone’s involvement to really move this forward.”