Founder & CEO of CalmWave, Ophir is a 25-year Enterprise IT veteran, merging AI with healthcare to objectively improve hospital operations.
Artificial intelligence is already changing the world in many ways. It’s helping us identify cancers that radiologists sometimes don’t see. It’s helping design new drugs that could heal diseases once thought incurable. Someday, it may even help us get around in self-driving taxis, buses, trains and airplanes.
However, this progress could slow significantly if we don’t focus our efforts on building trust.
It’s important to remember that people have always struggled to trust technology at first. One reason companies paid people to operate elevators back when they were new was so that people could trust that the elevator would bring them to the correct floor. They didn’t necessarily need an operator, but they were there to build trust.
Any professional will feel the same way when AI enters their office for the first time. Remember, they may have heard rumors of how AI works or read horror stories of AI making mistakes. The AI revolution is frightening to many people and uncomfortable for even more. Building tools isn’t enough to make AI work. We must build trust.
One successful way I’ve found to do so is by showing people the data and rationale behind an AI’s decision-making, effectively creating a transparent AI.
I’ll use my company, CalmWave, as an example. Our technology is designed to help doctors and nurses reduce the number of unnecessary alarms from all of the machines in an ICU. However, the idea (and criticality) of AI transparency applies to any context.
When my teams engage with a hospital system and begin implementing our platform in a hospital ICU, we ask for a year’s worth of data from their systems—which can include information like heart rate, blood pressure checks, IV drip settings and more. We feed all of that information into our system, and then when we begin presenting our AI technology to physicians and nurses, we show them how the system would have made decisions with past patients.
In our presentations, we show them when the AI would have recommended they change alarm settings for the vital sign limits of a patient’s bedside monitor and how that not only would have reduced the number of alarms going off but also still accurately and safely set the alarms at the right settings to alert staff of a patient’s need.
The problem with AI currently is not necessarily what it’s doing but that no one understands how it does what it does. Providing a frame of reference, as well as showing the math behind the recommendations and the ramifications of implementing those suggestions, is what builds trust. This approach is intended to give non-engineers easy-to-understand insights into the reasons for the actions taken as the AI engine makes decisions.
In the ICU, doctors, nurses, and patients do not just go along with technology—regardless of what it is but especially something as potentially powerful as AI. AI should never just suggest changing a setting on a vital signs bedside monitor without context. However, if it first identifies a patient’s trend lines and explains what output it expects from the patient given specific parameters, doctors and nurses can see the rationale and use that information to make the final decision.
Hopefully, one day, we’ll be able to trust AI to do its job the same way we trust doctors and nurses today when we go to a hospital for care. But just like the elevator operators of old, we need to ensure users understand how the systems work and give them meaningful reassurance that they do so safely.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
