Business lessons on adopting machine learning in healthcare
By Swarna Kuruganti, Digital Operations Transformation, Moffitt Cancer Center
Machine Learning (M/L) is a computational engine that drives Artificial Intelligence (AI) systems. M/L has been successful across many industries with popular applications like Amazon Alexa, and social media feeds that augment human cognition and experiences. For healthcare, M/L applications offer the promise of dramatic transformations by processing large volumes of complex data to intelligently aid in productive operational processing and complex decisions.
Like many others in the healthcare industry, our stakeholders at Moffitt Cancer Center have eagerly welcomed the opportunity to explore M/L solutions that can transform service delivery, quality, and costs. We are learning along the way that adopting M/L solutions requires organizational commitment and tenacity, with rich lessons strewn along the path before we can achieve our desired results.
We have been selective in our approach to M/L adoption. Some decisions that have helped us point our compass broadly in our desired direction include:
- Selecting a mix of strategic (clinical) and non-strategic (operational) service areas that can benefit from innovative M/L solutions
- Partnering with SaaS (Software as a Service) solutions to jump start our learnings with M/L talent, experience and to lower our total cost of ownership, an important goal for our IT teams
- Finding health tech partners with experienced teams whose product vision matches our needs
- Selecting solutions using supervised learning methods of M/L, and engaging in an early-stage exploration of solutions using unsupervised models
- Exploring use cases:
- in business operational areas for early successes to help earn leadership trust about the technology and ROI delivered, paving the way into clinical areas
- across many M/L based services, including computer vision, natural language processing, operational data processing, virtual (assistant) agents
- Selecting internal stakeholders who are willing and able to invest time and effort
It is not always about the technology.
M/L is another technology implementation, but with many new nuances and atypical implementation complexities.
- Designing the human experience along with the technology solution – Introducing an M/L based solution changes the human experience. For a clinician, the uncertainty of depending on clinical results from a black-boxed learning algorithm can raise concerns about the risk to clinical decision making. Or, in the case of operations staff, the solution can instill a fear of automation and potential job loss.
It is critical to carefully identify and manage unsaid needs, fears, preferences and expectations of the working team and their leadership. We are continuing to refine how best to design the human experience with an M/L solution, coordinated with the technical approach. - Internal capacity of teams – While software vendors offer estimates of work, their estimates typically only cover implementing their software. Team capacity planning for some M/L solutions may also need to consider the involvement of atypical teams such as data councils, ethics committees, adjacent initiative teams and their committees.
An additional team capacity leakage is from the effort needed to “train” supervised M/L algorithms to predict outcomes. Without any near-term benefits, such overextended effort can trigger stakeholder fatigue. And yet this investment of time and effort is critical to benefit from a working M/L algorithm. Finding a senior business leader to sponsor the effort can be critical to accommodating such extended time and effort. - Readiness to experiment – Early-stage startups with innovative applications of M/L can be attractive to adopt even in the early stages. In clinical teams where burnout can be a high concern, the need and effort to validate these tools within a time-sensitive services setting often poses a critical dilemma. The dilemma of whether to use a product in its early stage of development with little to no proven record or wait till it has a more established product features set needs thoughtful navigation.
And when it is about the technology..
The market is awash with many well-funded startups and established technology firms with unique M/L healthcare applications. Despite the choices or because of them, working with SaaS vendors can be a mixed experience.
- Software selection – Health tech startups sometimes differentiate their go to market approach by developing their M/L value proposition for different slices of the same value chain. This can make the solution comparison difficult and the solution selection effort lengthy and effort intensive. The process can move a tad faster with clearly defined internal needs and internally agreed upon evaluation criteria.
- Tolerance for an ‘evolving’ product – Most healthcare stakeholders are accustomed to fulfilling their needs “out of the box” from a purchased product. However, the ‘learning’ period of an M/L algorithm requires end-users to work within an iterative process of trial and validation, before they get real value from the product. An influential business champion’s involvement can be critical to keeping end users engaged and motivated throughout such iterative efforts.
- Training data and time – Sourcing data to train and time to train some M/L models can in itself be a learning exercise.
- Supervised algorithms require “training data sets” often more than 3X the planned data. In some cases, the technology must continue to be augmented with human till a more mature AI solution can be applied.
- Assuming the training data was of high quality, often there is still added effort needed by both vendors and business teams to validate the deployment of a trained M/L model in a real business environment.
- ROI (Return on Investment) with human involvement – although both supervised and unsupervised M/L models aim to reduce or eliminate human effort, they continue to require some human involvement. Supervised models need human effort to support the model training time. Unsupervised models need inputs from multiple user groups into the data acquisition, preparation, and evaluation of results to avoid unintentional biases built into the model. As a result, the ROI of M/L efforts often needs to be re-evaluated periodically to account for unplanned time and effort with ongoing human involvement.
While healthcare M/L solutions are still evolving, we as an industry have much work to do to prepare for innovative M/L algorithms. In the meanwhile, each of these learnings and experiences are proving to be invaluable in our quest to find innovative solutions to our many complex challenges.
Swarna Kuruganti leads Digital Operations Transformation in Moffitt Cancer Center. Swarna and her team, work with Moffitt Cancer Center’s business operations teams including clinical and non-clinical team members, revenue cycle, payor, to enable transformative goals while delivering improved productivity and efficiencies with emerging technologies.
Editing Contributors: Dr. Issam El Naqa, Founding chair of Department of Machine Learning; Santosh Mohan, VP Digital Innovation; Jaime Gallo, Journey Owner, Digital Operations Transformation