Generative AI: Keys to successfully implement GenAI
By Damian Fernandez-Lamela, VP Data Science & Analytics -Global, Fossil Group, Inc.
Generative Artificial Intelligence –or GenAI for short- has the potential to improve productivity and reduce costs in most companies. But as with any disruptive technology, there are keys to implementing it in a business environment that will allow us to extract as much value as possible and reduce the risk of spending a lot of money and not getting much out of it. The following framework is based on my experience implementing AI and other technologies over the last three decades.
- Identify use cases first -and prioritize them using their ROI vs. cost
I recommend starting by identifying the use cases as the first step, or in other words, focusing on the application of the technology to create a business opportunity or to solve a problem. For example, we may want to use GenAI to create long form product descriptions on our website using our brand voice that resonates better with our consumers. In my career, the worst technology implementation failures that I’ve seen happened when they put the technology first instead of the use case first. After identifying all the use cases, using that business lens, we should be able to calculate what will be the upside of implementing it in terms of increased revenue or reduced costs and create business cases for each use case. In some cases, we may be able to run a quick AB test or proof of concept to validate our hypotheses. Following our example, we could run an AB test to see if we can increase online conversion by using those new product descriptions. Once we calculate the ROI of each business case, we should be able to identify the costs in terms of the level of effort, either internal, external, or both, to implement them. With that information in place, we can plot the business cases in a two matrix with the level of effort and impact on each axis. Obviously, we want to tackle first the low hanging fruit of the cases with a low level of effort/cost and high impact. We may also want to start planning and working towards a high level of effort and high impact cases.
In most cases, the outcome of GenAI will be utilized by a person. Having human oversight as part of the process will ensure that we mitigate multiple risks, including hallucination problems or mistakes from the system, before those mistakes make it to the headlines of a newspaper.
- Mitigate risks – in particular privacy, hallucinations and legal risks
There are certain risks that are specific to GenAI that we need to be aware of and we need to mitigate. Just to list three major risks, we need to make sure that our confidential information doesn’t get exposed, we need to be aware of the tendency of GenAI systems to hallucinate or invent details and portray them as facts and we need to be careful of potential copyright infringement issues given some active disputes from copyright owners of the data that was used to train the models. I wrote another article about the risks of GenAI with a larger list of risks and their associated mitigation recommendations that you can find here.
- Make technology decisions based on business goals
There are some specific technology decisions regarding GenAI that we need to make based on a balance of costs vs rewards. The first one is: are we going to build our own large language model -LLM- or are we going to use an existing one? With few exceptions of large high tech companies or well-funded GenAI vendors, most companies are probably better off using an existing model. For now, the cost of creating an LLM is so high that seldom you can justify creating your own. This may change if the costs come down or if the performance of smaller models with high quality data probes is on par or better. The next decision is: should we use models as is or add additional training layers? In most cases, we may want to add additional training to the models, for example, to bring our brand voice in the case of text models. In instances where we need to integrate the GenAI system into our existing systems, for example, with an e-commerce chatbot, we need to decide if we should use an external service integrator or if we should do this internally. Unless this is going to be a core competency of the company and especially at the beginning of the journey, you are probably better off enlisting the help of a service integrator with prior experience doing this. We also need to decide if we are going to have the system being cloud based vs. on premises. In this case, it will probably depend on how sensitive the data is. In most cases, you are probably all alright with a cloud-based solution, but if you are handling very sensitive health care data, you may want to consider on premises or potentially putting the sensitive data in the devices of the consumers. Another decision is if we are going to use an open-source or a proprietary model. Here, we need to look at the costs of using the models, with open source being cheaper in the long term but potentially requiring more work upfront. The last decision that we need to make is if we are going to use a private –local- version of the model vs a public version. If you want to maintain the confidentiality of the interactions of the models, then a private-local model is the way to go.
- Have a robust change management plan
In most cases, the outcome of GenAI will be utilized by a person. Having human oversight as part of the process will ensure that we mitigate multiple risks, including hallucination problems or mistakes from the system, before those mistakes make it to the headlines of a newspaper. Unfortunately, in some business cases like e-commerce chatbots, the oversight will have to be built into the technology itself and tested thoroughly before release. In any case, we will need to carefully manage the change management aspect of implementing GenAI. For example, we may want to create employee guidelines to explain how to use the systems and what is expected from each role going forward to avoid misunderstandings. We may also want to provide tailored training to certain key team members, and keep restricted access for certain sensitive use cases. Another best practice is to identify who is the owner and leader of each use case that could manage the proof of concept and implementation, and to create a cross functional AI team that could provide companywide guidance, avoid duplication and enable knowledge sharing.
In summary, in order for companies to extract value and reduce the chance of failure when implementing GenAI, we need to identify the business cases first -and prioritize them, mitigate the risks, make technology decisions based on business goals and have a proper plan for change management.