The Reality of Artificial Intelligence: Wisely Using AI

By Danny Miller, CISO, Texas A&M University System

What is grabbing the headlines in both news sites and social media are two ends of the spectrum when it comes to the use and effects of artificial intelligence.  On one end, a jump head-long into the use of artificial intelligence-enabled technologies showcases the glory of new capabilities and functionality available.  On the other, a doom-and-gloom Eeyore-like attitude towards all that is artificial intelligence.  Still, others are dismissive of this technology class: I was recently told that AI was simply “a tool,” a passing fad that would burn out soon enough, and that its promise was greatly hyped.  Where does the truth lie?  As always, somewhere in the middle.

First, let’s level set on my view of artificial intelligence, machine learning, and generative artificial intelligence.  Are these terms interchangeable?  No, and indeed, it is important to realize the difference between them.  Think of a large bubble that contains other bubbles.  The largest of those, which includes all other bubbles, is called “artificial intelligence” or “AI.”  It is inclusive of all subsets of this subject area.  When defining artificial intelligence, think of it as a field of computing science that includes the capabilities of performing analysis and tasks that were or are previously done by humans.  Now, let’s drop another bubble within the AI bubble called “generative artificial intelligence” or “GenAI.”  GenAI is focused on new, novel, original, or unique analysis, conclusion formulations, and content that did not previously exist.  Activities such as large-scale data analysis against a set of disparate data sources to come up with various conclusions are one example of using GenAI to “find the needle in the haystack” scenario.  Also, if you’ve been following it lately, you’ve seen this technology do some amazing things in the video and audio areas of GenAI, namely with Sora.

I’m all for the use of AI systems, but we must be wise stewards of these systems, realizing how we use them is to keep humans in the loop of their operation.  That is, instead of allowing AI systems to run un-checked, we need to manage and govern them, especially when the use-cases of these AI systems are impactful to the public.

Finally, let’s drop one more bubble within the AI bubble, and we’ll call it “machine learning” or “ML.”  Like GenAI, it has its unique place as well.  Think of ML as more focused on specific areas of interest, like an ML learning all about a network, what devices are attached to it, and what each of those devices having functions on the network are doing.  The ML system will learn all of that over time, having a baseline most of the time (supervised learning) to start with and then learning more and more about the environment (unsupervised learning), being able to do specific things it was tasked to do so, such as discerning what devices are acting differently or poorly on a network, detecting possible attacks on the network and everything attached to it.

One last word on this technology is the use of the term “large language model” or “LLM.”  Without getting too much into the details of how an LLM underpins each AI system, think of an LLM as the accumulation of all learning that a particular AI system has in its intellect.  You can also think of it personally:  an equivalence is that if you took the entirety of your training and experience over your lifetime, that makes up your personal LLM.  To an AI system, this works the same way, which also means that an LLM is not static, but growing and maturing over time.

As we see these new and powerful capabilities emerging from newer versions of AI engines and with growing LLMs, it is important to consider not only how they may be used, but how they may impact the organizations that we are a part of.  Some of these are cultural and ethical considerations, while others surround safety and security in the use of AI.  Here are some considerations that you, as a leader in your organization, should address:

• Can you explain the decision-making process of the AI system in use to users and stakeholders?  These AI systems are both the summation of the training and experience of its underlying LLM, and how it was programmed and configured.  One of the key concerns I have is related to the LLM in an AI system, because they are vulnerable to both data poisoning (someone feeding incorrect, misleading, or deceptive information into the LLM) and bias.  There have been numerous reports of one or more parties manipulating the performance of an AI system by exposing the AI system to incorrect, incomplete, and misleading information to purposely affect the performance and results of AI operations, affecting its LLM.  That is aside from the fact that in some instances, in the absence of complete information, there are some AI systems that have hallucinated results, providing fabricated or misleading results.  The inscrutability of how an AI system does its work and where it received its training is a major risk.
• Are there mechanisms to track and audit the AI’s actions and decisions?
• Is there a process for timely applying security patches and updates to the underlying software? These are for what I call private versions of AI systems, those implemented like applications and without open or direct access to the Internet.
• Is the AI system designed with security best practices to prevent unauthorized access or data breaches?
• Have you ensured that the training data (the LLM) used for the GenAI model adheres to privacy regulations (e.g., GDPR, HIPAA)?
• Is the data storage that the AI system operates encrypted, both during transit and at rest?
• Have you implemented data anonymization and minimization techniques to reduce the risk of exposing sensitive information?  By now, we’ve all read and heard of actual incidents where information exposed to a publicly available AI system became public information, in effect, becoming a data breach.  That can affect not only sensitive personal information (SPI), but also confidential information and intellectual property.  Once it’s exposed in a public AI system, it’s like placing it out on the Internet for everyone to see and consume.
• Has the AI system been subjected to security testing, vulnerability assessments, and penetration testing?
• Have you implemented proper authentication and authorization mechanisms to control access to the GenAI system?

What should you do in the near-term to address risks to your organization?  Here are some steps to take:

• Inventory all AI systems in use in the organization.
• Establish an approval process for the introduction and implementation of AI systems in your organization.
• Define the objectives and scope of the AI system in use, including the potential threats and vulnerabilities that could compromise its functionality, integrity, or availability.
• Conduct a risk assessment to identify and prioritize the most critical risks and their impact on the AI system and its stakeholders.
• Implement appropriate security measures, such as encryption, authentication, access control, monitoring, testing, and auditing, to mitigate the identified risks.
• Establish a governance framework to ensure the accountability, transparency, and ethical use of the AI system, as well as compliance with relevant laws and regulations.
• Monitor and review the performance and behavior of the AI system and update the risk assessment and security measures as needed.
• Understand what contracts and agreements that your organization has with external parties.  What information do they have of yours and what rights do they have to the information being used and operated on that you are a custodian of?

I’m all for the use of AI systems, but we must be wise stewards of these systems, realizing how we use them is to keep humans in the loop of their operation.  That is, instead of allowing AI systems to run un-checked, we need to manage and govern them, especially when the use-cases of these AI systems are impactful to the public.