Navigating Large Codebases with AI-Powered RAG Chatbots
By Sujin Prabhakar, Director – Data Management & Reporting, Nuvance Health
Handling a vast codebase like ours with over 6,000 reports, often leads to operational inefficiencies, particularly when search functionality is limited. Without effective search tools, engineers spend considerable time locating reports or developing custom scripts to fulfill simple requests, slowing response times and creating bottlenecks. To address these challenges, we developed a Retrieval-Augmented Generation (RAG) chatbot using AWS Bedrock, specifically designed to support our engineering team and streamline report development.
Challenges in Codebase Management
In large organizations, codebase inefficiencies create substantial friction. Limited searchability means that engineers frequently duplicate existing work or rely heavily on custom scripts, adding to development time and resource strain. Our goal was to introduce an AI-powered solution to streamline discovery and reduce redundant work.
The RAG chatbot has already shown promise in transforming our codebase into a fast, accurate information retrieval resource, empowering engineers to focus on innovative work. Through continuous refinement, we aim to make large codebases more manageable and further improve operational efficiency and user satisfaction.
Our RAG Chatbot Solution
Using AWS Bedrock, our RAG chatbot enables engineers to search the codebase intelligently. It can find duplicate reports, suggest alternatives, and match existing reports with incoming requests, helping the team access information quickly and efficiently. The chatbot also integrates documentation, providing users with general answers, complete with traceable citations for accountability and clarity.
Ensuring Security and Continuous Improvement
Security is essential when handling sensitive information. We applied Bedrock’s guardrails to filter responses, protect content integrity, and uphold data privacy standards. A built-in feedback system also allows users to suggest improvements, making the chatbot a continuously evolving tool that adapts to user needs.
Measuring Impact with ITSM Metrics
We track IT service management (ITSM) metrics like service request turnaround time and ticket throughput to assess the tool’s value. Sentiment analysis of user feedback offers additional insight, helping us gauge user satisfaction and identify areas for enhancement.
Rolling Out the Solution
Our phased rollout started with a subset of the engineering team as early adopters. Their feedback allowed us to refine the chatbot’s functionality before expanding access. Next, we plan to integrate the RAG chatbot into our ticket management system to proactively address requests, aiming to reduce ticket volume and enable quicker self-service solutions.
Looking Ahead
The RAG chatbot has already shown promise in transforming our codebase into a fast, accurate information retrieval resource, empowering engineers to focus on innovative work. Through continuous refinement, we aim to make large codebases more manageable and further improve operational efficiency and user satisfaction.