Hierarchical Management of AI for Automated Monitoring and Query Resolution

Abstract

Large Language Models (LLMs) have demonstrated their abilities in understanding text and answering questions by processing text queries. LLMs have demonstrated their capabilities across a wide range of applications. However, there are concerns regarding potential biases and ethical concerns from the LLMs, creating questions regarding the trustworthiness of such systems, particularly in critical domains and for end-users. This paper introduces a hierarchical LLM architecture, where queries are first handled by a small LLM and escalated to a large LLM if the small model declares a lack of confidence.The paper further introduces an LLM monitoring framework where large models moderate the behavior of small models by logging the unexpected behavior that includes potential biases and ethical concerns. Hierarchical monitoring facilitates the administrators who monitor the behavior. Small LLMs are selected since they consume fewer resources and ensure cost-efficient responses. The new approach mitigates the limitations of small LLMs and opens up new possibilities. This approach combines cost-efficient computation with robust monitoring and opens up new possibilities for the ethical use of AI.The experiments successfully validated the approach by showing significant improvements in safety and accuracy. The source code is available at github.com/Pro-GenAI/AI-Hierarchy.