Intent detection in AI chatbots: a comprehensive review of techniques and the role of external knowledge

Abstract

Artificial intelligence (AI) chatbots have become essential across various industries, including customer service, healthcare, education, and entertainment, enabling seamless, and intelligent user interactions. A key component of chatbot functionality is intent detection, which determines the underlying purpose of user queries to provide relevant responses. Traditional intent detection methods, such as rule-based and statistical approaches, often struggle with adaptability, especially in complex, dynamic conversations. This review examines the evolution of intent detection techniques, from early methods to modern deep learning and knowledge-enriched models. It introduces the domain type-conversation turns-adaptivity-external knowledge (DCAD) classification, highlighting its significance in improving chatbot accuracy and contextual awareness. The paper categorizes existing intent detection models, analyzes their applications across various sectors, and discusses key challenges, including data integration, language ambiguity, and ethical concerns. By exploring emerging trends and future directions, this review underscores the critical role of external knowledge in enhancing chatbot performance and user experience.