Downloads: 6 | Views: 37 | Weekly Hits: ⮙4 | Monthly Hits: ⮙6

Research Paper | Engineering Applications of Artificial Intelligence | India | Volume 13 Issue 4, April 2024

Sahara: Virtual Companion - Exploring Multimodal Empathetic Conversational AI leveraging Ensemble Learning with Humor

Smit D. Gaikwad | Aishwarya B. Iyer | Krishna S. Talluri | Pradip Salve

Abstract: Exchanging conversations for increasing mindfulness and combatting loneliness helps reduce and prevent mental health problems. Anxiety, depression, and stress among other mental health disorders are growing in today?s world that is driven by speed and changes, severely affecting the lives and well-being of people across all ages and diversities. The present research aims to design a virtual companion which resolves problems with an inclusive approach, is aimed at looking at a problem through different perspectives to get a holistic understanding, using the best natural language processing models. This is completed by utilizing modern text generation models like DialoGPT and T5 Transformers. The model works by using Prompt engineering, Retrieval Augmented Generation (RAG), and fine-tuning techniques on an augmented dataset, text-to-speech engines such as Speech T5 along with Meta?s Massively Multilingual Speech (MMS), that can produce naturalistic computer speech in multiple languages with speed that carries the nuances of human communication. The goal is to offer an empathetic, friendly solution that meets the mental health needs of people with varied diversities. Using the unique blend of AI and concern, this research tries to develop an innovative tool for empathic care and support people to be resilient in the face of life?s trials, by generating real-world based solutions and responses, including humor as per the user?s receptivity to build rapport and indulge the user in engaging conversations and journalling.

Keywords: Therapeutic Conversational Agent, DialoGPT, Meta?s Massively Multilingual Speech, Whisper large v2, Speech T5

Edition: Volume 13 Issue 4, April 2024,

Pages: 1501 - 1507