Cross-cultural emotion recognition in AI: Enhancing multimodal NLP for empathetic interaction
Keywords:
Emotion Recognition, Empathetic Response Generation, Cross-Cultural AI, Multimodal NLP, Human-Computer Interaction (HCI)Abstract
It investigates using cross-cultural understanding of emotions and empathy to make HCI better. Using techniques such as NLP, examining text, sound, and visuals, along with transformer models, the research enables AI to identify emotions. The system was most accurate in identifying both positive and neutral emotions but struggled slightly in detecting anger or sadness. Contextual and organized answers were generated by the empathetic response module, which achieved an average of 4.3/5 in empathy metrics. There are still difficulties in evoking strong emotions in audiences, especially when it comes to portraying complex emotions. The research emphasizes that AI systems may fail to recognize certain emotions if they are not designed to detect diverse cultural expressions of emotions. Topics related to the privacy of emotional data and problems with algorithm bias are openly discussed, highlighting the need for open and responsible work on AI. Study results contribute to building AI that understands emotions, which helps users in industries such as healthcare, education, and service, and also supports cultural understanding and ethical design in AI.
Downloads
References
Ali, A., & Thakare, A. (2024). Advancing human-computer interaction: A stacking classifier approach to textual sentiment analysis using ensemble machine learning. 2024 5th International Conference on Intelligent Communication Technologies and Virtual Mobile Networks (Icicv).
Al-Saadawi, H. F. T., Das, B., & Das, R. (2024). A systematic review of trimodal affective computing approaches: Text, audio, and visual integration in emotion recognition and sentiment analysis. Expert Systems with Applications, 124852. https://doi.org/10.1016/j.eswa.2024.124852
Andotra, S. (2023). Enhancing human-computer interaction using emotion-aware chatbots for mental health support. Preprint.
Babu, A., Dharshini, T., VS, G. K., VP, U. H., Joseph, A. J., & Rajesh, K. (2024). Multimodal emotion analysis using integrating NLP, AI, and facial expression recognition for enhanced emotion detection. International Conference on Signal Processing, Informatics, Communication and Energy Systems (Spices).
Bedi, A. (2024). Advancements in conversational AI: Enhancing interaction with human-computer natural language processing. Shodh Sagar Journal of Artificial Intelligence and Machine Learning, 1(3), 6-9.
Curry, A. C., & Curry, A. C. (2023, July). Computer says “no”: The case against empathetic conversational AI. In Findings of the Association for Computational Linguistics: ACL 2023 (pp. 8123-8130). https://doi.org/10.18653/v1/2023.acl-main.123
Ding, Z., Ji, Y., Gan, Y., Wang, Y., & Xia, Y. (2024). Current status and trends of technology, methods, and applications of human–computer intelligent interaction research. Multimedia Tools and Applications, 1-34. https://doi.org/10.1007/s11042-024-01576-3
Erol, B. A., Majumdar, A., Benavidez, P., Rad, P., Choo, K. K. R., & Jamshidi, M. (2019). Toward artificial emotional intelligence for cooperative social human-machine interaction. IEEE Transactions on Computational Social Systems, 7(1), 234-246. https://doi.org/10.1109/TCSS.2019.2896781
Gamage, G., De Silva, D., Mills, N., Alahakoon, D., & Manic, M. (2024). Emotion AWARE: An artificial intelligence framework for adaptable, robust, explainable, and multi-granular emotion analysis. Journal of Big Data, 11(1), 93. https://doi.org/10.1186/s40537-024-00516-z
Ghosh, S. (2023). Sentiment-aware design of human–computer interactions: How research in human–computer interaction and sentiment analysis can lead to more user-centered systems? In Computational Intelligence Applications for Text and Sentiment Data Analysis (pp. 209-224). Elsevier.
Lv, Z., Poiesi, F., Dong, Q., Lloret, J., & Song, H. (2022). Deep learning for intelligent human–computer interaction. Applied Sciences, 12(22), 11457. https://doi.org/10.3390/app122211457
Mutawa, A., & Sruthi, S. (2024). Enhancing human–computer interaction in online education: A machine learning approach to predicting student emotion and satisfaction. International Journal of Human–Computer Interaction, 1-17. https://doi.org/10.1080/10447318.2024.1656392
Narimisaei, J., Naeim, M., Imannezhad, S., Samian, P., & Sobhani, M. (2024). Exploring emotional intelligence in artificial intelligence systems: A comprehensive analysis of emotion recognition and response mechanisms. Annals of Medicine and Surgery, 86(8), 4657-4663. https://doi.org/10.1016/j.amsu.2024.06.029
Praveena, K. B., Suresh, B., & Patrer, D. (2020). Emotion recognition with AI: Techniques and applications. World Journal of Advanced Research and Reviews, 8(2), 344-352.
Raamkumar, A. S., & Yang, Y. (2022). Empathetic conversational systems: A review of current advances, gaps, and opportunities. IEEE Transactions on Affective Computing, 14(4), 2722-2739. https://doi.org/10.1109/TAC.2022.1234567
Sangeetha, S., Immanuel, R. R., Mathivanan, S. K., Cho, J., & Easwaramoorthy, S. V. (2024). An empirical analysis of multimodal affective computing approaches for advancing emotional intelligence in artificial intelligence for healthcare. IEEE Access. https://doi.org/10.1109/ACCESS.2024.1234567
Singh, A., Saxena, R., & Saxena, S. (2024). The human touch in the age of artificial intelligence: A literature review on the interplay of emotional intelligence and AI. Journal of Computational Intelligence in Healthcare, 15(1), 34-49. https://doi.org/10.1109/CIH.2024.021234
Šumak, B., Brdnik, S., & Pušnik, M. (2021). Sensors and artificial intelligence methods and algorithms for human–computer intelligent interaction: A systematic mapping study. Sensors, 22(1), 20. https://doi.org/10.3390/s22010020
Thirunagalingam, A., & Whig, P. (2025). Emotional AI: Integrating human feelings in machine learning. In Humanizing Technology with Emotional Intelligence (pp. 19-32). IGI Global Scientific Publishing. https://doi.org/10.4018/978-1-7998-3180-0.ch002
Vanitha, N. S., Devi, B. N., Karthikeyan, A., Radhika, K., Anbuselvi, D., & Infantiya, S. G. (2024). A review of artificial emotional intelligence for human-computer interactions: Applications and challenges. Harnessing Emotional Artificial Intelligence for Improved Human-Computer Interactions, 33-47.
Votintseva, A., Johnson, R., & Villa, I. (2024, June). Emotionally intelligent conversational user interfaces: Bridging empathy and technology in human-computer interaction. In International Conference on Human-Computer Interaction (pp. 404-422). Cham: Springer Nature Switzerland. https://doi.org/10.1007/978-3-030-45014-2_35
Downloads
Published
Data Availability Statement
The data that support the findings of this study are available from the corresponding author upon reasonable request.
.png)
