Shimonoseki City University

Shimonoseki City University
MENU
Shimonoseki City University

Shimonoseki City University
MENU
By visitor
Entrance examination information
  • LINE
  • facebook
  • Youtube

TextSize

Language

TextSize

Language

Shimonoseki City University

TOPICS

Notice

#Research

[Research Results] Our faculty members received the Best Paper Award at the International Academic Conference!

Understanding the personality of AI! Scientifically comparing 36 types of AI’s emotional understanding capabilities

Professor Shirataka of Faculty of Data Science received the Best Paper Award at the 21st International Conference on Intelligent Information Hiding and Multimedia Signal Processing (IIHMSP 2025) held in Taichung City, Taiwan from October 15 to 17, 2025. This award was highly evaluated for its groundbreaking research that scientifically elucidated the "individuality of emotional understanding" of 36 types of AI models from 7 companies that are leading the world in a large and unified manner.

This study was presented at the special session "AI-Enhanced QoL, Economics, Affective Computing and Modern Technology", and was selected as the best paper in the special session from approximately 60% of the total meeting adoption rate (86 out of 143 submissions).

❓ Why did you need this study?

Large-scale language models (LLMs) such as ChatGPT are rapidly permeating into our lives, but the question of which AI can better understand human emotions has not yet been fully understood.

Previous research has focused on evaluating specific AI models individually, making it difficult to compare multiple AIs provided by different companies fairly on the same basis. This is because the specifications of each company's API differed, and it was technically difficult to evaluate uniformly.

Furthermore, it was not clear how the evaluation of emotions would change when AI was given different "personalities (personas)" such as "university students", "poetryers", and "researchers".

In order to solve these issues, Prof. Shirataka and his colleagues developed a standardized experimental platform that can fairly compare AI from different companies on the same playing field, and conducted a comparative study of large-scale AI emotional understanding capabilities.

How did you proceed with your research?

In this study, we conducted a large-scale comparison of 36 AI models from seven of the world’s leading companies (OpenAI, Google, Anthropic, xAI, DeepSeek, Meta, Alibaba).

In the experiment, AI was asked to read Japanese literary works (such as "Money and Pistol" by Hisasaku Yumeno and Kotaro Takamura's "Dragged dadori" by Kotaro Takamura), and evaluated the four emotions of "interest", "surprise", "sorrow", and "anger".

In addition, we introduced a unique method of giving AI four different "personalities (personas)", "university students", "literary researchers", "poeters", and "robots" to analyze emotions from each point of view.

We analyzed 4,067 data points collected by this large-scale experiment using rigorous statistical methods such as dual distribution analysis (ANOVA) and the Krascal Wallis test.

Discovered by surprise!

From this large-scale experiment, the following important facts have been scientifically demonstrated:

1. AI also has an “emotional personality”!

Statistical verification reveals that each AI developer (vendor) has a significant difference in emotional evaluation trends (p < 0.05 for all emotional dimensions).

More interestingly, some models had a "fixed personality" and no matter what persona they were given, their evaluation did not change much. On the other hand, we found that other models, as "persona chameleon", flexibly change the evaluation according to the personality given.

2. The content of the text greatly affects the emotional evaluation!

Statistical evidence (all p < 0.001) demonstrated that the content of the text explains the variation of more than 50% of the emotional assessment.

This shows that many AI models can capture and properly reflect subtle emotional nuances of literary works. For example, the evaluation of AI was clearly different between interesting "money and pistol" and sad "ragged dark".

3. Reliable AI, Developing AI

In this experiment, the models of Anthropic, xAI and DeepSeek showed perfect reliability without ever failing to evaluate them. On the other hand, some models did not return numerical evaluations, and it was also found that there are still some developing parts.

This is an important indicator of which model you should choose when using AI in your business or service.

This research

The findings demonstrated in this study provide practical guidelines for model selection when developing emotion-sensitive AI applications.
In the future, it is expected that these results will be applied to lead to the development of the following services.

In addition, TUFS students will be able to experience such cutting-edge research and acquire the ability to create the next generation of AI technologies on their own. At Faculty of Data Science, we are working on research to realize a better coexistence of AI and human beings, not just learning technology.

Comments

"I am very honored to receive such a wonderful award. This study is not just a tool, but an important step in becoming a partner that understands human emotions and enables richer communication.
In particular, we believe that the ability to compare 36 types of AI models with uniform standards is of great significance in terms of overcoming issues that have been difficult in previous research. By scientifically clarifying the "emotional personality" of each AI, it is possible to select the best AI according to the application.
I would like to express my sincere gratitude to the co-authors, Prof. Nakatani, Prof. Watanabe, and the students who continue to study with a spirit of exploration every day. Data science is an academic study that considers how the power of AI can be used for the future of society. We hope that the results of this research will resonate in the hearts of everyone aiming to become a data scientist in the future, and will be a catalyst for learning at our university."

Info

Click here for the English version.



[Contact information]
Shimonoseki City University
Corporate Planning Department Public Relations and Branding Strategy Division
TEL. 083-253-8967

･