Can we have an understanding with observation only?

Can we have an understanding with observation only?

Recently, the rapid advancement in technology and artificial intelligence science has accelerated the convergence of technology and the human world. As we read Klara and the Sun, the author addresses some post-humanism concerns—such as the displacement of humans, whether robots can outsmart people, and whether humans become less trusted than smart algorithms. When we start reading the first chapter, Klara, an artificial friend robot, seems to make interpretations, as she or it observes the people from inside the store; some of her observations contradict what she predicts about people—sometimes, for example, she is confused about whether people feel sad or happy. Therefore, since those earlier interpretations from the store are based on observation, we need to ask these questions:

  • How does Klara infer people’s emotions?
  • Can we understand people’s emotions from their facial expression and body language?

As we move to the big concerns demonstrated in the novel and manifested in the current world we live in, we are faced with these new questions:

  • Have machines outsmarted humans in emotional intelligence?
  • What is emotional intelligence?
  • Can machines understand people without having emotions?
  • What are emotions?

By providing such questions, we will try to delve into some of those concerns mentioned above, regarding post-humanism concerns, as well as some contemporary research about machine learning, psychology and emotions.

Can we understand people from their facial expressions alone?

“The more I watched, the more I wanted to learn, and unlike Rosa, I became puzzled, then increasingly fascinated by the more mysterious emotions passers-by would display in front of us.”

This quote expresses the confusion and the curiosity demonstrated by Klara to understand the complicated emotions of humans, and her enthusiasm to step out from the store, as she waits to be selected by a kid.

But how does Klara infer the world and emotional clues in front of her? Klara senses the world—and as she gathers data, this data is used as inputs for the machine learning models integrated in her assembly system and algorithms. So, what is machine learning? And what is a model?

How does Klara work?

Machine learning is simply algorithms that allow systems to learn from existing data and to perform several tasks including: classification, prediction and decisions without direct explicit prewritten rules. On the other hand, models are those systems that are already trained and ready to be deployed in robots such as Klara, translation tools, and recommendation systems like YouTube, Facebook, etc.

Klara is a robot and so it uses computer vision models, because her sensory data are mostly pictures—her interpretations are largely dependent on the data she was trained on.

Analogy between computer vision systems and human vision systems
An analogy between computer vision systems and human vision systems.

The picture above shows an analogy between computer vision systems and human vision systems. The analogy here is that both systems classify their observations based on pre-taught rules and past experiences, as with humans; unlike humans, whose interpretations are taught and context dependent, machines use pretrained models and data. As opposed to this, Klara does not seem to have sound interpretation, and there are gaps in her understanding, so how does this happen? Does this happen similarly to humans? Can we infer people’s emotions based only on observations?

Research results on computer vision algorithms and human emotions

In today’s flourishing AI, there is a growing demand for smart systems for recognizing people’s emotions from their facial expressions, including healthcare, marketing, and human-computer interaction. A recent study in 2026, conducted by Zineb Bougriche, Jakub Osuchowski, Maciej Dąbroś, and Michał Tomaszewski (Opole University of Technology), came to study the precision of human annotators and computer vision models—such as YOLOv8 and CAM3D—at labeling emotions correctly, utilizing the CMU, CAM3D, and BEAST datasets as references.

The key finding of the study was that human gestures were found to be more informative than facial expressions. Furthermore, when facial features were completely blurred out, the AI model was able to classify emotional states accurately using bodily and hand cues achieving 88% accuracy on full anonymized images. In addition, when the AI focused specifically on hand gestures alone, its performance jumped significantly—reaching 97.92% accuracy on the BEAST dataset.

Despite the accuracy, we need to question what these datasets are about. These datasets were gathered by researchers to annotate emotional states based on facial expressions, such as CMU and CAM3D, and body language, like BEAST—which stands for the Bodily Expressive Action Stimulus Test.

According to a leading psychologist and researcher, Lisa Feldman Barrett, a specialised scientist in human emotions, she argues that these annotated data, which classify emotions based only on facial expressions, are not accurate and are biased. She considers emotions as not universal, and facial expressions do not convey a specific emotion—she refers to this as Lack of Specificity—a scowl can mean anger, but people demonstrate it not only with the expression of a scowl:

“A scowl is an expression of anger, not the expression of anger.”

— Lisa Feldman Barrett

In addition, when she was studying how non-Western societies in Tanzania understand facial expressions, their perception was not the same. Therefore, according to her, facial expressions are less reliable as they lack the cultural context.

Statistical visualisation showing scowl expression versus perception of anger
Statistical visualisation: expression of anger via a scowl vs. perception of scowls as anger.

On the left, we have the expression of anger; this statistical visualisation shows that more than 70% of the time we often do not express the emotion of anger with a scowl; on the other side, more than 65% of people on average perceive scowls as an expression of anger.

Going back to the studies on computer vision, she stated that most Emotion AI research comes from perception studies where subjects are asked to judge stereotypical poses made by actors. These perception study data are used to train AI models to infer emotions incorrectly, without accounting for the variation in the expression of emotions.

In conclusion, it is less reliable to form an understanding of how someone feels based only on observation of their facial expressions, or as Lisa Feldman Barrett said:

“A scowl is not inherently an expression of anger. In this culture we've learned to construct a category for anger in a variety of situations that transforms physical movements like a scowl... as something that has meaning as anger.”

— Lisa Feldman Barrett

Have machines outsmarted people already? Are we no longer the smartest entities on earth?

As we discussed, while it is very hard to form understanding with observation, Lisa Feldman Barrett’s research was done in 2019; so, are machines still underdeveloped, or have they developed?

In Klara and the Sun, we see that kids trust robots more than humans. The research we are about to discuss seems to be a sign of a self-fulfilling prophecy of Kazuo Ishiguro’s expectation of a world where we are not the only smartest creatures. A team of researchers from the University of Geneva and the University of Bern in Switzerland tested six AIs, including ChatGPT. This test aimed at evaluating whether these LLMs could appropriately react to a charged emotional situation, referring to this as emotional intelligence. This research was conducted in December 2024 and published in May 2025. The results of this research were as follows:

  • The LLMs achieved significantly higher scores with 82% correct answers.
  • Whereas humans scored 56%.

While the research unveils high scores for LLMs, emotional intelligence is a vague term to claim that LLMs are better than humans at—which may suggest that they are more efficient at adhering to prewritten rules rather than assuming they have emotional intelligence. As we reflect more, we can add this question: Can LLMs be emotionally intelligent without emotions?

Emotional intelligence

What is emotional intelligence? It is simply the ability to understand, regulate and handle our emotions and others’ in a positive way. It is also the ability to empathize with other people’s emotions. Daniel Goleman, a psychologist and journalist, famously known for his book “Emotional Intelligence: Why It Can Matter More Than IQ”, separates between three terms:

  • Cognitive empathy
  • Emotional empathy
  • Empathic concern

As opposed to Lisa Feldman Barrett’s definition of emotions—which we will see later—he considers emotions as biological functions that can transmit from one neurological system to another, meaning from a person to a person—which he refers to as emotional empathy, feeling what someone else feels. For example, when I see someone suffering, I also get instantly stimulated; this condition can lead to burnout and emotional exhaustion for people who work in hospitals, where they are constantly exposed to the suffering of others. Cognitive empathy is the ability to understand the motivation and circumstances of others and how they feel. While AI does not have emotional empathy, Daniel Goleman wishes AI had empathic concern—which is the capacity to offer help to people as they expose their emotions.

Can machines understand people without having emotions?

‘Those people seem so pleased to see each other,’ Manager said. And I realized she’d been watching them as closely as I had.

‘Yes, they seem so happy,’ I said. ‘But it’s strange because they also seem upset.’

‘Oh, Klara,’ Manager said quietly. ‘You never miss a thing, do you?’ Then Manager was silent for a long time, holding her sign in her hand and staring across the street, even after the pair had gone out of sight. Finally she said: ‘Perhaps they hadn’t met for a long time. A long, long time. Perhaps when they last held each other like that, they were still young.’

‘Do you mean, Manager, that they lost each other?’

She was quiet for another moment. ‘Yes,’ she said, eventually. ‘That must be it. They lost each other. And perhaps just now, just by chance, they found each other again.’ Manager’s voice wasn’t like her usual one, and though her eyes were on the outside, I thought she was now looking at nothing in particular. I even started to wonder what passers-by would think to see Manager herself in the window with us for so long. Then she turned from the window and came past us, and as she did so she touched my shoulder. ‘Sometimes,’ she said, ‘at special moments like that, people feel a pain alongside their happiness. I’m glad you watch everything so carefully, Klara.’

Reflecting on Daniel Goleman’s conception of emotions and this passage, Klara seems to have what he calls empathic concern, even though she lacks the emotional empathy. As she starts her company with Josie, she is always concerned with Josie’s health; she is always altruistic, selfless and kind. Still, Klara is not as emotionally intelligent as humans; she lacks the shared experiences people have with their friends, family and their society. The Manager gives us the impression that she had in that passage a shared story with that couple, who demonstrated two opposed feelings—which Klara failed to understand, except her Manager.

Emotions are not built in, they are built.

“The idea is that this is actually also what emotion categories are. Facial movements and bodily changes, physical actions have physical functions for your body, but their meaning emotionally, their meaning is social reality and it changes depending on which culture you're in and who you are.”

— Lisa Feldman Barrett

Maybe one of the reasons why Klara has not succeeded in understanding humans’ emotions is that, as a machine, she was not trained to do that. Lisa Feldman Barrett does not consider emotions as something universal; she believes that emotions are a social reality, something that doesn’t exist by itself, but something we collectively agree to define and express in our society, like money, nationalism or any idea that we as humans have created.

Whether emotions are something we inherit when we are born or a social reality, we should ensure that AI is safe to use and does not produce harm. We should ensure that AI is not used unethically. As AI might be involved in cybersecurity attacks, war crimes and vilification, the need for AI that adheres to the universal values that most societies agree with should be the priority for an ethical AI that contributes to the development of humanity, not a tool for chaos and destruction.

References

  • Barrett, L. F., Adolphs, R., Marsella, S., Martinez, A. M., & Pollak, S. D. (2019). Emotional expressions reconsidered: Challenges to inferring emotion from human facial movements. Psychological Science in the Public Interest, 20(1), 1–68.
  • Tomaszewski, M. (2026). Human and AI perspectives: Comparing emotion recognition from facial expressions and hand gestures. Cognitive Computation, 18, Article 74.
  • European Commission. (2025). Can AI understand emotions better than humans? Community Research and Development Information Service (CORDIS).
  • Microsoft Research. (2019, May 29). Can machines perceive emotion? [Video]. YouTube. https://www.youtube.com/watch?v=7uhhM3UyHvc
  • BetterListen & WisdomFeed. (2024, April 12). Empathy and human connection in the age of AI with Daniel Goleman [Video]. YouTube. https://www.youtube.com/shorts/8PfAcBubllM

Comments

Popular posts from this blog