Imagine telling an AI to do something and watching it make the wrong move.
But before you can react with more words, the machine already knows something is off. It detects, from your brain activity, that its interpretation does not match what you intended and adjusts what it is doing! That may sound like AI can read your mind before you have expressed the thought.
Korean researchers have developed an AI system that can use brain signals to detect when a machine has misunderstood a person’s intention.
Well, it cannot read your thoughts in the science-fiction sense, but researchers at South Korea’s The Korea Advanced Institute of Science and Technology (KAIST) have taken an important step towards AI detecting when its understanding of your intention is wrong.
The technology, developed by a KAIST team led by Professor Sang Wan Lee in collaboration with Microsoft Research Asia, is called Neural Value Alignment, or NVA. The basic idea is surprisingly simple: instead of making AI rely only on what people say or write, the researchers want it to also detect signals from the brain that reveal when something has gone wrong.
Consider a familiar situation. You ask an AI assistant or robot to perform a task. It follows your instruction, but does something you did not actually intend. Normally, you have to correct it by saying something such as, “No, that’s not what I meant.” KAIST’s system is designed to detect that mismatch from brain activity, potentially allowing the machine to correct itself without waiting for the person to explain.
This is possible because the brain reacts when reality does not match expectations. The researchers used electroencephalography, or EEG, to record electrical activity from people’s brains while they interacted with an AI system.
They focused on two types of signals. One indicates that the result was different from what a person expected. The other indicates that the situation itself has not developed as the person expected. The technical names are reward prediction error and state prediction error, but the important point is that both can provide clues about what a person thinks should be happening.
The researchers found that these signals could be distinguished in brain activity. They then used them together to help an AI work out two separate things: whether it had chosen the wrong action, and whether it had misunderstood the person’s goal in the first place. That distinction matters because the same action can sometimes serve different purposes, while different actions can achieve the same goal.
The research is still a long way from an AI that casually reads people’s minds. The experiment relies on EEG equipment to measure brain signals, and the researchers are demonstrating a scientific method for detecting mismatches between human intentions and AI behaviour. It is not a consumer product that can currently be plugged into ChatGPT, a phone or a home robot.
The next stage is to develop this idea for more natural cooperation between people and machines. KAIST says the approach could eventually be applied to physical robots in homes and factories, autonomous vehicles, medical robots and rehabilitation systems. The team’s ongoing work with Microsoft Research Asia is aimed at developing brain-computer technologies that allow humans and AI to communicate and collaborate more naturally.
There is no announcement that this particular technology is being released as a commercial product. For now, it remains a research advance. But its significance is clear: the future of AI may not depend only on machines understanding what we say. It may also depend on machines becoming better at recognising when what they have understood is not what we actually meant.
Reference