Artificial intelligence (AI) is moving beyond chatbots that simply answer questions, with a new generation of “AI agents” being designed to take actions on behalf of people and businesses.
These agents can be asked to book a flight, analyse financial information, write and run computer code, manage customer enquiries or complete other complicated tasks with relatively little human involvement.
That creates a new problem: what happens if an AI agent makes a mistake, behaves unexpectedly or starts doing things it was never supposed to do?
Nvidia is developing technology aimed at addressing this risk by creating a system that can monitor AI agents while they work and intervene when their behaviour becomes dangerous or goes beyond their instructions.
The concern is sometimes described as an AI agent “going rogue”. This does not necessarily mean a machine suddenly becoming conscious or deciding to attack humans. In most cases, it means an AI system pursuing a task in an unintended way.
For example, an agent might be told to find the cheapest way to complete a business task. If given too much freedom, it could potentially access information it should not see, make unauthorised changes to computer systems or take actions that create financial or security risks.
The more powerful agents become, the bigger this problem could become because they can connect to databases, websites, software and other AI systems.
How is Nvidia solving this?
Nvidia’s approach is based on adding a layer of security around AI agents. The system can observe what an agent is attempting to do, assess whether the action is allowed and block or interrupt activity that violates predefined rules.
Think of it as a security guard standing between an AI agent and the computer systems it has been authorised to use.
This is different from simply checking an AI’s answer for accuracy. A chatbot giving you a wrong answer is one problem, but an AI agent making a wrong decision and then acting on it can have much more serious consequences.
The technology is therefore aimed at making AI agents more useful without giving them unlimited freedom.
For ordinary people, this matters because AI agents are likely to become increasingly common in everyday services, from banking and shopping to workplace software and customer support.
The basic idea is simple: the more AI can do for us, the more important it becomes to have systems that can stop it when it tries to do something it should not.