Artificial Analysis has launched a new version of its Intelligence Index, a ranking system designed to show how well artificial intelligence models perform on demanding tasks, from writing software to conducting scientific research, as developers and businesses seek more reliable ways to compare increasingly powerful AI systems.
Artificial Analysis is an independent AI benchmarking and research company that evaluates and compares AI models on performance, speed, cost and other measures to help users and businesses choose the right systems.
The company, founded by Micah Hill-Smith and George Cameron, describes itself as an independent AI benchmarking organisation that evaluates models, AI agents, computing infrastructure and the chips used to run them.
The latest version, Intelligence Index v5, combines nine evaluations into a single score out of 100, with equal weighting for four categories: knowledge work, coding, science and general intelligence, each contributing 25% to the overall result.
For everyday AI users, the index offers a way to compare models beyond brand names and marketing claims, although the highest-ranked system may not necessarily be the best for every task.
For businesses, the rankings provide another tool for evaluating potential AI suppliers, particularly when performance must be weighed against operating costs and speed, with Artificial Analysis also publishing estimates of the cost and time required to complete benchmark tasks.
Rather than simply testing whether a chatbot can answer questions correctly, the index:
- Tests AI beyond simple question-and-answer tasks.
- Evaluates how well models modify software projects.
- Measures their ability to automate business workflows.
- Assesses performance in solving scientific problems.
One of the biggest changes is the use of private tests or answers withheld from models, which now account for 60% of the index, according to Artificial Analysis, a move intended to make it harder for AI developers to improve their scores by training models on questions that later appear in benchmark tests.
The developers clarify that the index is not a definitive measure of intelligence, however, and its results depend on the tests selected and the weighting assigned to them, meaning its rankings may not reflect how well a model performs in every real-world situation.
The launch reflects a wider challenge for the AI industry: as models improve, familiar tests can become too easy to distinguish the strongest systems, increasing the need for harder evaluations that better reflect how people and businesses actually use AI.
Click here to read more on this topic.