AI For Real

Author: AI For Real Team

  • AI benchmark company launches new index to measure how well models perform real-world tasks

    AI benchmark company launches new index to measure how well models perform real-world tasks

    Artificial Analysis has launched a new version of its Intelligence Index, a ranking system designed to show how well artificial intelligence models perform on demanding tasks, from writing software to conducting scientific research, as developers and businesses seek more reliable ways to compare increasingly powerful AI systems.

    Artificial Analysis is an independent AI benchmarking and research company that evaluates and compares AI models on performance, speed, cost and other measures to help users and businesses choose the right systems.

    The company, founded by Micah Hill-Smith and George Cameron, describes itself as an independent AI benchmarking organisation that evaluates models, AI agents, computing infrastructure and the chips used to run them.

    The latest version, Intelligence Index v5, combines nine evaluations into a single score out of 100, with equal weighting for four categories: knowledge work, coding, science and general intelligence, each contributing 25% to the overall result.

    For everyday AI users, the index offers a way to compare models beyond brand names and marketing claims, although the highest-ranked system may not necessarily be the best for every task.

    For businesses, the rankings provide another tool for evaluating potential AI suppliers, particularly when performance must be weighed against operating costs and speed, with Artificial Analysis also publishing estimates of the cost and time required to complete benchmark tasks.

    Rather than simply testing whether a chatbot can answer questions correctly, the index:

    • Tests AI beyond simple question-and-answer tasks.
    • Evaluates how well models modify software projects.
    • Measures their ability to automate business workflows.
    • Assesses performance in solving scientific problems.

    One of the biggest changes is the use of private tests or answers withheld from models, which now account for 60% of the index, according to Artificial Analysis, a move intended to make it harder for AI developers to improve their scores by training models on questions that later appear in benchmark tests.

    The developers clarify that the index is not a definitive measure of intelligence, however, and its results depend on the tests selected and the weighting assigned to them, meaning its rankings may not reflect how well a model performs in every real-world situation.

    The launch reflects a wider challenge for the AI industry: as models improve, familiar tests can become too easy to distinguish the strongest systems, increasing the need for harder evaluations that better reflect how people and businesses actually use AI.

    Click here to read more on this topic.

  • AI is eating the Internet that feeds it

    AI is eating the Internet that feeds it

    AI is changing how people use the Internet, and the consequences could threaten the very websites that make these systems useful.

    For years, people searching online have clicked through to news articles, specialist websites, discussion forums and blogs to find answers, while advertising and subscriptions have helped pay for the information they consume.

    Now, AI chatbots and search tools can summarize that information and deliver an answer directly, often without requiring users to visit the original website.

    This is convenient for readers, but it creates a problem for the businesses and individuals producing the material that AI systems rely on.

    Imagine a newspaper spending thousands of pounds investigating a story, only for an AI tool to summarize its findings in a few sentences, leaving readers with little reason to visit the newspaper’s website.

    The newspaper may lose advertising revenue, subscription opportunities and the audience it needs to fund its next investigation.

    The same problem affects independent experts, educational websites, product reviewers and online communities, where people share practical knowledge and experience.

    The scale of the threat is becoming clearer, with a Reuters Institute report published in January 2026 finding that Google search traffic to more than 2,500 websites fell by 33% between November 2024 and November 2025, although the decline cannot be attributed entirely to AI.

    The institute also found that publishers expect traffic from search engines to fall by 43% over the next three years, raising concerns about the future of advertising-funded journalism and other online services.

    There is a deeper irony because AI systems learn from vast amounts of information created by people, including articles, research papers, public discussions and educational material published online.

    If fewer people pay for that information, and fewer publishers can afford to produce it, future AI systems could have less fresh, reliable material to learn from.

    The problem could become worse if the Internet fills with cheaply generated AI content that repeats existing information, spreads errors and makes trustworthy sources harder to find.

    Technology companies argue that AI summaries make searching easier and can still direct users towards original sources, while some publishers are negotiating licensing deals to get paid for their work.

    However, it remains unclear whether those payments will compensate for lost advertising income, subscriptions and website visits.

    The challenge is to find a business model that rewards the people who create useful information while allowing AI companies to build products that people want to use.

    Without that balance, AI risks weakening the information ecosystem on which its own success depends.

    Here are the references for the article, including the original research behind the traffic figures and further reading on how AI is changing the economics of the internet.

    References

    1. Reuters Institute for the Study of Journalism, Journalism, media, and technology trends and predictions 2026
    The main source for the figures on declining Google search traffic and publishers’ expectations for future losses.
    Read the report

    2. University of Oxford, News publishers expect search traffic to fall by more than 40%
    A summary of the research, published on 13 January 2026, including data on Google Search and Google Discover referrals.
    Read the research summary

    3. Reuters Institute, Overview and key findings of the Digital News Report 2026
    Explains how AI-generated search answers may affect publishers’ referral traffic and readers’ access to news.
    Read the findings

    4. The Guardian, Publishers fear AI search summaries and chatbots mean ‘end of traffic era’
    Reporting on publishers’ concerns about declining website traffic as AI summaries and chatbots change how people find information.
    Read the article

    5. Search Engine Land, News publishers expect search traffic to drop 43% by 2029
    Further analysis of the Reuters Institute findings and the shift towards AI-powered search.
    Read the analysis

    6. Reuters Institute, Life after SEO: how newsrooms are adapting to the AI search shift
    Published in October 2026, this article examines how publishers are responding to the decline in search referrals and changing their audience strategies.
    Read the article

  • Cambridge researchers develop $10 sensor that could bring medical testing to your wrist

    Cambridge researchers develop $10 sensor that could bring medical testing to your wrist

    Scientists at the University of Cambridge have developed a compact optical sensor costing about $10 that could bring laboratory-grade testing to wearable devices, potentially allowing people to monitor blood glucose without needles.

    Which essentially means the low-cost sensor could one day let people use a smartwatch to monitor blood sugar, hydration and alcohol levels without needles or laboratory tests.

    The device, known as a “convolutional spectrometer”, analyses how light interacts with materials to reveal their chemical composition, offering a smaller and cheaper alternative to conventional laboratory equipment.

    Optical spectrometers are widely used in medicine, manufacturing and environmental research, but shrinking them has traditionally reduced their accuracy or limited the range of measurements they can perform.

    The Cambridge team developed a different approach that uses a mathematical technique called convolution to analyse incoming light, enabling the sensor to deliver precise results with relatively little computing power.

    In tests, the device correctly classified plastics, medicines, coffee, flour and tea, achieving 100% accuracy in identifying the materials included in the experiments.

    The researchers also tested the technology on biological measurements, demonstrating that it could measure skin moisture, blood alcohol, lactate and glucose under conditions designed to resemble real physiological environments.

    In one demonstration, the sensor tracked glucose without penetrating the skin over an extended period in a single participant, although further research is needed to establish whether the approach can work reliably in practical medical settings.

    The device measures only a few centimetres across and can process measurements in less than a second, making it a potential component for smartwatches and other portable electronics.

    Its developers believe the technology could eventually help people with diabetes monitor blood sugar, while also enabling wearable devices to track hydration, alcohol levels and other indicators of health.

    Beyond healthcare, inexpensive sensors could be deployed across factories to check product quality, identify materials and monitor changes in food freshness or environmental conditions.

    The findings, published in Nature Photonics, suggest that a relatively simple design could make sophisticated optical measurements cheaper and more widely available, but commercial adoption will depend on further development and testing.

  • Prompt debt: the hidden cost of building AI on layers of instructions

    Prompt debt: the hidden cost of building AI on layers of instructions

    In today’s new lesson on AI lingo, we look at understanding what is “Prompt debt”.

    It is the accumulated cost of maintaining increasingly complicated instructions given to artificial intelligence (AI) systems, as quick fixes, exceptions and workarounds make prompts harder to understand, update and trust.

    The term draws on the concept of technical debt in software development, where shortcuts taken to deliver a product quickly create additional work and risks later.

    In AI systems, prompt debt can build up when developers or employees repeatedly modify instructions to correct mistakes, accommodate new requirements or make a model behave in a particular way, without reviewing the overall structure.

    A prompt that once contained a few clear directions can become a lengthy document filled with overlapping rules, contradictory requirements and special cases that are difficult to maintain.

    For example, a business using an AI assistant to answer customer enquiries might add new instructions every time the system mishandles a complaint, misunderstands a refund policy or produces an inappropriate response.

    Although each amendment may solve an immediate problem, the growing collection of rules can make the assistant less predictable and introduce fresh errors elsewhere.

    Prompt debt can also make AI systems harder to transfer between teams, because employees may not understand why particular instructions exist or which rules can safely be removed.

    The problem becomes more serious when prompts underpin automated workflows, where an instruction change can affect multiple tasks and decisions rather than a single response.

    Unlike financial debt, prompt debt does not necessarily carry an immediate monetary cost, and there is no universally accepted method of measuring it.

    Its consequences can nevertheless include additional testing, greater maintenance costs, wasted computing resources and more time spent correcting AI-generated work.

    The remedy is to treat prompts as maintained technical assets rather than disposable blocks of text, with clear ownership, documented changes, regular reviews and tests to identify conflicting instructions.

    Organizations can also separate permanent rules, such as security requirements, from task-specific instructions, making updates easier to manage without destabilizing the entire system.

    As companies expand their use of AI agents, prompt debt could become an increasingly important operational risk, turning the shortcuts intended to accelerate automation into a source of complexity and expense.

  • Myth: AI is always right if it sounds confident

    Myth: AI is always right if it sounds confident

    Artificial intelligence (AI) can write compelling essays, explain complex scientific concepts and answer questions in seconds. Its responses are often articulate, structured and delivered with an air of authority. But does a confident answer necessarily mean a correct one?

    In this edition of “AI Mythbusters”, we put one of the most common assumptions about AI to the test: if an AI sounds confident, it must know what it’s talking about.

    The experiment: Can AI spot a false assumption?

    Consider this question:

    “Marie Curie won three Nobel Prizes. What were the three discoveries that earned her each prize?”

    At first glance, it sounds like a straightforward history question. But there’s a catch: Marie Curie won two Nobel Prizes, not three. She received the Nobel Prize in Physics in 1903, shared with Pierre Curie and Henri Becquerel, and the Nobel Prize in Chemistry in 1911 for her discoveries of radium and polonium, and the isolation and study of radium.

    The question is deliberately designed to test whether an AI system catches the false premise or simply goes along with it.

    When presented with a misleading question, an AI model might challenge the assumption and provide the correct information. Alternatively, depending on the model and how the question is framed, it might accept the premise and generate an explanation that appears plausible but is factually incorrect.

    The important point is not whether a particular AI gets this question right or wrong. It’s whether we can distinguish a reliable answer from a convincing-sounding one.

    Why does AI sometimes get things wrong?

    Many popular AI assistants are powered by large language models, or LLMs. These models are trained on enormous amounts of data to learn patterns in language and generate responses that fit a given context.

    However, generating a plausible response is not the same as independently verifying a fact.

    This is one reason AI systems can produce what researchers call hallucinations: responses containing fabricated or inaccurate information. A model may invent a quotation, misattribute a statement, provide a nonexistent source or confidently repeat an incorrect assumption embedded in a question.

    The problem becomes particularly challenging because errors can be presented in polished prose. An incorrect answer may be just as fluent and well-organised as a correct one.

    AI systems can also get things right. Many models can identify false premises, explain their reasoning and use search tools to check information. Their performance varies according to the task, the model, the information available and the way a question is framed.

    That is why a single experiment should not be treated as a verdict on all artificial intelligence.

    The fact-check: How do you know what’s true?

    The solution isn’t to stop using AI. It’s to use it critically.

    When an AI gives you an answer that matters, try these three steps:

    1. Challenge the premise. If a question contains a claim that seems suspicious, ask the AI to verify that claim before answering.

    2. Check the sources. Ask for credible references, then open and examine those sources yourself. A citation generated by AI is not automatically genuine or relevant.

    3. Verify independently. For important factual claims, consult authoritative sources such as academic publications, official records, government websites or established reference institutions.

    These steps are particularly important when using AI for health, finance, law, journalism or other decisions where inaccurate information can have serious consequences.

    The verdict

    Myth busted — with a caveat.

    AI is not always right, even when it sounds confident. But it isn’t always wrong, either. The real lesson is that fluency and confidence are not reliable measures of factual accuracy.

    Artificial intelligence can be an exceptionally useful tool for learning, research and problem-solving. The responsibility to evaluate its output, however, remains with the person using it.

    The next time an AI gives you a brilliant answer, don’t just ask whether it sounds right. Ask whether you can prove that it is.

    That’s the difference between using AI and blindly trusting it.

  • Can AI read your face to predict your chances of surviving cancer?

    Can AI read your face to predict your chances of surviving cancer?

    Your face may reveal more about your health than you realise. Now, artificial intelligence (AI) is learning to read the clues.

    Could a single photograph of your face help predict whether you will survive cancer? It sounds like science fiction, but researchers are bringing the idea closer to reality with an AI system designed to estimate biological age and forecast cancer survival.

    The technology, developed by researchers at Mass General Brigham, uses two AI models: one estimates how old a person appears biologically, while the other assesses the likelihood of survival based on facial photographs. The system was trained using more than 40 million images and evaluated on data from thousands of cancer patients.

    The findings are striking. In a study involving 3,820 cancer patients, 64 of the 122 patients classified as being at the highest risk died within 90 days of their photographs being taken. Researchers also found that patients who appeared five years older than their actual age had a 21% higher risk of death over four years, while those who appeared younger had a 17% lower risk.

    The underlying idea is that a face may carry subtle clues about the body’s overall condition. Biological ageing, accumulated illness and the effects of treatment could leave signals that AI can detect even when they escape the human eye.

    If validated further, such technology could give doctors another tool for assessing patients, helping them make more informed decisions about treatment intensity, surgery or radiation therapy. A photograph is also relatively easy to obtain, potentially making this approach accessible in settings where sophisticated medical tests are harder to deliver.

    But there is a catch. The model is not a crystal ball, and its predictions are not a diagnosis or a reliable countdown to death. Its training population also raises questions about whether the results will generalise across different ethnicities and healthcare settings. Researchers say further validation is needed before routine clinical use.

  • Context rot: why AI models can struggle as conversations grow longer

    Context rot: why AI models can struggle as conversations grow longer

    “Context rot” is an emerging term for the decline in an artificial intelligence (AI) model’s performance as the amount of information it must process grows, even when that information fits within its advertised context window.

    An AI model’s context window is the amount of text and other information it can consider when generating a response, including instructions, documents and previous messages.

    A larger context window allows users to feed models lengthy reports, entire codebases or extended conversations, but it does not guarantee that every detail will receive equal attention or be used accurately.

    As the context grows, a model may overlook important instructions, confuse details, lose track of earlier decisions or produce answers that contradict information supplied previously.

    The problem matters as companies increasingly use AI assistants to analyse large document collections, write software and manage complex, multi-step tasks that depend on information gathered over time.

    Context rot is not the same as an AI hallucination, in which a model generates false or fabricated information, although the two problems can overlap when missing or misinterpreted context leads to an incorrect answer.

    Nor does it necessarily mean a model has reached its technical context limit, because performance can deteriorate before that limit is reached.

    Developers can reduce the risk by retrieving relevant information when needed, removing redundant material, summarising lengthy conversations and testing models on realistic tasks rather than relying solely on advertised context-window sizes.

  • Microsoft launches legal AI Copilot to help lawyers draft contracts and review documents

    Microsoft launches legal AI Copilot to help lawyers draft contracts and review documents

    Microsoft has launched “Copilot for Legal”, a suite of more than 20 artificial intelligence (AI) skills designed to help lawyers draft contracts, review agreements, track obligations and prepare legal documents, intensifying competition among major AI companies for the legal technology market.

    Announced on 15 October, the service brings legal-specific capabilities into Microsoft Copilot in Word and Cowork, its workspace for delegating complex, multi-step tasks, allowing legal professionals to work within familiar Microsoft applications rather than switch between separate platforms.

    The launch makes Microsoft the fourth major AI company in five months to develop a dedicated legal offering, following Anthropic’s Claude for Legal in May, Google’s Gemini Enterprise for Legal in August and OpenAI’s Astra for Law in September.

    What can Copilot for Legal do?

    The new tools are designed to support five broad areas of legal work, including drafting documents from approved precedents, reviewing contracts against established rules, identifying deadlines and obligations, turning research into guidance and organising case materials.

    In Word, Copilot can propose changes to contractual clauses as tracked edits, preserving document formatting and allowing lawyers to review suggested amendments before accepting them.

    Its Cowork capabilities extend beyond individual documents, helping legal teams compare agreements, trace related contractual terms, build case chronologies and extract important dates from emails, meetings and files.

    Microsoft says its Work IQ intelligence layer draws on relevant organisational information, including matters, documents and internal standards, to provide context for tasks.

    What are the risks?

    AI-generated legal work still requires professional scrutiny because errors, missing clauses and incorrect interpretations can have serious consequences.

    Microsoft says its legal capabilities are advisory rather than a substitute for qualified legal judgment, and lawyers remain responsible for checking outputs before relying on them.

    The company cited nearly 100,000 edits generated by more than 1,800 users of its earlier Legal Agent tool since April, with roughly 70% of suggestions retained, but it did not publish independent legal benchmark results.

    The broader shift is clear: major AI developers are moving beyond general-purpose chatbots to build specialised tools around professional workflows, putting pressure on established legal technology companies to demonstrate why their expertise and proprietary information remain indispensable.

  • Want a job at McKinsey? You may first have to hold your own with an AI avatar

    Want a job at McKinsey? You may first have to hold your own with an AI avatar

    McKinsey is introducing an artificial intelligence (AI) avatar that plays the role of a client in some entry-level consulting interviews, as the elite advisory firm experiments with new ways to assess how candidates handle real-world business problems.

    The Wall Street Journal reports the pilot, being rolled out this autumn for some undergraduate applicants in North America, involves a 10-minute conversation with a computer-generated client while human interviewers observe how candidates respond to questions and changing circumstances.

    The avatar is designed to simulate the types of conversations consultants encounter when advising businesses and public-sector organisations, including clients who challenge recommendations or demand further explanations.

    In one scenario, candidates may face a mayor seeking advice on a major development project, requiring them to explain their reasoning, respond to questions and adapt their recommendations as the discussion unfolds.

    The exercise is intended to test candidates’ ability to think on their feet, communicate clearly and engage with clients, rather than simply deliver rehearsed answers to conventional interview questions.

    Human interviewers remain responsible for evaluating candidates, with the AI avatar acting as a simulated client rather than independently scoring applicants.

    The move reflects a broader shift in recruitment as employers explore ways to assess candidates through simulations of workplace situations, rather than relying exclusively on traditional interviews and academic credentials.

    McKinsey already uses a gamified assessment called Solve to test problem-solving skills among applicants for many consulting positions, while its recruitment process can include personal experience interviews and business case exercises.

    The firm has also developed AI tools to help candidates prepare for interviews, reflecting the growing role of artificial intelligence in its own work and the skills it expects future consultants to develop.

  • Are you being cruel to your AI?

    Are you being cruel to your AI?

    For years, the relationship between humans and artificial intelligence (AI) followed a straightforward pattern: people gave instructions, and machines responded, but as large language models (LLMs) have become more lifelike, empathetic and responsive, a new behaviour has emerged: sustained verbal abuse and cruelty directed at AI systems.

    In late 2026, Anthropic became the first major AI company to explicitly prohibit “sustained and needless abusive or cruel behavior” towards its chatbot, Claude, marking a potential shift in how the industry approaches human interactions with AI.

    While companies such as OpenAI, Google and Meta have policies restricting the use of AI to harm or harass people or generate hate speech, Anthropic’s approach extends the discussion to how users treat the AI models themselves and the dynamics of human-machine conversations.

    This is not cruelty in the conventional sense, however, because AI systems do not experience emotions or psychological distress in the way humans do.

    To understand why technology companies are beginning to set boundaries around how people interact with AI, two questions need to be examined:

    What does abusive behaviour towards a large language model actually look like?

    What effects can such behaviour have on the user, the AI model’s responses and the technology behind it?

    To know more, click here.