AI Passes the Name Test, Fails the Real One

An AI Adviser for Liver Cancer That Outperformed Doctors

What if an AI could tell your oncologist something they missed — and the numbers suggest it might add 22 months to your life?

Liver cancer — hepatocellular carcinoma, or HCC — kills around 800,000 people a year worldwide. Treatment decisions are brutally complex: tumor size, liver function, the patient's overall health, and local clinical guidelines all have to be weighed simultaneously. Think of it like a very high-stakes cooking recipe where every ingredient is calibrated for one specific person, and the wrong substitution can be fatal. A team trained an AI model called HCC-STAR specifically on roughly 30,000 cancer cases, then validated it on 6,668 real patients from 12 hospitals across China. The model doesn't just output a treatment name — it walks through its clinical reasoning step by step, like a consultant explaining their logic out loud before writing a prescription. The headline numbers are striking. When researchers modeled what would have happened if patients had followed HCC-STAR's recommendations, median survival reached 51 months. Under the BCLC guidelines — the standard international framework — it was 29 months. Under China's own CNLC guidelines: 32 months. In head-to-head tests, the model also outperformed both resident and attending physicians in treatment accuracy, and sped up physician decision-making when used as an assistant. Here's the catch — and it is a big one. The survival comparison is hypothetical. Researchers didn't run a clinical trial where doctors actually followed the AI's advice and then tracked patients for years. They modeled what "might have" happened. That is very different from proof that patients would live longer. The statistical blinding in the physician comparisons is also not fully described. A prospective trial — where real patients receive AI-guided care over time — is still ahead. Treat the numbers as a promising signal, not a clinical verdict.

AI Can Name Your Food But Has No Idea How Much It Weighs

Your AI nutrition coach can spot a bowl of ramen instantly — then confidently tell a diabetic patient it's fine to eat.

Imagine handing a photo of your dinner plate to a brilliant food critic. They can name every dish without hesitating. But ask them how much the pasta weighs, and they start inventing numbers. Now imagine they're also your dietitian, and you have diabetes — and they just told you confidently that dish is safe for you. That, roughly, is the situation with today's AI vision models and food, according to a new benchmark called OmniFood-Bench. Researchers tested six large vision-language models — including GPT-5.1, Gemini-3-Flash, and Qwen3-VL-8B — on three escalating tasks. First: identify the food. Second: estimate its weight and macronutrients from a photo. Third: give dietary advice to someone with a specific health condition like diabetes or high cholesterol. The results have a lopsided, uncomfortable shape. At naming dishes, the models perform close to human level. But estimating mass from a 2D image? The paper calls it "catastrophic failure." The researchers name this the "Semantic-Physical Gap" — models are very good at recognizing what something is (a category), but have almost no reliable grip on its physical properties (how heavy it is, how much fat or sugar it contains). The safety finding is the most alarming: models frequently hallucinated safe dietary recommendations for diabetic patients who should have been flagged as high-risk. The catch: the paper does not give us precise failure rates. "Catastrophic" and "frequently" appear in the framing, but the exact numbers are not fully reported in what is currently available. Treat the direction as reliable, the severity as not yet precisely quantified.

Getting AI Agents to Argue With Each Other Cuts Chemistry Errors by 79%

Chemistry is where AI hallucination stops being embarrassing and starts being dangerous — so one team built a system where AI agents police each other.

Chemistry is a brutal domain for AI. A molecule's name can differ from a related one by a single letter, and getting it wrong is not just embarrassing — in drug discovery or industrial processes, it can matter enormously. Hallucination, where a model confidently states something false, is especially toxic when precision is non-negotiable. A research team built a framework they call G-Frame — think of it as a quality-control assembly line where multiple AI agents argue with each other, organised around principles borrowed from game theory. One agent proposes an answer, another challenges it, a third adjudicates based on rules. It's a bit like having a peer-review process baked directly into the training pipeline, rather than bolted on afterward. They used G-Frame to generate over 360,000 chain-of-thought training examples and nearly 200,000 question-answer pairs from chemical literature, then fine-tuned a 7-billion-parameter model they call OmniChem. The headline result: a 79.46% reduction in hallucinations compared to the base model, measured by their custom chemistry evaluation tool. On standardised chemistry benchmarks, OmniChem-7B matched GPT-4o mini — a model roughly five times its size. Here's the catch: the primary benchmarks include ones built by the same team, which is a known conflict in AI evaluation — you want tests designed by people who don't have a stake in the outcome. The hallucination reduction is also measured using an AI judge, which has its own failure modes. This is a genuinely interesting result, but independent replication on external benchmarks would make it much more convincing.

Line up today's three stories and a pattern jumps out. AI systems have an odd shape to their competence: very capable at the surface layer — naming a dish, proposing a cancer treatment, generating chemical text — but prone to a specific kind of failure one level deeper, where physical estimation, safety edge cases, or chemical precision actually matter. OmniFood-Bench makes that gap visible in food. HCC-STAR shows that with enough domain-specific training data and careful validation, you can push past the surface layer in ways that genuinely surprise experienced clinicians. OmniChem shows one method — adversarial multi-agent pipelines — for doing the same in chemistry. The shared implication: general-purpose models are approaching a ceiling in high-stakes domains, and the real gains are coming from narrow, carefully curated training. The open question is whether any of these gains survive independent clinical or scientific validation.

The most important next step for HCC-STAR is a prospective clinical trial — one where doctors actually follow the model's recommendations and track real patients over time. That has not happened yet, and until it does, the 51-month survival figure lives in the land of modelling rather than medicine. On the hallucination front, watch whether OmniChem's results replicate on benchmarks built by independent groups: if they do, the game-theory training pipeline becomes a serious template for other high-precision domains like law or pharmacology.

• Wikipedia: Hepatocellular carcinoma — background on the disease and treatment landscape
• Wikipedia: Hallucination in artificial intelligence — what it means and why it happens