Your Chat History Knows More Than Your Doctor Does

An AI Reads Your Therapy Chats to Spot Depression Severity

What if the words you type to a mental health chatbot already contain a reliable signal of exactly how depressed you are?

Here is the setup. A team at Slingshot AI fine-tuned a very large language model — a 27-billion-parameter version of Qwen, roughly the scale of the models powering today's major chatbots — on transcripts of real conversations between users and an AI mental health platform. The model's job was to predict a PHQ-9 score: the standard nine-question depression assessment that clinicians use worldwide. You have probably seen it at a doctor's office. 'Over the last two weeks, how often have you felt little interest or pleasure in doing things?' Think of it like a thermometer for mood embedded in your words. A thermometer does not ask how feverish you feel — it reads the signal directly. This model reads patterns in how you phrase things, what you focus on, what you avoid, without you filling in a separate form. The numbers are genuinely striking. On a held-out test set of 842 users, the model achieved a correlation of 0.80 between its predictions and actual PHQ-9 scores. At the clinical threshold for moderate-to-severe depression, it correctly identified who crossed that line about 91% of the time. The catch — and it is a real one — is that all of this data came from a single proprietary platform. Crucially, the labels used to train the model were partly generated by another AI, Claude Opus, not human clinicians. There was no external validation on a different population, a different app, or a different country. Before anyone uses this near a real patient, these results need to replicate outside the one walled garden they came from. A correlation of 0.80 inside the building is not the same as 0.80 in the world.

Veterans on Bikes, Smartwatches Watching Their Nervous Systems

Ten veterans rode bicycles across a gruelling multi-day course while a smartwatch quietly watched their nervous systems for signs of overload.

Project Hero sent veterans on a demanding endurance cycling event — the kind where physical exhaustion is the point. A research team attached a twist: some of the cyclists also wore a smartwatch running a machine learning model trained to detect hyperarousal in real time. Hyperarousal is what happens when the nervous system gets stuck in high-alert mode — imagine a home alarm system that keeps triggering even when nothing is wrong. For veterans carrying PTSD or related symptoms, it is a constant, draining feature of daily life. The smartwatch tracked heart rate and movement. When the model flagged a likely episode, the app sent a notification prompting the veteran to check in with themselves. The group that combined cycling with the digital tool showed more stable symptom trajectories over the study period — measured weekly via validated scales for anxiety, depression, and PTSD — compared to the cycling-only group, whose symptoms began escalating toward the end. Now for the catch, and it is a large one. This was a pilot trial. Seven people in the main arm. Three in the cycling-only comparison. Four more in a non-randomised at-home monitoring group. You cannot draw firm conclusions from numbers this small, and the researchers say so themselves. Participants also flagged usability problems — the alerts arrived without enough guidance on what to actually do next. One more nuance worth sitting with: the model detected hyperarousal more accurately in people who were already more symptomatic. It works best for those who need it most. Whether that is a feature or a warning depends on what comes next.

Why Sophisticated AI Keeps Losing to Simple Maths in Medicine

Across 140 medical prediction tasks, deep learning and gradient-boosted trees kept losing to basic linear regression — and now we know precisely why.

Across 140 prediction tasks drawn from the UK Biobank — one of the largest health databases on the planet, covering conditions from mental health to heart disease — deep learning models and fancy gradient-boosted trees kept losing to basic linear regression. Not occasionally. Consistently. A team of researchers dug into why, and the answer is mathematical and a bit humbling. Imagine trying to read a page of text through a piece of frosted glass. Big block letters — simple, blunt signals — you can still make out. But the fine print disappears entirely. Now imagine the frosted glass is measurement noise: the imprecision baked into questionnaire scores, blood pressure readings, self-reported symptoms. Linear models read the block letters. Nonlinear models — the sophisticated kind that detect subtle, complex patterns — need the fine print, which noise has already destroyed. The team, whose core results were formally verified in the Lean proof assistant (think: a computer checking the maths line by line), proved that every level of complexity you add to a model requires exponentially better measurement. A two-way interaction between variables is blurred by noise squared. A three-way interaction, by noise cubed. At measurement quality typical in medicine — including mental health questionnaires — that fine structure is gone before the model ever sees the data. More data will not rescue a noisy measurement. Only measuring more precisely will. The takeaway is not 'stop building better models.' It is: right now, the bottleneck is your ruler, not your calculator. For mental health research, where most tools are self-report scales filled in once a week, that is a quiet but urgent message.

Here is what today's three stories are actually saying, read together. We are building an impressive set of tools for detecting mental health states passively — from chat logs, from smartwatches, from statistical models of medical records. The ambition is real and the early results are genuinely interesting. But all three papers bump into the same wall from different directions: measurement quality. The chatbot model's labels were partly generated by another AI, not a clinician — noisy ground truth. The veteran trial's smartwatch worked best for the most symptomatic participants — highest signal, lowest noise. And the noise paper proved mathematically that no amount of algorithmic cleverness escapes bad measurement. The honest position is this: we are not running short of model architectures. We are running short of precise, reliable, validated ways to measure mental states in the first place. The next real step forward in this field will probably look more like a better questionnaire or a better biological marker than a bigger neural network.

The LLM-from-chat-transcripts approach is moving fast — the next thing to watch for is an independent external validation study on a different platform, ideally with clinician-assigned labels rather than AI-generated ones. That is the result that would actually move the needle on clinical credibility. On the measurement noise front, it will be worth watching whether the UK Biobank team's framework gets picked up by psychiatric genetics researchers, where the argument about reliability caps applies directly to polygenic risk scores.

• Wikipedia: PHQ-9 depression scale explained
• Wikipedia: Post-traumatic stress disorder — symptoms and measurement
• Wikipedia: Measurement uncertainty — why measurements are never perfect