282

Papers

11/11

Roadblocks Active

0

Connections

When reinforcement learning is applied to train multimodal models on document question-answering, models spontaneously suppress their own reasoning traces and converge to direct visual inspection strategies — the reasoning capability actively gets in the way. A perception-only training approach that skips intermediate reasoning tokens achieves equivalent accuracy while cutting inference token usage by more than 60%. This matters because it challenges the assumption that chain-of-thought reasoning universally improves multimodal AI, and suggests significant efficiency gains are available by matching training strategy to task type.

██████████ 0.9 reasoning-reliability Preprint

ReadSaveConnections

Instructions that sound completely harmless in text can encode genuine physical danger when executed by a robot or autonomous agent — and current text-level content filters miss this entirely. This paper shows that physical danger and content danger produce separable, detectable signals in the internal hidden states of LLMs, and a simple logistic regression probe trained on those states detects physical danger with 86–88% accuracy and roughly half the false-positive rate of LLM-based judges. The finding reframes AI safety for embodied agents: the problem is not what an instruction says, but what it causes in the physical world.

██████████ 0.9 alignment-safety Preprint

ReadSaveConnections

Instead of distilling reasoning by having a large teacher model write out explanations in natural language, this paper distills it through verified Python programs: the teacher generates candidate programs, keeps only those that execute correctly and produce the right answer, and uses those to train a much smaller student. A 7B student trained this way scores 87.00 exact match on the TAT-QA financial reasoning benchmark, substantially outperforming its 72B teacher (78.46 EM). This demonstrates that execution verification is a powerful filter for knowledge transfer, and that smaller models can surpass their teachers when the training signal is constrained to provably correct outputs.

██████████ 0.9 reasoning-reliability Preprint

ReadSaveConnections

Evaluating AI security agents only by whether they succeed masks enormous variation in cost efficiency. This paper introduces a cost-success lens that tracks both inference spend and tool call costs, revealing that scaled open-weight models can approach frontier proprietary systems on offensive CTF tasks while remaining cost-competitive — but that defensive SOC investigation tasks do not improve with more reasoning budget at all. The asymmetry suggests that offensive and defensive security AI have fundamentally different scaling properties, which has direct implications for where to invest compute.

██████████ 0.8 agent-tool-use Preprint

ReadSaveConnections

Standard benchmarks that only check final answers miss intermediate workflow failures that would be catastrophic in real engineering contexts. StructureClaw introduces artifact-centered evaluation — checking every intermediate output an agent produces, not just the end result — and shows that a full automatic workflow with typed tool interfaces and shared artifact state raises success rates from 56.8% to 88.6% on structural engineering tasks. The remaining 11% failure rate is concentrated in safe handling of invalid numerical inputs, pinpointing a concrete and dangerous gap for deployment in high-stakes domains.

██████████ 0.8 agent-tool-use Preprint

ReadSaveConnections

Existing multimodal AI benchmarks do not capture what visually impaired users actually need: real-time proactive warnings about upcoming hazards during navigation. VIABench fills this gap with 761 first-person videos recorded by actual visually impaired users, annotated across three task types and totaling 14,526 samples. Current state-of-the-art multimodal models perform poorly on the proactive reminder task in particular, exposing that real-time predictive video understanding for navigation remains a fundamentally unsolved problem rather than a marginal gap.

██████████ 0.8 multimodal-understanding Preprint

ReadSaveConnections

Rather than asking a language model to reason through logic problems in natural language, this system encodes regulatory logic directly into Z3 — a mathematical constraint solver — and physics problems into executable Python, then uses a self-correction loop to validate outputs before returning them. At only 4 billion parameters, this neuro-symbolic pipeline achieves a perfect score on the physics task in a competitive evaluation. The result reinforces that symbolic grounding of the right kind can substitute for much larger model scale on structured reasoning tasks.

██████████ 0.8 reasoning-reliability Preprint

ReadSaveConnections

Multi-agent search systems fail in a specific and predictable way: as the interaction history grows, agents lose track of what they have already searched and enter repetitive loops that waste budget. SearchOS addresses this by externalizing search progress into an explicit shared state — a persistent record of what has been tried and what remains unresolved — so agents can coordinate without re-reading the entire history. This architectural insight (shared external state rather than shared context) is likely to generalize well beyond web search to any multi-agent task with long horizons.

██████████ 0.8 agent-tool-use Preprint

ReadSaveConnections

A smaller model (GPT-4o-mini) wrapped in a structured harness with retrieval augmentation, schema validation, and human-in-the-loop gates scores 4.08 out of 5 on average across ten human raters on an academic supervision task, while a raw GPT-5 baseline with no scaffolding scores 1.23. Eight of ten raters reject the null hypothesis of no difference at the 5% significance level. The result is a concrete, quantified demonstration that engineering around a model matters far more than the underlying model's raw capability for reliability-sensitive applications.

██████████ 0.8 hallucination-grounding Preprint

ReadSaveConnections

Across nine systems submitted to the MediaEval Medico 2025 GI endoscopy challenge, improvements in answer accuracy did not reliably translate into improvements in clinical reasoning quality or faithfulness of explanation. Systems that enforced structured reasoning and required explicit grounding of answers in image evidence behaved more consistently across different question types than systems relying on parameter-efficient adaptation alone. The practical lesson: for clinical AI, evaluation should measure explanation quality and reasoning faithfulness separately from answer accuracy, because optimizing one does not guarantee the other.

██████████ 0.7 reasoning-reliability Preprint

ReadSaveConnections