257

Papers

11/11

Roadblocks Active

0

Connections

Five leading AI vision-language models (including Gemini 2.5 Pro and GPT-5) were systematically tested on medical image question-answering tasks and found to perform poorly at localizing anatomical structures, with the best model correctly identifying regions only 19% of the time under strict overlap criteria. More dangerously, all models made laterality errors — confusing left and right — which is clinically critical. This matters because it demonstrates that strong general performance on benchmarks does not translate to safe, reliable behavior in medical imaging, and that asking a model to first locate a region before answering actually makes its answers worse.

██████████ 0.9 hallucination-grounding Preprint

ReadSaveConnections

This study injected subtle data corruptions (in PDFs, spreadsheets, images, and audio) into 614 paired runs of multi-agent workflows and found that agents can produce correct final answers even when their internal reasoning traces diverge significantly, and vice versa — fail silently when traces look normal. Existing verification guardrails consistently missed these contamination events. This is significant because it shows that checking whether an agent 'did the right steps' is unreliable as a safety signal, which has direct implications for deploying AI agents in business and scientific workflows.

██████████ 0.9 reasoning-reliability Preprint

ReadSaveConnections

Echo-α is an AI system for ultrasound analysis that combines organ-specific lesion detectors with a general-purpose vision-language model, coordinating both through a two-stage training process involving supervised instruction-following followed by reinforcement learning. Tested across two independent hospitals on renal and breast ultrasound, it outperforms both purely specialized detectors and general multimodal models. The approach demonstrates that orchestrating narrow expert tools with broader reasoning — rather than replacing one with the other — improves clinical AI performance.

██████████ 0.9 multimodal-understanding Preprint

ReadSaveConnections

When asked to convert circuit diagrams into hardware description code, multimodal AI models were found to largely ignore the actual image and instead exploit meaningful variable names in the text header to retrieve memorized code templates — a phenomenon the authors call the 'Mirage.' Replacing all identifier names with anonymous placeholders caused sharp performance drops across all eight tested models, exposing that apparent visual reasoning was text-based pattern matching. A remediation model (VeriGround) trained with anonymized data and refusal augmentation partially closes the gap, and code and benchmarks are publicly released.

██████████ 0.9 hallucination-grounding Preprint

ReadSaveConnections

This paper introduces a 105-task benchmark derived from real business workflow demand signals, covering HR, multi-system operations, and management processes, evaluated on 13 frontier language models in sandboxed environments. The best-performing model achieves only 66.7% task completion, and no model crosses the 70% threshold, indicating that reliable end-to-end automation of real-world multi-step business tasks remains unsolved. The benchmark is designed to stay current by periodically refreshing tasks from live demand signals, which addresses the staleness problem of static evaluations.

██████████ 0.8 agent-tool-use Preprint

ReadSaveConnections

Standard supervised fine-tuning (SFT) used to prepare multimodal models for reinforcement learning introduces a distribution mismatch that degrades both original capabilities and downstream learning — a problem PRISM addresses by substituting SFT with on-policy distillation, where the model learns by comparing its own outputs against a stronger teacher. The method improves results across three reinforcement learning algorithms (GRPO, DAPO, GSPO) without requiring access to the teacher model's internals. This matters because SFT-then-RL is currently the dominant training recipe for capable AI models, and this work suggests a systematic flaw in that pipeline.

██████████ 0.8 reasoning-reliability Preprint

ReadSaveConnections

This paper tests how well current large language models can translate plain-language financial instructions ('swap 1 ETH for USDC') into valid blockchain transactions, using 300 days of real Ethereum mainnet data to construct realistic test cases and evaluating correctness by actually executing the generated transactions on a simulated fork. Even the best models produce syntactically valid but functionally incorrect transactions frequently, meaning the intended on-chain state change does not occur. Retrieval-augmentation and larger models help with logical consistency but do not reliably fix execution correctness, exposing a gap between language fluency and operational reliability.

██████████ 0.8 reasoning-reliability Preprint

ReadSaveConnections

This theoretical paper argues that all current AI agent memory systems — including vector databases, retrieval-augmented generation, and scratchpads — perform lookup by similarity rather than genuine memory, and that this imposes a provable ceiling on how well agents can generalize to novel combinations of tasks they have not seen before. The distinction matters because similarity-based lookup can recall past examples but cannot abstract rules; the paper draws on neuroscience's Complementary Learning Systems theory to frame when each approach is appropriate. The argument is formal rather than empirical, so the 'provable ceiling' claim requires independent verification of the presented theorems.

██████████ 0.8 agent-tool-use Preprint

ReadSaveConnections

CARE proposes a structured process for building AI agents in scientific domains, organizing the workflow into defined stages with explicit human approval gates, and introducing 'helper agents' that translate informal domain knowledge from experts into formal, reviewable specifications for developers. The methodology is positioned as an alternative to ad-hoc agent development, where misalignment between what domain experts intend and what developers implement is a common source of failures. As a methodology paper without large-scale empirical evaluation, its claims about effectiveness remain to be validated in diverse deployment contexts.

██████████ 0.8 agent-tool-use Preprint

ReadSaveConnections

This survey maps the full peer review pipeline — from initial review generation through rebuttal, meta-review, and manuscript revision — and organizes existing AI approaches into fine-tuning, agent-based, and reinforcement learning paradigms. It finds that current methods can assist with structured stages of the process but that evaluation frameworks for AI-generated reviews remain inconsistent and often lack grounding in what human experts actually need. The paper is relevant because AI-assisted peer review is increasingly deployed in practice, yet there is no consensus on what 'good enough' looks like or how to measure it.

██████████ 0.7 hallucination-grounding Preprint

ReadSaveConnections