277

Papers

10/10

Roadblocks Active

1

Connections

Standard reinforcement learning fine-tuning (GRPO) improves the final answers of vision-language models but generates reasoning chains that are logically inconsistent or visually ungrounded up to 24.5% of the time — a hidden failure mode invisible in accuracy numbers. FGRPO adds explicit constraints via Lagrangian optimization so that the model is penalized whenever its reasoning trace contradicts its conclusion or fails to properly reference the image, reducing inconsistency from 24.5% to 1.7% while also improving visual grounding by 13%. This matters because AI systems deployed in high-stakes visual tasks (medical, navigation, robotics) cannot be trusted if their stated reasoning is detached from the actual evidence they used.

██████████ 0.9 reasoning-reliability Preprint

ReadSaveConnections

Adversaries can manipulate AI agents that browse the web or use computers by inserting a carefully crafted image patch (constrained to imperceptible pixel changes within ℓ∞ ≤ 8/255) that hijacks the attention mechanism of the underlying vision-language model, steering the agent to select a target product instead of the user's preferred choice. The attack works not by fooling the model's text output directly but by concentrating visual attention on the adversarial region, and it transfers from white-box open-weights models to fine-tuned black-box variants. This is a concrete, practical threat to any agentic AI system that operates autonomously in commercial environments where third parties can influence what appears on screen.

██████████ 0.9 agent-tool-use Preprint

ReadSaveConnections

Training medical AI models typically requires expensive expert annotations for every reasoning step; MedVR eliminates this by using two self-supervised mechanisms — one that monitors when the model is uncertain about its own visual interpretation and triggers targeted image re-examination, and another that aggregates agreement across multiple reasoning attempts to create its own training labels. Applied to Qwen2.5 VL-7B, this achieves state-of-the-art performance across six public medical visual question-answering benchmarks without any human-annotated reasoning data. The approach is significant because it suggests that reliable medical visual reasoning can be bootstrapped from outcome supervision alone, removing the major bottleneck of annotation cost.

██████████ 0.9 hallucination-grounding Preprint

ReadSaveConnections

MolmoWeb trains 4B and 8B vision-language models to control web browsers using only screenshots — no HTML or accessibility tree access — by combining 100K+ synthetic trajectories, 30K+ human demonstrations, and GUI perception data into a single training mix. The 8B model outperforms GPT-4o-based agents on web navigation benchmarks, and test-time scaling via running four parallel attempts and selecting the best roughly doubles task completion rates (35% to 61% on Online-Mind2Web). This is notable because it demonstrates that open-weight models at modest scale can match or exceed much larger closed systems on practical web automation tasks, and the planned public release of data and code could shift the research baseline.

██████████ 0.9 agent-tool-use Preprint

ReadSaveConnections

Reward models — the systems used to train AI agents to behave correctly — are typically evaluated on single responses, but real-world agents operate over long sequences of actions. This paper introduces a benchmark specifically for trajectory-level reward modeling and finds that all three major families of reward evaluators (generative, discriminative, and LLM-as-judge) degrade sharply as task horizons grow, particularly in environments where agents use external tools. The finding matters because it exposes a fundamental gap: the alignment techniques currently used to make agents safe and reliable have not been tested at the length and complexity of real deployments.

██████████ 0.9 alignment-safety Preprint

ReadSaveConnections

By embedding vision-language models as agents inside a complex 3D commercial game (Pokémon Legends: Z-A) with only raw pixel input, PokeGym reveals that the primary failure mode is not high-level planning but physical deadlock — situations where the agent becomes stuck and cannot recover. Weaker models are oblivious to being stuck, while stronger models recognize entrapment but still cannot escape, a distinction the paper calls metacognitive divergence. This reframes where effort should go: improving visual planning logic matters less than improving low-level situational recovery, which is largely absent from current benchmarks.

██████████ 0.8 embodied-ai Preprint

ReadSaveConnections

Slow, deliberate chain-of-thought reasoning is expensive to run for every action an embodied agent takes, but applying it everywhere is wasteful. HiRO-Nav uses action entropy — a measure of how uncertain the model is — as a cheap signal to decide when deliberate reasoning is actually needed, activating it only for the roughly 30% of actions that are genuinely high-stakes (threshold at the 70th entropy percentile). Training combines supervised fine-tuning with two-stage reinforcement learning that separately optimizes reflexive and deliberate modes, achieving better navigation performance with lower computational cost than always-on reasoning. This is a practical efficiency recipe that could generalize to any embodied or agentic system built on large vision-language models.

██████████ 0.8 efficiency-scaling Preprint

ReadSaveConnections

Standard GRPO reinforcement learning normalizes reward advantages linearly, which can allow a few extreme examples to dominate gradient updates and leads to unequal learning across different task types. G²RPO replaces this with a mathematically principled mapping (via optimal transport) that forces the advantage distribution to match a standard normal curve, guaranteeing that no single task type or outlier example dominates training. Applied to Qwen3-VL-8B and evaluated across 18 benchmarks in six visual task categories, the approach improves reasoning consistency and cross-domain generalization without architectural changes — making it a drop-in upgrade for anyone using GRPO-based multimodal training.

██████████ 0.8 reasoning-reliability Preprint

ReadSaveConnections

Long-term memory in AI agents typically degrades as conversations extend because relevant past information gets lost or diluted. MemCoT addresses this without any retraining by pairing a dual zoom mechanism — one that retrieves precise supporting evidence, another that pulls broader context — with a short-term memory system that tracks what has already been searched and how queries have been decomposed. Evaluated on two established long-term memory benchmarks using GPT-4o-mini and Qwen2.5-14B as backbones, it achieves state-of-the-art results, suggesting that intelligent retrieval orchestration at inference time can substitute for architectural changes or expensive fine-tuning.

██████████ 0.8 long-context Preprint

ReadSaveConnections

When using vision-language models to automatically generate image-editing instructions (e.g., 'move the red chair to the left'), over 47% of outputs from strong baseline models contain critical errors — wrong directions, ambiguous viewpoints, missing attribute detail — that make them unusable for training downstream systems. EditCaption first fine-tunes Qwen3-VL on 100K curated examples targeting these specific failure modes, then applies direct preference optimization on 10K human-annotated pairs that explicitly contrast correct and flawed instructions, reducing critical errors to 23% and raising correctness from 42% to 66%. The result is a practical data pipeline fix for a systemic problem in synthetic data generation: AI-generated training data for AI is often quietly broken in ways that compound across training stages.

██████████ 0.8 hallucination-grounding Preprint

ReadSaveConnections