279

Papers

10/10

Roadblocks Active

0

Connections

Five leading vision-language models — including Gemini 2.5 Pro, GPT-4o, and o3 — were tested on medical image question-answering tasks that required locating anatomical or pathological regions; the best model achieved only 19% accuracy at localization and all models showed dangerous confusion between left and right sides of the body. Worse, a 'self-grounding' pipeline where models first locate then answer actually degraded accuracy relative to answering without localization. This matters because it shows that high benchmark scores on medical AI conceal a concrete failure mode that would be dangerous in clinical deployment.

██████████ 0.9 hallucination-grounding Preprint

ReadSaveConnections

When asked to convert circuit diagrams into hardware description code, multimodal models score well on standard metrics even when the diagram is replaced with a blank image — they are pattern-matching on module name identifiers rather than reading the circuit. Replacing those identifiers with anonymous labels causes sharp accuracy drops across eight tested models, exposing that apparent visual reasoning is largely a shortcut exploit. The paper proposes identifier anonymization during training and refusal augmentation as concrete fixes, making this one of the cleaner diagnosis-plus-remedy papers on multimodal grounding failures.

██████████ 0.9 hallucination-grounding Preprint

ReadSaveConnections

In multi-agent AI workflows, introducing corrupted or adversarially perturbed data into shared artifacts (documents, tables) does not reliably cause visible errors — agents sometimes reach correct answers through structurally deviant execution paths, and sometimes produce wrong answers with structurally normal traces. Across 614 paired runs on real benchmark tasks using three different language models, the authors identify three contamination signatures: silent corruption, detour-with-recovery, and early termination. This is important because it means correctness of the final output is a poor proxy for whether a multi-agent pipeline was compromised.

██████████ 0.9 agent-tool-use Preprint

ReadSaveConnections

Standard practice for training multimodal AI is to first do supervised fine-tuning (SFT) and then apply reinforcement learning, but this paper shows SFT shifts the model's output distribution in ways that hurt subsequent RL training — with perception errors and reasoning errors drifting in distinct, compounding patterns. PRISM inserts an adversarial distillation step between SFT and RL, using a mixture-of-experts discriminator to realign distributions before RL begins, and this consistently improves performance across three different RL algorithms at 4B and 8B model scales. The practical takeaway is that the order and nature of post-training stages matters more than previously assumed.

██████████ 0.9 reasoning-reliability Preprint

ReadSaveConnections

Training AI agents to perform real computer tasks requires realistic environments, but creating thousands of these by hand is impractical; this paper automates the process by generating 1,000+ synthetic user computers — complete with realistic file systems, documents, and spreadsheets — and then running agents on month-long simulated productivity objectives over 2,000+ turns each. Agents trained on these synthetic environments improved on both in-domain and out-of-domain productivity benchmarks, and 100 environments plus 500 simulation reports are publicly released. The significance is methodological: it demonstrates a scalable path to training data for long-horizon computer-use agents without requiring real user data.

██████████ 0.8 agent-tool-use Preprint

ReadSaveConnections

MED-VRAG retrieves information from a 350,000-page medical literature corpus by treating pages as images rather than extracted text chunks, then iteratively refines its answers using a memory bank that accumulates evidence across retrieval rounds. On four standard medical QA benchmarks, this approach achieves 78.6% average accuracy, with the image-based retrieval alone contributing +1.0 point and the iterative refinement adding another +1.5 points over simpler baselines. The result suggests that preserving document layout and visual structure during retrieval carries meaningful signal that text extraction discards.

██████████ 0.8 multimodal-understanding Preprint

ReadSaveConnections

This benchmark tests whether LLMs can translate plain-English financial instructions — like 'swap 1 ETH for USDC on Uniswap' — into correct blockchain transactions, using 300 days of real Ethereum mainnet data to build nearly 32,000 test cases across 11 transaction categories. The key finding is that models frequently produce syntactically valid outputs that nonetheless fail to achieve the intended financial state change, and multi-step transaction sequences remain largely unsolved even by top models. Evaluating correctness via actual execution on a forked blockchain environment is a methodological contribution that sets a higher bar than text-similarity metrics for this class of tasks.

██████████ 0.8 reasoning-reliability Preprint

ReadSaveConnections

This survey maps the state of AI agents that operate graphical user interfaces (web browsers, desktop apps) trained with reinforcement learning, identifying why supervised fine-tuning alone breaks down: it cannot handle long sequences of actions where early mistakes only become apparent many steps later. A key practical finding is that input/output latency with live GUI environments is becoming a bottleneck, pushing researchers toward world-model-based training where agents simulate the GUI in their own representations rather than interacting with the real one. As a survey it introduces no new experiments, but its taxonomy of reward architectures is useful for understanding where the field is heading.

██████████ 0.8 agent-tool-use Preprint

ReadSaveConnections

Using the Iterated Prisoner's Dilemma as a controlled test bed, this study shows that showing a vision-language model images depicting concepts like kindness or aggression measurably shifts its subsequent cooperative or competitive behavior — even though the images are irrelevant to the task. Color-coding of game reward matrices also influences decisions, and different model architectures vary substantially in how susceptible they are to this visual priming. This matters for AI safety because it demonstrates that deployed multimodal systems can have their decision-making altered by ambient visual context in ways that are neither intended nor transparent.

██████████ 0.8 alignment-safety Preprint

ReadSaveConnections

This survey maps how reinforcement learning is being adapted for LLM-based agents that must set their own goals, reflect on failures, and plan over long time horizons — capabilities that go well beyond the episodic reward structures RL was originally designed for. The authors organize the space around four agent components (action, planning, memory, tools) and argue that handling real-world uncertainty requires cognitive-like mechanisms such as meta-reasoning and self-reflection built into the learning loop. As a narrative survey without original experiments it carries limited evidentiary weight, but it provides a useful conceptual map for understanding where agentic RL research is currently focused.

██████████ 0.8 reasoning-reliability Preprint

ReadSaveConnections