267

Papers

10/10

Roadblocks Active

0

Connections

This study analyzed 1,794 CLI coding agent trajectories across 7 frontier LLMs and 3 agent scaffolds, finding that most failures stem from knowledge gaps (epistemic errors) and begin within the first few execution steps — yet go undetected until recovery is no longer possible. The annotated dataset of 63,000+ steps with inter-rater agreement (Cohen's κ 0.78–0.94) is publicly released, making this a durable resource. This matters because it shifts the design question from 'how do we recover from failures?' to 'how do we detect knowledge gaps before agents commit to a wrong path?'

██████████ 0.9 agent-tool-use Preprint

ReadSaveConnections

Direct power measurements on edge hardware show that generating output tokens costs 11–39x more wall-clock time than processing input tokens, and that average inference power varies less than 5% regardless of image resolution, visual complexity, or prompt type. The bottleneck in edge vision-language model deployment is not the vision encoder — it is output verbosity during the memory-bound decode phase. For anyone deploying AI on constrained hardware, this means controlling output length matters far more than compressing inputs.

██████████ 0.9 efficiency-scaling Preprint

ReadSaveConnections

Analysis of over 5,000 public agent skills found that 70% contain at least one logical relation — prerequisites, mutual exclusions, or sequencing constraints — and that six tested LLM backbones (including Codex and Claude Code) violate these at unsafe rates up to 70%. Violations lead to concrete harms including privacy leaks and dangerous configuration changes. This reveals a systematic gap: the AI agent ecosystem has accumulated a large library of skills whose internal logic deployed models cannot reliably respect.

██████████ 0.9 alignment-safety Preprint

ReadSaveConnections

A 35B sparse mixture-of-experts model with only 3B parameters active per token matches the performance of ~100B-class dense models through a post-training pipeline that combines multi-teacher on-policy distillation with reinforcement learning — achieving a 17% end-to-end training speedup. The key mechanism is a routed reverse-KL distillation objective that prevents the 'seesaw' degradation that typically occurs when mixing reward signals across domains. This is a concrete demonstration that post-training optimization, not additional pre-training compute, can close large capability gaps.

██████████ 0.8 efficiency-scaling Preprint

ReadSaveConnections

Built from MIMIC-IV records of patients with at least 15 hospitalizations, this benchmark tests LLM agents on factual recall, temporal reasoning, and long-horizon clinical decisions over extended EHR histories. LLMs handle explicit timestamps reasonably well but fail at implicit time inference, and RAG or memory augmentation improves information retrieval without improving actual decision-making quality. The finding that decision performance depends on the immediate context window rather than historical aggregation suggests current memory architectures are not yet fit for longitudinal clinical AI.

██████████ 0.8 long-context Preprint

ReadSaveConnections

Using 5,620 evaluation instances drawn from real patient-doctor consultations on JD Health, this benchmark finds that frontier LLMs satisfy as many or more positive clinical criteria as human physicians but also trigger significantly more negative (safety-sensitive) criteria. Image information is critical — models degrade substantially without it. The practical implication is that LLMs have broad clinical knowledge but lack the cautious error-avoidance behavior that makes medical responses safe to deploy.

██████████ 0.8 alignment-safety Preprint

ReadSaveConnections

OpenProver connects a Planner-Worker-Verifier agent architecture to the Lean 4 proof assistant, so every candidate proof is automatically checked for correctness by a formal type-checker rather than trusted on the LLM's word. A shared 'whiteboard' scratchpad lets parallel worker agents pool intermediate findings, improving search efficiency. This approach structurally eliminates proof hallucination — wrong proofs are rejected by the verifier, not just flagged as likely wrong — making formal mathematics one of the cleaner paths to reliable LLM reasoning.

██████████ 0.8 hallucination-grounding Preprint

ReadSaveConnections

A controlled experiment held policy content fixed while randomizing the endorsing country (US, EU, China, Russia) across calls to GPT-4, Claude Sonnet, Gemini, and DeepSeek. Three Western-developed models rate China- and Russia-endorsed policies substantially lower than identical US- or EU-endorsed ones; DeepSeek does not show this pattern in numeric responses but exhibits sharp penalties for China and Russia when asked to justify its scores. This is evidence that major LLMs carry geopolitical biases that activate differently depending on whether the model is forced to articulate its reasoning.

██████████ 0.8 alignment-safety Preprint

ReadSaveConnections

A 146M-parameter behavioral auditor model trained on a hyperbolic geometry substrate achieves AUROC 0.804 in detecting sycophancy, confabulated memories, and dependence-fostering behaviors — outperforming frontier zero-shot LLM judges (AUROC 0.721) and identifying violations that trained human raters could not reliably agree on (Fleiss κ=0.074). The mechanism is linear read-out probing of frozen representations rather than generation, making it fast and interpretable. This suggests small, specialized models can be more practical safety monitors than large general ones for well-defined alignment failure modes.

██████████ 0.8 alignment-safety Preprint

ReadSaveConnections

AgentKGV applies an agentic retrieval-augmented framework with iterative query rewriting to knowledge graph fact verification, improving macro-F1 by 5.5 points over single-turn RAG on long-tail predicates, with an additional 9.4 point gain from two-stage training. GRPO reinforcement optimization halves average search calls from 3.24 to 1.63 without accuracy loss. This matters because knowledge graphs are a primary grounding resource for enterprise AI, and long-tail predicates — where single-turn retrieval fails — are exactly where factual errors cause the most damage.

██████████ 0.8 hallucination-grounding Preprint

ReadSaveConnections