Do LLM Trading Agents Actually Work?

The honest, evidence-based answer as of mid-2026: most LLM trading agents evaluated by the most recent live benchmarks fail to beat a simple buy-and-hold baseline, a smarter chatbot is not a better trader, and how an agent is architected seems to matter more than which model powers it. This page reads three independent 2025 benchmarks straight, with every finding quoted or closely paraphrased from the paper's own abstract.

StockBench — Most Agents Lose to Buy-and-Hold

StockBench (arXiv 2510.02209, October 2025) is a contamination-free benchmark: it uses real market data (prices, fundamentals, news) from after each evaluated model's training cutoff, so a model cannot have memorized the outcome. Agents make sequential daily buy, sell, or hold decisions across a range of proprietary and open-source LLMs, scored on cumulative return, max drawdown, and Sortino ratio.

Quoted directly from the paper's abstract: “most models struggle to outperform the simple buy-and-hold baseline, while some models demonstrate the potential to achieve higher returns and stronger risk management.” That is the headline finding this page leads with — a minority of agents show real skill, and the majority do not clear the lowest possible bar.

LiveTradeBench — 50 Live Days, 21 Models, No Correlation With Chat Benchmarks

LiveTradeBench (arXiv 2511.03628, November 2025) runs agents live rather than on a fixed backtest window, streaming prices and news across two structurally different markets — U.S. stocks and Polymarket prediction markets — and has agents output percentage portfolio allocations at each step. Its abstract confirms the scale directly: “50-day live evaluations of 21 LLMs across families.”

Three findings, quoted from the abstract:

1. “high LMArena scores do not imply superior trading outcomes” — general chat-quality rank is a poor predictor of trading skill;
2. “models display distinct portfolio styles reflecting risk appetite and reasoning dynamics” — different models behave like genuinely different traders, not interchangeable engines;
3. “some LLMs effectively leverage live signals to adapt decisions” — the capability exists in a subset of models, consistent with StockBench's minority-of-agents finding above.

When Agents Trade (Agent Market Arena) — Architecture Beats Backbone

When Agents Trade (arXiv 2510.11695, October 2025) introduces “Agent Market Arena” (AMA), a lifelong real-time benchmark that isolates architecture from model choice on purpose: it runs 4 distinct agent architectures (a single-agent baseline, plus three progressively more structured multi-agent/memory designs) across 5 LLM backbones (GPT-4o, GPT-4.1, Claude-3.5-haiku, Claude-sonnet-4, Gemini-2.0-flash), live on both crypto and stock markets.

Its headline finding, quoted from the abstract: “agent frameworks display markedly distinct behavioral patterns, spanning from aggressive risk-taking to conservative decision-making, whereas model backbones contribute less to outcome variation.” This is the strongest available evidence for a claim that shapes how AlgoDrill frames the whole Agentic AI module: how you build the agent matters more than which frontier model you plug into it. See Multi-Agent Trading Architectures for what “how you build it” actually looks like across three published designs.

AI-Trader — Another Entry in a Small, Growing Evidence Base

The open-source HKUDS AI-Trader project has a companion academic benchmark, “AI-Trader: Benchmarking Autonomous Agents in Real-Time Financial Markets” (arXiv 2512.10971, December 2025) — further evidence the field is actively building evaluation infrastructure, not just agent frameworks. This page does not assert specific numeric findings from that paper beyond confirming its existence and stated purpose, since its results tables were not accessible for direct quotation this session; see LLM Trading Research Landscape for the project's open-source details.

What This Means, Honestly

Three independent live-evaluation efforts, published within about a month of each other in late 2025, converge on a similar, unglamorous picture: most off-the-shelf LLM trading agents do not reliably beat a passive baseline; a model's general chat-quality benchmark rank is a weak-to-absent predictor of its trading skill; and how an agent is architected appears to matter more than which specific model powers it.

None of these are peer-reviewed-journal results yet — they are 2025 arXiv preprints, evaluated over windows as short as 50 trading days. AlgoDrill's own Backtesting Pitfalls module and Deflated Sharpe Ratio page apply exactly this caution to any small-sample trading result, including the site's own recorded code-walkthrough backtests — the same discipline belongs here. Read these benchmarks as early, credible signal from a fast-moving field, not as a settled verdict.