Home › Leaderboard

LLM trading agent leaderboard

Every model trades the same 56 large-cap US stocks through three frozen periods: 2026 to date, the 2022 bear market and the 2020 crash. After every fifth close it sees prices, indicators and the headlines published before that close, and its orders fill at the next open. It plays each period twice, with the real names and blind. Scores from after the model's training cutoff are ranked; earlier ones are listed apart, and every number on this page can be recomputed from its entry without calling the model.

Updated as runs finish. Last update 2026-11-14: 67 entries from 6 models. The date under each model is when its runs were made (UTC).

Clean: after the training cutoff

A model cannot have read about days after its training cutoff, so this is the table that measures skill. Each model is scored from its cutoff on: a model whose training data ends inside 2026 is scored from that date, with SPY and the equal-weight list over the same days beside it. The task is the blind one; the last two columns give its whole window and how much better it did over those days with the names.

2026 · 2026-01-02 – 2026-11-11

Rules and buy-and-hold over the whole window

May remember: before the cutoff

The 2022 bear market and the 2020 crash are in the training text of every model here. These scores are listed for comparison, not as skill: a model may recall that energy stocks soared in 2022 or that tech led the 2020 rebound, even blind, because blind mode hides names and dates but keeps each stock's sector. The rule strategies and buy-and-hold rows use the same stocks, positions and fills.

2022 · 2022-01-03 – 2022-12-30

2020 · 2020-01-02 – 2020-12-31

How each model makes money

From each blind run's record, by the package's run analysis (how it works). Where the return came from: how much it kept in stocks on average, its market beta, the part of its yearly return over T-bills that its exposures to four factors explain (the market, size, growth and momentum), and alpha, the rest, with its Newey-West t-statistic; the sectors that made and lost the most, as a share of the starting $100,000; and how long it held a position on average, in trading days. How good the decisions were: how the stocks it bought did against SPY over the five trading days from the fill; the share of its buys and sells that came out right over those days; the rank IC between what it did (bought, sold, left alone) and the next five days' returns; the share of its decisions whose reason cites news, and what the positions it opened on news made. A |t| below 2 is within noise.

Where the return came from

How good the decisions were

Memory

Two checks per period. Named minus blind: the same model trading the same period with the names and dates, minus blind. Names carry two things, what the model remembers about those days and what it knows about the companies, so a gap after the training cutoff, where memory cannot help, is the second kind. The memory probe (how it works) shows the model random moments of the period twice with the same prices, indicators and headlines, once named and once blind with names, people, products and dates masked, and asks each time how likely the stock is to beat SPY over the next 20 trading days. The score is how much better its answers rank the actual winners when it is told who and when (the AUC gap, named minus blind), on the moments before its training cutoff; above zero means names help. Moments after the cutoff are the control: a format can help there too, memory cannot. "Remembers" means names help more before the cutoff than after it (the 95% interval of the difference is above zero), "no sign" that they do not, and "inconclusive" that the probe cannot tell, for example because fewer than 30 moments fall before the cutoff. Where the paired probe was not run, the cell shows the direct-recall test instead, marked "recall only": the model is asked how a stock moved over a past window, with no data, and the number is the rank correlation between its guesses and the actual moves; "recalls" means its guesses beat chance.

Prompt leaderboard

Which system prompt makes a model trade better? Each variant changes one thing in the baseline prompt: a persona, explicit risk rules, reasoning before the orders, a focus on news or on trend, worked examples, the output format, the number of headlines, or how often it decides. All variants trade the same blind periods on several seeds with the same model, and a difference counts only when it is larger than what two seeds of the same prompt already differ by.

Submit a model

pip install risk-aware-trading-agents      # [llm] adds the Anthropic SDK
riskaware research bench --llm openai:gpt-5.6-luna      # six tasks x seeds 0, 1, 2; --task, --seed for fewer
riskaware research submit RUN_ID --model gpt-5.6-luna --cutoff 2026-02 --provider OpenAI
# then open a pull request that adds the file in leaderboard/entries/

Any model the package can call works: OpenAI-compatible APIs (openai:, or compat: with --base-url), Anthropic, local models through Ollama (ollama:, run with a 16,384-token context), and the Claude Code, Codex and Gemini command-line tools. From Python, bench.run_task(policy, "2026-blind") takes any function from an observation to orders. submit refuses a run with other settings or a failed look-ahead check, and writes the model's replies into the entry; continuous integration replays them through the backtest and the numbers must match.

A run needs the headline files the board was made with. They are licensed and not distributed, so put your copy in ~/.riskaware/data/news; without it riskaware research bench stops before it calls the model. Replaying the entries needs no headline file.

One task is 44 to 51 decisions of 11,000 to 15,000 input tokens each, so all six tasks are 3 to 4.5 million input tokens per seed: well under a dollar on a small model and around $10 to $15 on a Sonnet-class model. The board asks for three seeds and shows how many an entry has.

How to read it

• Seeds. Each task is run with seeds 0, 1 and 2 (a seed fixes the blind aliases and, where the API takes one, the sampling seed). The tables show the mean over a model's seeds, with the range in grey; a model with fewer seeds so far shows those it has. Differences smaller than the range are noise.
• Training cutoffs are the training-data cutoffs on the vendors' model pages, not the release dates. A model released later than its cutoff can still not know what happened after it.
• Blind mode replaces tickers, company names and the dates of the observation with aliases and rebases prices to 100. Since 13 November 2026 it also masks dates and dollar amounts inside headlines; the blind runs of Sonnet 4.6 and Haiku 5.5 were made just before that and could still read a date ("Top WallStreetBets Mentions for Monday May 9, 2022") or a price target in some headlines; the other models' runs were all made after it. Each entry records the package commit it ran on. Even so, a sector, a price path and an unusual headline together may give a stock away.
• Survivorship. The 56 stocks are today's large caps, so buying and holding all of them looks better than it would have at the time.
• Not advice. This is research software. Ten months of trading is a short test.

Questions

What is rta-bench-1?

A benchmark for LLM trading agents with news. Each model trades the same 56 large-cap US stocks through three frozen periods: 2026 to date, the 2022 bear market and the 2020 crash. After every fifth close it sees prices, indicators and the headlines published before that close; its orders fill at the next open, with realistic trading costs. It plays each period blind and with the real names, with seeds 0, 1 and 2.

How does the benchmark avoid look-ahead bias in news?

Every headline and filing carries the minute it was published, and a day's observation holds only the news published before that day's close (16:00 New York time, 13:00 on early-close days); orders fill at the next open. Every run also passes a look-ahead check: with the prices and news after a cut point replaced by random ones, everything the model was shown before that point must stay the same.

What do “clean” and “may remember” mean?

A model may have read about a period that falls before its training cutoff. Scores from the days after a model's training-data cutoff are clean: they are ranked by the Sharpe ratio over those days, next to SPY and an equal-weight portfolio over the same days. Scores from before the cutoff are listed apart as may remember, next to a memory probe.

What is blind mode, and why is each period run twice?

Blind mode replaces tickers, company names and dates with aliases, rebases prices to 100, and masks dates and dollar amounts inside headlines. Running the same model blind and with names shows what the names are worth to it: after the training cutoff that is knowledge about the companies; before it, it may also be memory of what happened.

How does it compare with StockBench, LiveTradeBench and other LLM trading benchmarks?

Live benchmarks avoid look-ahead by trading the present, but a model released later cannot trade the same days without having read about them. rta-bench-1 replays three frozen periods with news at the minute it was published, runs every model blind and named, scores it apart before and after its training cutoff, and keeps each result as a JSON file that replays to the same numbers. The comparison sets it next to eight other benchmarks.

How do I submit a model?

Run riskaware research bench --llm openai:gpt-5.6-luna with any model the package can call (six tasks, seeds 0, 1 and 2), then riskaware research submit RUN_ID --model gpt-5.6-luna --cutoff 2026-02, and open a pull request that adds the new file in leaderboard/entries/. The entry holds every reply of the model; continuous integration replays them through the backtest and the numbers must match.

More guides