AI Can Research, But Can It Forecast?
A new benchmark shows top AI models score well on research but poorly on forecasting—here's what that means for your business numbers.
What happened
A new benchmark called Investor's Last Exam tests whether AI agents can do equity research and investment forecasting using only the evidence available at decision time [1].
It was built and led by experienced buy-side investors and fund managers, drawing on a database of 3.43 million document versions across nine source categories [1].
The evaluation covers 86 assignments and scores research quality, forecast accuracy, and simulated investment performance together [1].
The two leading models, GPT-6 Astra and Claude Opus 5.5, scored 68.1 and 66.9 out of 100 overall [1].
But on the Forecast and Model portion specifically, even the top scorer managed only 52.5 [1].
The researchers conclude that even state-of-the-art models fail to translate overall research capability into predictive performance [1].
Why it matters to a small business
If you have started leaning on AI tools to help project revenue, demand, or growth, this study is a useful warning sign.
That split matters for an owner because it is easy to see a confident-sounding AI summary and assume the number behind it is just as solid. This research suggests the two things are not the same skill, and the forecasting part is the one still lagging.
What to do this week
A few practical checks to run before you trust an AI-assisted number in your own planning.
- Treat any AI-generated forecast, projection, or model as a first draft, not a final answer, and check the key assumptions it used.
- Split the work: let AI pull together research, data, and summaries, but keep a specific person responsible for the final number that goes into a plan or budget.
- Pick one forecast you already rely on this week and ask what evidence it is actually built on, the same way this benchmark asks of AI models.
How we think about it
Every system Digital Solomon builds ships with one job and one number, not a promise that AI gets every number right. Naming a limit is part of the job, not an admission against it. Operations is about giving a team back the hours spent retyping and chasing, which means putting AI to work where it is strong and keeping a person owning the parts of the job, like a forecast, where the number has to be right.
Sources
- arXiv.org: Investor's Last Exam: Can AI Agents Do Equity Investment Without Seeing the Future?
- litmaps.com: Litmaps | Your Literature Review Assistant
- alphaxiv.org: alphaXiv
Researched and drafted with AI from the sources below. Reviewed and approved by Marcus Barber before publishing.