Paper: Towards a Science of Scaling Agent Systems
TLDR
The authors compare one agent with four types of agent teams under the same tools, prompts and token budget.
Teams help a lot on tasks that divide into parallel parts, for example financial analysis (+80.8%). Teams hurt a lot on tasks with strict step order, for example Minecraft planning (−70.0%).
If one agent already solves about 45% of a task or more, extra agents usually add cost but not accuracy. This “capability saturation” effect is the most robust result of the paper.
If we were to use it, -> Our problem will be more like, for this problem (e.g. Finance Agent) Which architecture is the best.
Configs used:
Figure from Kim et al. 2025, arXiv:2512.08296. Open full size
Results:
Figure from Kim et al. 2025, arXiv:2512.08296. Open full size
The 2 extremes
Finance Agent tends towards the structures, best with Centralized.
PlanCraft tends more towards sequential, central Single agents planning.
Others are more or less in between
Problem
They use some of the SOTA benchmarks, not that easy to run and a lot of headache