Paper: https://arxiv.org/abs/2605.13172
Code: https://github.com/Mr-lander/DESBenchmark
tldr
They have a simulated factory setting (cells, areas, …) and they run 4 coordinations protocols for LLM agents, with many metrics and results that shows which is better when
The environment
Plant -> areas -> cells layers in a factory simulatison
Cells may fail or reject work and the planner has to correct that in different ways
The question
when do different coordination paradigms succeed or fail? A paradigm is a pattern for three things: how information flows, who has decision authority, and how conflicts get resolved. The paper tests four paradigms:
- **Centralized**: the Plant picks the Cell directly. Cells must accept.
- **Hierarchical**: the Plant picks an Area, the Area picks a Cell, and the Cell must accept. A failure goes up to the Plant.
- **Heterarchical**: Cells bid for work in a mediated contract-net round. A contract net is a protocol where a manager announces a task, contractors bid, and the manager awards it. Cells may reject.
- **Holonic**: the tree stays, but Cells may reject, and the Area first tries a sibling Cell. The Plant acts only when local recovery fails. A holon is a unit that is both a whole and a part of a larger unit.
Task details and how It’s done and how errors are introduced is explained in the paper but it’s just some details.
Code looks fine to run and reproduce some of the results
They also have different instances of the problem
Figure from Wang et al. 2026 (DESBench), arXiv:2605.13172. Open full size
Weibull aging: error rate increase with age, Load-dependent: error rate increase with load
Metrics
Figure from Wang et al. 2026 (DESBench), arXiv:2605.13172. Open full size
Alongside different frameworks and LLMS, they report a lot of numbers to see
The results is of-course, the best one is dependent on the context with some clear patterns here and there
