A public library of job-shop instances that records the best known bounds for each instance and compares four solvers in a fixed 10-minute run.
Each figure has two buttons. Copy DataviewJS copies an interactive block for Obsidian (Dataview plugin, JavaScript queries on). Copy HTML copies a static snapshot of the current state.
For most hard scheduling instances, nobody knows the optimal makespan. Instead, the library keeps the best known lower bound and upper bound for each instance, from any source. Each solver result is scored against these two bounds. Thus, one archive serves as the reference for all comparisons.
The site defines the library as three parts:
The archive accepts bounds from any source: papers, meta-heuristics, long runs and the reference engines. If a reference engine reproduces a published bound, the archive credits the engine.
The library has two problem collections now. RCPSP and workforce scheduling are planned, but the site gives no date.
Neither page states the objective in a formula. The bounds are makespan values and the bound methods are makespan bounds, so the objective is makespan minimization (inferred). Makespan is the end time of the last operation.
Each instance has a JSON file. The JSON lists operations (job, step, option, machine, duration) and precedences (before, after). The same schema covers job shop, reentrant job shop, flexible job shop and arbitrary precedence graphs. The legacy text formats are also kept:
#jobs #machines. Then each line is one job, as machine and duration pairs in operation order.-1 -1.#jobs #machines average_flexibility. Each operation gives its number of options, then machine and duration pairs. Machines start at 1.#operations #precedences #machines, then precedence lines, then one options line per operation. Machines start at 0.The bounds are in bks.json. Each instance has a status, a lower bound and an upper bound. Each bound has a history list with value, date, solver, hardware, time and certificate. A certificate is a stored solution or proof that a person can check. Many history fields have the value "?".
Job-shop results in papers are often hard to compare. Authors use different subsets, time limits, hardware and reference values. This library fixes one set of reference bounds and one run protocol. Thus, a new method can report one number against a stable target.
The library also shows which instances are still open. An open instance is an instance with no proof of optimality. For an open instance, a new lower bound or a new upper bound is a real result. The archive records it with the source and the date.
For this project: the REALM-Bench JSSP notes plan to add "OR-Tools CP-SAT reference makespans" for the J1 instances (DMU, TA, ABZ, SWV, YN). All five families are in JSPLib. Thus, jobshop/solutions/bks.json can supply the reference bounds directly, and the J1 gap can use the best known bound instead of a new CP-SAT run. This link to the project is my inference from the project note knowledge/claude/realm-bench-jssp-findings.md.
The maintainers do not generate new tasks. They collect published instance sets from 1963 to 2025 and convert them to the shared JSON format. JSPLib has 3 groups:
Each engine reports a lower bound LB and an upper bound UB for each instance. The score compares them to the best known bounds LBbest and UBbest. The site gives three geometric averages:
An lb value of 1.00 means that the engine finds the best known lower bound. An ub value of 1.00 means that the engine finds the best known schedule. A gap of 0% means that the engine proves optimality on all instances. The site prints lb and ub without the 1/N factor. As printed, they are products, not averages. I added 1/N here because the reported values (for example 0.89) only make sense as averages (inferred).
Use the figure below to see one instance. The values for ta40 come from the JSPLib page: the Carlier lower bound is 1617, and the optimum is in [1658, 1669].
The archive puts each instance in one class. The class depends on the time that the fastest reference engine needs to prove optimality:
FJSPLib uses the same classes, but with one difference. In FJSPLib, "closed" means that a publication claims the optimum, but no reference engine has reproduced the proof yet. When an engine reproduces it, the instance moves to toy, easy, medium or hard. JSPLib lists no instances in that state.
Only JSPLib has a results table. FJSPLib gives the raw results in its solutions folder but no summary table. "Solved" means that the engine returned a valid schedule. "Optimal" means that the engine also proved optimality in 600 s.
OptalCP has the best value in each group. Its lb ratio is 0.99 or 1.00 in all groups, so its lower bounds are almost always the best known bounds. CPO and CP-SAT are close on upper bounds (ub 1.03 for both on all instances). But CP-SAT proves fewer instances optimal (44% against 51%). Cplex, the MIP engine, proves 7% optimal and returns no schedule for the industrial and stress-test groups.
No engine proves an open instance optimal in 10 minutes, as expected. On the stress test, CP-SAT returns no schedule in 600 s. The JSPLib page says that CP-SAT can use the full run for preprocessing on tai_1000_1000. On the same instance, CPO and OptalCP need about one hour to find a first solution, but a hand-written greedy needs about one minute.
knowledge/claude/realm-bench-jssp-findings.md (REALM-Bench J1 families).