PUBLICATION REVISION 9
Methodology and evidence files
For each fixed case, the benchmark records what valid result each compatible tool produced with the listed settings, time limit, CPU limit, and memory limit. Errors, unsupported cases, and mathematical proofs are reported separately.
Fix the input
Each row records a scenario ID and SHA-256 file hash. Changing the assignment problem requires a new scenario and publication revision.
Run the tool
Locally installed tools start in fresh processes on the same machine with eight pinned CPUs. Hosted products use their normal public interface.
Check the schedule
A separate validator checks assignments, capacities, required constraints, objectives, and contact counts. It does not repair the schedule.
Resource and attempt contract
heavy
300 seconds
8 pinned logical CPUs for native processes
standard
60 seconds
8 pinned logical CPUs for native processes
- • Randomized settings use seeds 101, 202, 303, 404, 505 in fresh processes.
- • Deterministic settings are run once.
- • Input preparation, solving, and output conversion all count toward the stated time limit.
- • Times observed on hosted websites are not compared with times measured on the benchmark machine.
Status vocabulary
A missing schedule is not automatically a timeout, and a timeout is never an infeasibility proof.
- Optimal
- A zero lower bound or declared target was attained.
- Optimal, proven
- The solver returned a matching bound and verified proof evidence.
- Feasible
- At least one canonically valid schedule was returned.
- Infeasible, proven
- The exact hard-feasibility problem was proven infeasible.
- No candidate timeout
- The declared search envelope ended without a candidate.
- Error
- The process or evidence pipeline failed; no timeout or infeasibility is inferred.
- Unsupported
- The tool cannot represent the case or exceeds an observed product limit.
- Unavailable
- A runnable licensed environment was not available.
- Invalid output
- Output was returned but failed canonical validation.
- Pending verification
- The workflow or capability has not yet entered publication evidence.
- Not run
- No execution was admitted for this row.
Allowed and forbidden
Allowed
- Family- or case-specific formulations and symmetry.
- Disclosed solver-generated construction and staged search.
- Strongest exposed end-user quality settings.
Forbidden
- Embedded known schedules or answers.
- Starting schedules produced by another solver.
- Proxy cases, weakened semantics, or repaired output.
- Selection by a metric other than the declared objective.
Included tools, OR-Tools models, and corrections
Which tools are included
Eligible tools can express the exact row. Unsupported tools cannot represent its semantics or observed size. Unavailable means no reproducible licensed environment. Verify means the hosted workflow has not entered publication evidence.
OR-Tools process
Case-family formulations, symmetry, workers, constructors, and parameters are frozen before measured reruns. The archive includes the selected configuration and formulation source.
Corrections and reruns
A changed comparator receives a new evidence revision and fresh required attempts. Rejected, dry, or superseded evidence is excluded. A semantic case change requires a new scenario ID and publication revision.
Downloadable publication bundle
| File | Size | SHA-256 |
|---|---|---|
| publication.json | 138.6 KiB | sha256:3eca6a1e9058967882885bac7ec37e9ecf49b100b1703f7a84d65971d169346d |
| native-aggregate.json | 139.1 KiB | sha256:6942101a635237954646d981554ae1da83eb4187ff0c403738d82bc947ffc71e |
| publication.schema.json | 3.7 KiB | sha256:d2272bdcb0e58c921cccad78b2df3b7bc3d33fcacb8de49a7185e6f81b70438a |
| evidence.tar.gz | 2.2 MiB | sha256:30538c718eb1b7c4826092cf1f20c576b41ab852af85210decb0f605f1e392d5 |
Reproduce and verify
# verify the checked-in publication
cd webapp && npm run benchmark:data:check
# inspect the machine-readable native aggregate
jq '.summaries[]' public/benchmarks/public-group-generator-v1/revision-9/native-aggregate.json
# verify downloadable files
cd public/benchmarks/public-group-generator-v1/revision-9 && sha256sum -c SHA256SUMSEvidence versions used
Publication revision 9 resolves comparator-specific evidence revisions. Rejected and dry evidence is excluded by generation.