Expanding the Focus: Introducing the MIPfeas Benchmark

Posted on: 17 Mar, 2026 Last updated: 30 Oct, 2026 Solvers

Update (October 2026): This update brings all solvers up to their latest versions and directly resolves a key concern regarding the MIPfeas benchmark: hardware discrepancies across runs. Moving forward, all solvers are evaluated on a single, standardized platform: the NVIDIA DGX Spark, equipped with an ARM CPU (10 Cortex-X925 performance cores and 10 Cortex-A725 efficiency cores) and a GB10 GPU. By pairing a strong modern CPU with a capable GPU, the Spark system represents accessible, commodity hardware that offers a fair compromise for both CPU- and GPU-accelerated algorithms. To ensure an equitable playing field across the performance cores, solver worker and thread counts are capped at 10. Additionally, we replaced the previous distribution and violin plots with a clearer visualization of the primal integral across solvers. This iteration was prepared specifically for the MIPfeas benchmark presentation during Hans Mittelmann’s session at the 2026 INFORMS Annual Meeting.

Update (September 2026): The article covers new versions of cuOpt 26.08 and HiGHS 1.15.1 which comes with a parallel B&C now utilizing the 24 cores more effectively. Moreover, the result tables of the previous iterations have been added at the end of this page to see the progress over versions and hardware improvements.

Update (July 2026): The article has been extended and includes now result from SCIP Concurrent solver. SCIP-Conc is a parallel solver for mixed-integer programming, built on SCIP and has been around for some time. Recent activities for SCIP-Conc have fixed many open bugs and improved on the selection of options for the various threads. This benchmark uses a preview of SCIP-Conc of the soon to be released SCIP Optimization Suite 11.

Update (May 2026): The article has been extended and includes now result from the ReXi solver. ReXi is a parallel solver for mixed-integer programming, built on SCIP and SoPlex, focused on quickly finding strong primal solutions. ReXi is a private research project by Gioni Mexi and Daniel Rehfeldt . Moreover, the CPU based solvers runs have been moved to a fast CPU. The instances are small enough to run on the faster desktop CPU Intel Core Ultra 9 285K rather than the server CPU AMD EPYC 9B45.

Update (April 2026): The article has been extended to cover the latest release of the GAMS/cuOpt solver link (v0.0.6 ) for NVIDIA cuOpt 26.04 . Testing on the new NVIDIA B200 GPU reveals a significant performance jump: the geometric mean dropped from 0.0651 (on H100) to just 0.0383 — a result of both software improvements and the Blackwell architecture. Even when holding hardware constant on the NVIDIA GB10, the software alone drove the geometric mean down from 0.0887 to 0.0426, proving that the latest cuOpt version delivers a significant improvement in this benchmark on its own merits.


In the world of Mixed-Integer Programming (MIP), the “gap”—the distance between our incumbent solution and the dual bound—remains the gold standard. This mathematical rigor is precisely why we use MIP: it provides a proven certificate of quality that other heuristic methods cannot match. However, while a solution’s proof of optimality is crucial, the path to that solution is equally important.

In many fast-paced business environments, having a good feasible point is the primary requirement: a proven optimal solution or a good dual bound is secondary. In such situations, the MIP framework’s greatest strength is its flexibility. Even when a problem cannot be solved to optimality within a tight time window, MIP solvers often deliver high-quality feasible solutions very early in the search. By measuring how quickly a solver narrows the distance to the optimal value, we can see how MIP extends its reach into real-time decision-making and other realms where speed is as vital as proof.

Unlike pure heuristics, the MIP framework maintains a deterministic stopping criterion and continues to provide a dual bound, offering a formal architecture to prove when a solution can no longer be improved. By adding the Primal Integral to other well-known metrics like time to optimality, the MIPfeas benchmark helps to classify MIP solvers by their ability to deliver high-value feasible points in the critical opening minutes of a solve.

The Metric: Defining the Primal Integral

To capture this “quality over time” performance, we look to the Primal Integral, a metric introduced by Timo Berthold12 to evaluate the effectiveness of primal heuristics. Instead of just looking at the final result, the Primal Integral accounts for every improvement the solver finds throughout the entire run. To understand how the Primal Integral is calculated, we can trace the progress of a single model—in this case, the MIPLIB 2017 instance supportcase7.

During a solve, the branch-and-cut algorithm tracks two key values: the incumbent (the best primal feasible solution found so far) and the dual bound (the theoretical limit of how good the solution could possibly be).

Figure 1: Incumbent and Dual Bound over time for MIPLIB 2017 instance supportcase7

We transform and scale the area between incumbent values and the known optimal solution value by integrating the primal gap function p(t). Mathematically, if x* is the optimal value and x(t) is the incumbent at time t, the primal gap function p(t) (aka penalty) measures the relative difference, i.e. the relative distance from x(t) to x*. In the MIPfeas benchmark, we have slightly adapted the original penalty structure introduced by Timo Berthold taking advantage of all problems being minimization problems:

  • The “No-Solution” Penalty: If no incumbent is found, we set the penalty to 2. We believe that in a business context, the absence of any feasible solution is a significantly greater hurdle than a poor one.
  • Handling “Sign-Flip” and Zeros: When the incumbent and the optimal solution have opposite signs, we set the penalty to 1. When x* and x(t) are close enough to 0 (<1e-6), we set the penalty to 0. There are other ways of handling these cases. The scripts calculating the primal integral shared as part of this blog provide an alternative where the optimal value and all incumbents are shifted to strictly positive values allowing to apply the formula given below in all cases.
  • The Progress Metric: Once a solution with the correct sign is found, the penalty is at most 1 and decreases to 0 as the incumbent approaches the optimal value:

$$ p(\bar{x}(t)) = \frac{\bar{x}(t) - x^*}{\max(|\bar{x}(t)|, |x^ *|)} $$

The Primal Integral P(T) is the area under this curve over the total time T scaled, i.e. divided by T. So the value of the Primal Integral is between 0 and 2.

Figure 2: Primal Gap over time for MIPLIB 2017 instance supportcase7

Mathematically, the integral is expressed as $ P(T) = \int_{0}^{T} p(t) dt / T $

By focusing on the area rather than just the final point, the Primal Integral rewards solvers that are “fast out of the gate.” A solver that finds a 99% optimal solution in the first 10 seconds will have a significantly lower (better) score than a solver that finds that same solution only in the final 10 seconds of a 600-second run.

The Rise of GPU-Accelerated Heuristics

An intriguing side aspect of this benchmark has been the recent, rapid advancement in GPU-based heuristics. While commercial MIP vendors have so far stayed on the sidelines of this development, the availability of open-source implementations like NVIDIA cuOpt allowed us to put these GPU-based methods to the test against more traditional stalwarts.

Benchmark Design and Methodology

To ensure a rigorous and transparent comparison between traditional and emerging technologies, we standardized the testing environment across a diverse problem set:

  • The Test Suite: We selected 233 instances from the MIPLIB 2017 benchmark collection. To ensure the primal integral remains a meaningful metric of solution quality, we specifically excluded instances known to be infeasible.

  • Computational Limits and the “Power of Defaults”: To simulate a high-performance workstation environment, each solver was allocated a strictly enforced time limit of 600 seconds and utilized 10 threads (--workers 10 for ReXi).
    We explicitly note that for this benchmark, we performed a single run per instance-solver pair. In an ideal computational study, one would perform multiple runs using different random seeds or instance permutations to account for “performance variability”—the phenomenon where minor internal differences can lead to significantly different search paths. While we did not have the computational resources to perform multiple runs per combination, we believe that our large test set of 233 instances provides the statistical breadth necessary to mitigate the impact of “good or bad luck” on any single model, revealing the true underlying performance trends of each engine.
    All solvers in this initial round were run using their default settings. We chose this approach for several strategic reasons:

    • Fairness across Engines: Some solvers offer “meta-options” to prioritize feasibility over optimality, while others require manual adjustment of granular settings like heuristic frequency or branching rules. Sticking to defaults avoids the need for subjective “manual fiddling” to achieve parity.
    • The Robustness of Defaults: Solver developers invest significant resources into ensuring that default parameters perform reliably across a diverse library of instances, providing a stable “out-of-the-box” intelligence baseline.
    • Avoiding the Tuning Trap: While hyper-parameter tuning is powerful for specific models, it can often become a liability when software versions change or data distributions shift.
      By prioritizing defaults, we are evaluating the general-purpose strategies prioritized by the developers. However, we view this as a starting point. Future iterations of the MIPfeas benchmark may expand to include entries with tuned feasibility configurations or even pure heuristic solvers whose sole purpose is to produce high-quality solutions rapidly. This evolution will allow us to compare the rigorous search of a standard MIP solver against specialized engines designed specifically for real-time, high-speed decision-making.
  • The Contestants:

    • Open-Source Ecosystem: We evaluated Cbc, HiGHS, SCIP, and SCIP-Conc, representing the most widely used and established open-source MIP solvers. We also include ReXi in this category, although its licensing and public availability are not yet finalized. Unlike traditional solvers (and because base SCIP is strictly single-threaded), ReXi and SCIP-Conc take a fundamentally different approach to parallelization. Rather than dividing a single branch-and-bound tree across threads to explore open nodes concurrently, they orchestrate a portfolio of concurrent solves. Each worker thread executes an independent solver instance with a tailored strategy—such as aggressive cutting, heavy primal heuristics, or specialized branching rules. These instances run on the exact same problem in parallel while asynchronously exchanging primal incumbents, dual bound updates, and valid global cuts through a lightweight coordination layer.

    • GPU-Accelerated Innovation: We evaluated cuOpt, an open-source, GPU-accelerated solver requiring an NVIDIA GPU.

    • Commercial Performance: While we are unable to publish individual results for commercial solvers (see below), we believe it is essential to demonstrate the current performance gap between open-source solvers and the state-of-the-art in commercial optimization.

      To bridge the gap between open-source and commercial performance without violating legal terms, we have grouped the results of three major commercial solvers—COPT, CPLEX, and Xpress—into three synthetic baselines: the Virtual Best, Mean, and Worst Commercial Solvers (VBCS, VMCS, and VWCS).

      • The Virtual Best (VBCS): Created by taking the best (smallest) Primal Integral achieved by any of the three commercial solvers for every single instance.
      • The Virtual Worst (VWCS): Created by taking the worst (largest) Primal Integral per instance. This provides a reliable “lower bound” on what one can expect from a top-tier commercial engine.
      • The Virtual Mean (VMCS): The average of the three values per instance, offering a sense of the “average” performance across the commercial landscape.

      It is important to note that the VBCS represents an “upper bound” on the potential of commercial technology; no single solver likely achieved this level of performance across all 233 instances. However, by including this metric, we provide a clear “north star” for the benchmark. Conversely, the VWCS and VMCS provide a grounded reality of the commercial sector.

      This approach allows us, after many years of absence for some of these solvers from public benchmarks, to include them once again. It enables us to quantify exactly how much ground open-source solvers have gained while highlighting the remaining performance premium offered by the commercial sector in finding high-quality solutions under tight time constraints.

  • Computational Environment & Hardware:

    • All runs were conducted using GAMS (Linux arm64). The solver versions used were:
      • Cbc: 2.10.13
      • COPT: 8.0.7
      • CPLEX: 22.2.0.0
      • cuOpt: 26.10.0 (git hash: e93f7dc5q)
      • HiGHS: 1.15.1 (git hash: 04024d701f)
      • ReXi (git hash: 5d5e28f7cccf) based on ReXiLD (46d70a1d1f66), PaPILO (83b49bc3), customized SCIP (062aa158ff), and SoPlex (e7167403)
      • SCIP: 10.1.0 (git hash: c8e5737a84) with LP solver SoPlex
      • SCIP-Conc: 11.0.0 (git hash: c8e5737a84) with LP solver SoPlex
      • Xpress: 9.9.1
    • All runs were performed on an NVIDIA DGX Spark machine, equipped with an ARM CPU (10 Cortex-X925 performance cores and 10 Cortex-A725 efficiency cores) and a GB10 GPU. By pairing a strong modern CPU with a capable GPU, the Spark system represents accessible, commodity hardware that offers a fair compromise for both CPU- and GPU-accelerated algorithms. Thanks to NVIDIA generously providing hardware access, we also had an NVIDIA DGX B200 at our disposal. While evaluated, its high cost, limited availability, and lower CPU performance unfairly penalize CPU-based methods, ruling it out as the benchmark standard. We still report out-of-competition cuOpt runs on the B200 (Intel® Xeon® Platinum 8570, 24 threads) as an exhibition of top-tier enterprise capabilities.

Data Collection: The GAMS Solvetrace Advantage

Parsing solver logs is a notorious headache—formats change, and information is often missing. For this benchmark, we utilized the GAMS solvetrace facility. By using solver callbacks, solvetrace records the progress of the incumbent over time in a structured and consistent format. Crucially, it standardizes the measurement of wall-clock time across all engines, regardless of their internal timing defaults. This ensures a rigorous “apples-to-apples” comparison and allows for a more automated, high-fidelity calculation of the Primal Integral. The ReXi solver has no GAMS link yet, so the trace files have been created from the ReXi logs via a Python script supplied by the ReXi authors. We have verified (by acceptance sampling) the reported ReXi timing by checking the file date of the solution files created via --dump-solutions.

Note to the Community: Benchmarks Public or Private

One cannot mention benchmarking and optimization without considering the Mittelmann benchmarks . Over the course of many years his public benchmarks have informed the community and spurred discussion and development. We are grateful to Prof. Mittelmann for his work in this area and particularly for including the current MIPfeas results on his benchmark page .

Recognizing the legal complexities of “DeWitt clauses” —legal provisions that prohibit the public disclosure of comparative benchmark results— we have worked constructively with our commercial solver vendors to find a path forward. By introducing the Virtual Best, Mean, and Worst (VBCS, VMCS, VWCS) categories, we have successfully bridged the gap—incorporating state-of-the-art commercial results into a public benchmark while respecting necessary legal frameworks.

However, we recognize that for many users, “virtual” is not enough. We have made all our benchmarking tools and scripts publicly available for those creating private, internal benchmarks. If you hold a license for a specific commercial solver, our framework is designed to be plug-and-play. You can easily integrate your licensed solver into our scripts to generate a direct, private comparison against our public contestants in your own environment. Please note, however, that while our tools facilitate internal evaluation, most commercial solvers require explicit approval from the vendor before any comparative benchmark results involving their software can be published or shared publicly.

For the academic community, the barrier to entry is even lower. Researchers can obtain a GAMSPy academic license , which provides integrated access to all the open-source solvers mentioned here (Cbc, HiGHS, SCIP), as well as a wide array of commercial MIP engines. Since the solvetrace facility is natively available within the GAMSPy ecosystem, every academic with access to compute power—whether it’s a standard CPU cluster or high-end GPU hardware—can replicate this benchmark or, perhaps more importantly, run it on their own unique sets of models.

Acknowledgments

This benchmark is the result of a broad collaboration across the optimization community, and we are deeply grateful to those who provided the technical foundations and guidance necessary to bring it to life.

First, we would like to thank Hans Mittelmann for his invaluable guidance and for the honor of hosting this benchmark on his renowned benchmarking site. We also extend our gratitude to ZIB (Zuse Institute Berlin) for their support and guidance throughout this project.

We extend our sincere thanks to the NVIDIA team, specifically Burcin Bozkaya, Akif Coerduek, and Chris Maes. Their efforts in implementing the incumbent callback within cuOpt were essential to enabling the GAMS solvetrace facility, and their overall guidance was instrumental throughout the creation of this benchmark and the accompanying blog post. In the fifth iteration (September 2026), a run with cuOpt on the machine with the B200 provided to us had a significant setup time before cuOpt actually started real work. Thanks to Alice Boucher we identified ways to not record this setup time in the times used for the benchmark.

Finally, we are grateful for the valuable comments, insights, and support provided by our colleagues across the solver industry. Their expertise—ranging from high-level methodology to the fine details of the Primal Integral—was indispensable. Special thanks go to Timo Berthold (Xpress), Gerald Gamrath (COPT), Ferenc Katai (CPLEX), Gioni Mexi and Daniel Rehfeldt (ReXi).

Results

The following table summarizes the performance of the contestants, including the virtual commercial solver groups, across the 233 MIPLIB instances. The individual solvers are listed alphabetically, followed by the virtual solvers, to provide a clear reference for specific engine results.

How to present a large volume of Primal Integral (PI) data in a meaningful way is a nuanced question. In the original Primal Integral paper, the Arithmetic Mean was used because the primal gap is already a scaled measure of performance. However, following a discussion with Timo Berthold, we have introduced additional statistical rigor to this benchmark.

Because PI spans multiple orders of magnitude and functions similarly to a time measurement (dynamically scaled by the gap), we include the Shifting Geometric Mean with a shift of 0.001. This ensures that the mean is not unduly skewed by outliers or extremely small values, providing a more robust “center” for the performance distribution. Moreover, we added some other statistical information like the median and the interquartile range.

Since a Primal Integral of 2 represents a failure to find any integer-feasible point, we report the total count of instances (“Feasible found (#)”) where a solver successfully returned at least one primal feasible solution. We also record how many instances each solver managed to solve to optimality (“Proven Optimal (#)”), i.e. the solver not only found the optimal solution but also proved it to be the optimal solution. When viewed alongside the Primal Integral, this count offers a comprehensive look at how effectively a contestant balances its two primary tasks: the search for early, high-quality feasible points and the effort required to prove optimality.

cbc cuopt cuopt-b200 highs rexi scip scipconc VBCS VMCS VWCS
Arithmetic Mean 0.5391 0.0666 0.0642 0.3052 0.1503 0.3893 0.1747 0.0308 0.1044 0.1916
Arithmetic Mean [Scaled] 17.5241 2.1635 2.0883 9.9213 4.8870 12.6545 5.6782 1.0000 3.3941 6.2295
Geom. Mean (shift=0.001) 0.0980 0.0088 0.0076 0.0447 0.0103 0.0609 0.0168 0.0044 0.0122 0.0202
Geom. Mean (0.001) [Scaled] 22.3054 2.0011 1.7192 10.1833 2.3453 13.8699 3.8206 1.0000 2.7832 4.5940
Median 0.1012 0.0048 0.0044 0.0374 0.0058 0.0622 0.0116 0.0027 0.0082 0.0133
Interquartile Range 0.8471 0.0339 0.0272 0.2500 0.0396 0.4000 0.0743 0.0120 0.0470 0.0814
Feasible found (#) 191 232 231 217 223 207 223 232 226 221
Proven Optimal (#) 77 86 85 128 113 82 115 212 180 144

Visualizing the Performance Landscape

To provide a deeper look into these numbers, we utilize two complementary representations. In these charts, the solvers are ordered by their shifted geometric mean (from worst to best) to highlight the performance hierarchy.

The Bar Chart offers a direct comparison of the overall shifted geometric mean, serving as the primary scoreboard for the benchmark.

Figure 3: Shifted (0.001) Geometric Mean of Primal Integral P(T)

The plot below shows the cumulative distribution of primal integral values across solvers. For each solver, a curve displays the fraction of instances where its primal integral is at most x (shown on a logarithmic scale); curves positioned further up and to the left indicate superior performance. A curve terminates at the overall fraction of instances for which a feasible solution was found, as instances with a primal integral of 2 (unsolved within time limits) are excluded. The visualization is split into three panels by solver category:

  • Orchestrated portfolio solvers: ReXi and SCIP-Conc
  • GPU-accelerated solver: cuOpt
  • Traditional open-source solvers: HiGHS, SCIP, and Cbc

To aid comparison across groups, each panel overlays the solvers from the other panels in muted light grey, along with a shaded grey band spanning the virtual best to the virtual worst commercial solver (with the virtual mean dashed). A solid dot indicates each solver’s median primal integral.

Figure 4: Distribution of Primal Integral P(T)

Technical Appendix: The Data Pipeline

Model Pre-processing: MPS to GDX

The 233 MIPLIB instances are distributed in gzipped MPS format. Since GAMS and GAMSPy do not consume MPS files directly, we utilize the mps2gms utility. This tool converts the MPS file into a GDX (GAMS Data Exchange) file, which contains the complete model structure—objective function, bounds, variable types, and the constraint matrix. The instance, in form of a GDX file, can then be solved with GAMS or GAMSPy via the generic scripts mipfeas.gms and mipfeas.py. ReXi ran directlty on the gzipped MPS files via the command line option --workers 10 --timelimit 600.

An important side effect of the file conversion process is that constraints may be reordered compared to the original MPS file. While this reordering creates a different internal matrix structure than the source file, it is not expected to impact the benchmark in any significant way. We recognize that performance variability remains a factor for any individual model, regardless of whether the original or a permuted matrix order is used. As previously noted, we rely on our large test set of 233 instances to smooth out these variations, ensuring that the resulting aggregate metrics reflect the genuine performance trends of each contestant.

We also used the utility to force a consistent minimization sense across all models to simplify the downstream integral calculations.

# Example conversion loop
cd ${HOME}/mipfeas/mps_files
for file in *.mps.gz
do
   # For GAMS users
   mps2gms $file GMS= PY= CONVERTSENSE=MIN COLUMNINTVARSAREBINARY=1 
   # For GAMSPy users
   gamspy mps2gms $file --py /dev/null --convertsense MIN --columnintvarsarebinary 1
done

Job Submission and Execution

To handle the scale of 233 instances across multiple solvers, we utilized the Slurm workload manager. Each job was allocated 10 threads and a strict time limit.

A critical component of our execution script is the multi-layered timeout protection. Since some solvers may occasionally fail to respect the 600 seconds time limit, we wrap the call in a Linux timeout utility, while Slurm acts as the final arbiter to terminate the process:

cd ${HOME}/mipfeas/log_files
let "cnt=99"
for file in ../mps_files/*.gdx
do
  let "cnt++"
  f=`basename -- $file .gdx`
  for s in cbc cuopt cuopt-b200 highs rexi scip scipconc
  do
      # Submit to Slurm with appropriate hardware requests,
      # add options to specify partition, memory, GPU, and
      # other constraints
      sbatch --time 00:20:00 --cpus-per-task=10 -o ${HOME}/mipfeas/log_files/${f}_${s}.log runme.sh $f $s $cnt 
   done
done

The previous script submits the script runme.sh

#!/bin/bash
# Using /usr/bin/time to capture OS-level resource stats
if [ "$2" == "rexi" ]; then
    /usr/bin/time -v timeout -k 750 730 ReXi --instance ${1}.mps.gz --workers 10 --timelimit 600
else
    # For GAMS users
    /usr/bin/time -v timeout -k 750 730 gams mipfeas.gms --instance $1 --solver $2 optfile $3 resLim 600 threads 10 writeOutput 0 
    # For GAMSPy users 
    /usr/bin/time -v timeout -k 750 730 python mipfeas.py $1 $2 $3 600 10
fi

Creating the Solvetrace

The core of the benchmark relies on the mip/solvetrace facility. Our customized mipfeas.gms and mipfeas.py scripts accept the instance and solver names as parameters, then dynamically generate the solver-specific option files required to trigger the trace. For ReXi a Python script has been used to create trace file from the ReXi log files.

Because the syntax for solvetrace can vary slightly between solvers, the script handles these nuances internally to ensure that every run produces a standardized .trc file. These trace files contain the vital “step-function” data of incumbent improvements over wall-clock time.

Data Processing and Integral Calculation

Once the raw .trc files are generated, we use a Python script (create_primalintegral.py) to process the results and create the individual primal integral values and indicator for having found the optimal solution for each instance and solver and stores them in CSV (primalintegral.csv, optimalindicator.csv, and feasibleindicator.csv) and Excel (mipfeas.xlsx with sheets “Primal Integral Values”, “Optimal Solution Indicator”, and “Feasible Solution Indicator”) files. This script obtains the optimal or best-known solution from a CSV file optimal_objective.csv. Another Python script (create_plots.py) processes the CSV files primalintegral.csv, optimalindicator.csv, and feasibleindicator.csv to create the summary table and the two plots given above.

Downloads

All 233 instances (mps.gz and GDX format) with the generic GAMS and GAMSPy scripts are available for download (https://cloud.gams.com/s/SsDNbTdjk5Lo5QK ). The trace and log files of the runs (excluding the commercial solvers), the CSV and Excel files, together with the Python scripts are also available for download (https://cloud.gams.com/s/oQBzPAXDdzcKDNr ). Results from all iterations of the benchmark will be available in the directory “mipfeas” (https://cloud.gams.com/s/XkfCnXBBqbPPtaZ ).

Previous Results

Fifth Iteration (September 2026)

  • Solver versions:
    • Cbc: 2.10.11
    • COPT: 8.0.3
    • CPLEX: 22.1.2.0
    • cuOpt: 26.8.0 (git hash: 400863c1)
    • HiGHS: 1.15.1 (git hash: 04024d701f)
    • ReXi (git hash: 734c43d8897b+dirty) based on PaPILO (83b49bc3) and customized SCIP (6aabacba69) and SoPlex (1c2c4407), 10 workers
    • SCIP: 10.0.1 (git hash: 76a50c97d6) with LP solver SoPlex
    • SCIP-Conc: 11.0.0 (git hash: 6da078bd13) with LP solver SoPlex
    • Xpress: 9.8.1
  • Hardware:
    • Intel Core Ultra 9 285K
    • Intel(R) Xeon(R) 6960P with NVIDIA B200 180GB HBM3e
    • 20-core ARM-based (NVIDIA Grace Blackwell) with NVIDIA GB10 (120GB VRAM)
cbc cuopt-b200 cuopt-gb10 highs rexi scip scipconc VBCS VMCS VWCS
Arithmetic Mean 0.5722 0.1796 0.2081 0.3166 0.1890 0.4285 0.2687 0.0328 0.0994 0.1853
Arithmetic Mean [Scaled] 17.4266 5.4714 6.3394 9.6437 5.7553 13.0498 8.1843 1.0000 3.0281 5.6454
Geom. Mean (shift=0.001) 0.1015 0.0286 0.0393 0.0453 0.0165 0.0755 0.0274 0.0040 0.0105 0.0172
Geom. Mean (0.001) [Scaled] 25.6915 7.2427 9.9429 11.4722 4.1762 19.1005 6.9444 1.0000 2.6697 4.3524
Median 0.1084 0.0268 0.0328 0.0397 0.0111 0.0744 0.0211 0.0019 0.0058 0.0104
Interquartile Range 0.9308 0.1190 0.1960 0.2345 0.0963 0.5044 0.1732 0.0086 0.0342 0.0690
Feasible found (#) 188 226 226 214 223 204 216 232 227 221
Proven Optimal (#) 78 78 73 107 110 77 106 214 183 151

Fourth Iteration (July 2026)

  • Solver versions:
    • Cbc: 2.10.11
    • COPT: 8.0.3
    • CPLEX: 22.1.2.0
    • cuOpt: 26.4.0 (git hash: d9b7c96a)
    • HiGHS: 1.12.0 (git hash: 9fa8db6f9)
    • ReXi (git hash: 734c43d8897b+dirty) based on PaPILO (83b49bc3) and customized SCIP (6aabacba69) and SoPlex (1c2c4407), 10 workers
    • SCIP: 10.0.1 (git hash: 76a50c97d6) with LP solver SoPlex
    • SCIP-Conc: 11.0.0 (git hash: 6da078bd13) with LP solver SoPlex
    • Xpress: 9.8.1
  • Hardware:
    • Intel Core Ultra 9 285K
    • Intel(R) Xeon(R) 6960P with NVIDIA B200 180GB HBM3e
    • 20-core ARM-based (NVIDIA Grace Blackwell) with NVIDIA GB10 (120GB VRAM)
cbc cuopt-b200 cuopt-gb10 highs rexi scip scipconc VBCS VMCS VWCS
Arithmetic Mean 0.5722 0.1973 0.2089 0.3270 0.1890 0.4285 0.2687 0.0328 0.0994 0.1853
Arithmetic Mean [Scaled] 17.4266 6.0095 6.3621 9.9609 5.7553 13.0498 8.1843 1.0000 3.0281 5.6454
Geom. Mean (shift=0.001) 0.1015 0.0383 0.0426 0.0468 0.0165 0.0755 0.0274 0.0040 0.0105 0.0172
Geom. Mean (0.001) [Scaled] 25.6915 9.6806 10.7806 11.8335 4.1762 19.1005 6.9444 1.0000 2.6697 4.3524
Median 0.1084 0.0317 0.0443 0.0391 0.0111 0.0744 0.0211 0.0019 0.0058 0.0104
Interquartile Range 0.9308 0.1642 0.2094 0.2435 0.0963 0.5044 0.1732 0.0086 0.0342 0.0690
Feasible found (#) 188 227 227 213 223 204 216 232 227 221
Proven Optimal (#) 78 57 64 103 110 77 106 214 183 151

Third Iteration (May 2026)

  • Solver versions:
    • Cbc: 2.10.11
    • COPT: 8.0.3
    • CPLEX: 22.1.2.0
    • cuOpt: 26.4.0 (git hash: d9b7c96a)
    • HiGHS: 1.12.0 (git hash: 9fa8db6f9)
    • ReXi (git hash: 734c43d8897b+dirty) based on PaPILO (83b49bc3) and customized SCIP (6aabacba69) and SoPlex (1c2c4407), 10 workers
    • SCIP: 10.0.1 (git hash: 76a50c97d6) with LP solver SoPlex
    • Xpress: 9.8.1
  • Hardware:
    • Intel Core Ultra 9 285K
    • Intel(R) Xeon(R) 6960P with NVIDIA B200 180GB HBM3e
    • 20-core ARM-based (NVIDIA Grace Blackwell) with NVIDIA GB10 (120GB VRAM)
cbc cuopt-b200 cuopt-gb10 highs rexi scip VBCS VMCS VWCS
Arithmetic Mean 0.5722 0.1973 0.2089 0.3270 0.1890 0.4285 0.0328 0.0994 0.1853
Arithmetic Mean [Scaled] 17.4266 6.0095 6.3621 9.9609 5.7553 13.0498 1.0000 3.0281 5.6454
Geom. Mean (shift=0.001) 0.1015 0.0383 0.0426 0.0468 0.0165 0.0755 0.0040 0.0105 0.0172
Geom. Mean (0.001) [Scaled] 25.6915 9.6806 10.7806 11.8335 4.1762 19.1005 1.0000 2.6697 4.3524
Median 0.1084 0.0317 0.0443 0.0391 0.0111 0.0744 0.0019 0.0058 0.0104
Interquartile Range 0.9308 0.1642 0.2094 0.2435 0.0963 0.5044 0.0086 0.0342 0.0690
Feasible found (#) 188 227 227 213 223 204 232 227 221
Proven Optimal (#) 78 57 64 103 110 77 214 183 151

Second Iteration (April 2026)

  • Solver versions:
    • Cbc: 2.10.11
    • COPT: 8.0.3
    • CPLEX: 22.1.2.0
    • cuOpt: 26.4.0 (git hash: d9b7c96a)
    • HiGHS: 1.12.0 (git hash: 9fa8db6f9)
    • SCIP: 10.0.1 (git hash: 76a50c97d6) with LP solver SoPlex
    • Xpress: 9.8.1
  • Hardware:
    • AMD EPYC 9B45
    • Intel(R) Xeon(R) 6960P with NVIDIA B200 180GB HBM3e
    • 20-core ARM-based (NVIDIA Grace Blackwell) with NVIDIA GB10 (120GB VRAM)
cbc cuopt-b200 cuopt-gb10 highs scip VBCS VMCS VWCS
Arithmetic Mean 0.5747 0.1973 0.2089 0.3591 0.4285 0.0389 0.1115 0.2040
Arithmetic Mean [Scaled] 14.7742 5.0720 5.3696 9.2305 11.0141 1.0000 2.8657 5.2433
Geom. Mean (shift=0.001) 0.1121 0.0383 0.0426 0.0567 0.0755 0.0050 0.0132 0.0214
Geom. Mean (0.001) [Scaled] 22.4290 7.6560 8.5259 11.3398 15.1058 1.0000 2.6511 4.2911
Median 0.1209 0.0317 0.0443 0.0484 0.0744 0.0028 0.0078 0.0129
Interquartile Range 0.9451 0.1642 0.2094 0.3154 0.5044 0.0113 0.0449 0.0827
Feasible found (#) 192 227 227 212 204 232 227 221
Proven Optimal (#) 77 57 64 97 77 215 180 145

First Iteration (March 2026)

  • Solver versions:
    • Cbc: 2.10.11
    • COPT: 8.0.3
    • CPLEX: 22.1.2.0
    • cuOpt: 26.2.0 (git hash: f73da24d)
    • HiGHS: 1.12.0 (git hash: 9fa8db6f9)
    • SCIP: 10.0.1 (git hash: 76a50c97d6) with LP solver SoPlex
    • Xpress: 9.8.1
  • Hardware:
    • AMD EPYC 9B45
    • Intel(R) Xeon(R) Platinum 8481C @ 2.70GHz with NVIDIA H100 80GB HBM3
    • 20-core ARM-based (NVIDIA Grace Blackwell) with NVIDIA GB10 (120GB VRAM)
cbc cuopt-gb10 cuopt-h100 highs scip VBCS VMCS VWCS
Arithmetic Mean 0.5747 0.3065 0.2586 0.3591 0.4285 0.0389 0.1115 0.2040
Arithmetic Mean [Scaled] 14.7742 7.8797 6.6485 9.2305 11.0141 1.0000 2.8657 5.2433
Geom. Mean (shift=0.001) 0.1121 0.0887 0.0651 0.0567 0.0755 0.0050 0.0132 0.0214
Geom. Mean (0.001) [Scaled] 22.4290 17.7487 13.0263 11.3398 15.1058 1.0000 2.6511 4.2911
Median 0.1209 0.0967 0.0651 0.0484 0.0744 0.0028 0.0078 0.0129
Interquartile Range 0.9451 0.3289 0.3240 0.3154 0.5044 0.0113 0.0449 0.0827
Feasible found (#) 192 223 225 212 204 232 232 221
Proven Optimal (#) 77 60 57 97 77 215 180 145

  1. Berthold, Timo. “Measuring the impact of primal heuristics.” Operations Research Letters 41.6 (2013): 611-614. ↩︎

  2. Berthold, Timo, and Csizmadia, Zsolt. “The confined primal integral: a measure to benchmark heuristic MINLP solvers against global MINLP solvers.” Mathematical Programming 188.2 (2021): 523-537 ↩︎