There’s a class of industrial optimization problems that sits just out of reach of classical solvers. The physics-based simulation model exists and is accurate — but it’s a numerical black box. You can run it, but you can’t hand its equations to an optimizer. And even if you could, the model might have thousands of state variables, making direct optimization computationally prohibitive.
Part of why this class of problems has remained difficult isn’t just computational — it’s that assembling the pieces has required significant implementation effort that falls outside most researchers’ core focus. Embedding a trained neural network into an optimization model, for instance, means manually transcribing hundreds of weights, biases, and activation functions into algebraic constraints. That friction quietly shapes what approaches get tried at all.
A recent paper from researchers at McMaster University, working through the McMaster Advanced Control Consortium (MACC) hits both of these problems head-on, using the market-driven optimization of a cryogenic air separation unit (ASU) as the test case. The solution they land on is clean: reduce the problem dimension using principal component analysis (PCA), learn the dynamics with a neural network, then embed that network directly into an optimization model as algebraic constraints. What they get is a solver-friendly, closed-form representation of a complex industrial process — and a nearly 1,000x speedup over the original simulator-based approach.
The problem with high-fidelity simulators
High-fidelity first-principles models are built for accuracy, not for optimization. There is an inherent tradeoff: the more faithfully a model captures the physics of a complex process, the harder it becomes to optimize over. Surrogate models offer a practical middle ground — they learn from simulation data and produce a simplified representation that is fast and tractable enough to optimize over directly. The full-order model remains the ground truth; the surrogate is built from it, not instead of it.
Two barriers, one solution
The authors identified three distinct problems and addressed them together.
First, size. A full-order model containing over 3,800 correlated states is a large space. PCA reduces each full state snapshot to just 10 latent states, capturing over 99.9% of the variance. The reverse transform of the PCA model maps back to selected full-order states only when needed (e.g. for constraint checking), keeping the model size low.
Second, accuracy. While reduced-order modeling techniques may offer significant speed improvements, solution quality must be retained. In addition to selecting an adequate number of latent states, the authors implement and enforce a trust region during optimization based on constraining the approximated squared prediction error (SPE) metric derived from the PCA model.
Third, and more fundamentally, mathematical structure. A trained ReLU neural network is structurally much more straightforward than a high-fidelity dynamic simulation. A network with ReLU activations is just matrix multiplications and max(0, x) operations — both expressible as explicit algebraic equations. The network learned to predict how the 10 latent states evolve over one 15-minute control interval, given current latent states and 4 manipulated inputs. The network is also trained to predict the SPE metric corresponding to each evolved latent state. Once trained, every neuron’s computation can be written as a constraint. The optimizer doesn’t call the model — the model is part of the problem formulation.
Where GAMSPy comes in
Embedding a trained neural network into an optimization problem sounds straightforward in principle but is genuinely painful in practice. A network with even a modest number of neurons means hundreds of weight parameters, bias terms, and activation functions that all need to become explicit algebraic constraints — written correctly, consistently, and in a form the solver can work with. Doing this by hand is error-prone and time-consuming. Maintaining it as the network architecture changes is worse. For most researchers, this implementation burden is a real barrier that shapes what approaches they even consider.
GAMSPy removes that barrier. Embedding a trained neural network into an optimization problem is a supported, robust, actively developed capability — not a workaround or a one-off trick. The researcher describes the network architecture and hands over the trained weights; the framework handles the algebraic transcription. This is what allowed the authors to focus their effort on the broader surrogate modeling framework — the PCA projection strategy, the trust region design, the model validation — rather than getting bogged down in the mechanics of constraint generation.
Concretely, ReLU activations can be reformulated in multiple ways for embedding into an NLP or MIP — complementarity conditions, binary variables, or other representations depending on the solver and problem structure. GAMSPy supports these options, and in this case the complementarity-based reformulation via relu_with_complementarity_var was the natural fit for an NLP formulation.
The choice of CONOPT as the solver is also worth noting. This problem — neural network constraints mixed with physical operational constraints, across a 5-hour horizon — is large and nonlinear, with a solution landscape that has many local optima. Global optimizers would struggle to converge on a problem of this structure and size. CONOPT’s strength is finding high-quality local solutions efficiently, and for market-driven process optimization where you need a good answer fast, that’s exactly the right trade-off. The results bear this out: a 4.05-second solve with an objective value within 0.082% of the full-order model.
The numbers
The authors compare solve times and objective function values utilizing both the full-order and reduced-order models. A second-tier optimization problem is also solved to demonstrate and mitigate effects of nonuniqueness in the optimal input trajectory calculation.
| Solve time | Objective value | |
|---|---|---|
| Full-order model (gPROMS) | 3,337 sec | 222.03 |
| Surrogate Model - Tier 1 (GAMSPy) | 4.05 sec | 221.85 (actual) |
| Surrogate Model - Tier 2 (GAMSPy) | 1.45 sec | 221.94 (actual) |
High degree of accuracy. Nearly three orders of magnitude faster. The optimal policy — store liquid nitrogen during cheap electricity periods, evaporate during peak pricing — matched the full-order model’s recommendation exactly.
The SPE-based trust region constraint constraint also proved useful diagnostically: it flagged when the surrogate was being stretched near the compressor surge limit in the later part of the horizon, pinpointing exactly where additional training data would improve model accuracy in future iterations.
The broader pattern
The surrogate modeling approach demonstrated here is not specific to air separation. The same pattern applies to any high-dimensional process where a simulator exists but is too opaque or slow for direct optimization — reduce the state space dimensionality with PCA, learn the dynamics with a ReLU network, embed into GAMSPy, enforce a trust region via the SPE constraint. Any high-dimensional process with correlated states and an existing simulator is a reasonable candidate.
The research stands on its own as a contribution to surrogate modeling and process optimization. What GAMSPy contributed was removing a specific friction point — the transcription of trained neural networks into optimization constraints — that would otherwise have required significant implementation effort outside the core research scope. That the embedding step could be handled cleanly meant the methodology could be assembled and tested at realistic industrial scale, on a real process model, without that step becoming a bottleneck.
And that last point deserves emphasis. When embedding a neural network into an optimization problem is a supported framework feature rather than a manual exercise, researchers can focus on the hard parts: what to model. As optimization tools grow more fluent in the language of machine learning, the class of problems that can be attacked this way keeps expanding. Real-time optimization of complex industrial processes, previously out of reach for models of this size, starts to look feasible.
Acknowledgements
Thanks to Dr. Christopher Swartz and Kieran McKenzie of McMaster University for their contributions to this work and for their support in bringing this story together.
McKenzie, K., Swartz, C.L.E., Corbett, B. “Dynamic Surrogate Modeling Using Latent Variable Methods and Neural Networks for Market-Driven Operation of an Air Separation Unit.” Ind. Eng. Chem. Res. 2026, 65, 584–599.
This research was conducted through the McMaster Advanced Control Consortium (MACC) , a collaborative research program between McMaster University and industry partners focused on advancing process control and optimization.