Tutorial

The goal of this tutorial is to provide a compact overview of the basic functionality of the GAMS Python API. It allows the user to start immediately working with the API by providing a set of small examples based on the well-known transportation problem. These examples introduce several API features step by step.

# Getting started

The object oriented GAMS Python API is built on top of the different expert-level APIs and provides convenient access to GAMS from within Python. Examples using the API are located in apifiles/Python. The expert-level APIs can be found in apifiles/Python/api_36 for Python 3.6, in apifiles/Python/api_37 for Python 3.7, in apifiles/Python/api_38 for Python 3.8, in apifiles/Python/api_39 for Python 3.9, and in apifiles/Python/api_310 for Python 3.10. The object oriented API is located in apifiles/Python/gams. Making your Python installation aware of the modules provided by GAMS can be done in two fundamentally different ways. Which method to choose depends on your preference and your use pattern of GAMS and Python. Note also that the bitness of your Python version has to match the bitness of the GAMS system.

## Let Python find the GAMS Python files

There are many ways for a Python installation to find locations with additional packages. Here are two:

a) Extend/set the PYTHONPATH environment variable

The environment variable PYTHONPATH (https://docs.python.org/3/using/cmdline.html#environment-variables) can be used to let Python find the GAMS API.

macOS/Linux:

export PYTHONPATH=/path/to/gams/apifiles/Python/api_38
export PYTHONPATH=/path/to/gams/apifiles/Python/gams:$PYTHONPATH  Windows: set PYTHONPATH=C:\path\to\gams\apifiles\Python\api_38 set PYTHONPATH=C:\path\to\gams\apifiles\Python\gams;%PYTHONPATH%  b) Use the site package The site package (https://docs.python.org/3/library/site.html) can be used to include the GAMS directories. For example, create a file sitecustomize.py in your site-packages directory with the following content: import sys sys.path.append(r'C:\path\to\gams\apifiles\Python\api_38') sys.path.append(r'C:\path\to\gams\apifiles\Python\gams') Change the path separator to / for macOS and Linux. You find out where your site-packages directory is located by running python -m site and inspecting the output. Method b) is preferred if creating permanent environment variables is difficult (e.g. on macOS). ## Copy the GAMS Python files to the Python installation Open a shell/command prompt and make sure your Python interpreter is found. Change directories to your GAMS system directory with the Python 3.X API files and run python setup.py install, e.g. on Windows: cd C:\path\to\gams\apifiles\Python\api_38 C:\path\to\my\python\python.exe setup.py install Attention With setuptools>=60.0.0 a breaking change has been introduced that causes the GAMS Python files to be installed incorrectly. In this case it is required to set the environment variable SETUPTOOLS_USE_DISTUTILS to stdlib before running setup.py. macOS/Linux: export SETUPTOOLS_USE_DISTUTILS=stdlib  Windows: set SETUPTOOLS_USE_DISTUTILS=stdlib  Note Per default, the setup.py script tries to create a build directory before installing its content into the site-packages directory of the Python interpreter. In case of missing write permissions in the GAMS system directory (e.g. on macOS) it is possible to specify an alternative location using the following command: C:\path\to\my\python\python.exe setup.py build -b C:\path\to\build install Note For the use of the Embedded Code Facility for Python, certain dependencies might be required (e.g. ply). The directory /path/to/gams/apifiles/Python/thirdparty contains all the third party modules used by GMSPython. The content of this directory indicates which modules and packages should be installed in addition. Running the transport1.py example: In order to see that the Python installation is set up correctly to be used with GAMS, a simple test is to run the transport1.py example. Note that you have to change the path separator to / for macOS and Linux: C:\path\to\my\python\python.exe C:\path\to\gams\apifiles\Python\trnsport1.py  ## Working with Python and multiple GAMS Installations GAMS uses SWIG (http://www.swig.org/) to connect Python to GAMS functionality implemented in dynamic link libraries or shared objects (e.g. gdxdclib64.dll to access GDX functionality). GAMS provides all files necessary to access these libraries from Python in the apifiles/Python/api_3X directories. Due to changes in the library API, SWIG version, Python version, etc, these files, that you might have copied to your Python installation in case you choose the copy method from above, are highly GAMS distribution dependent. If you work with a single GAMS installation this is very easy but if you need to work with different GAMS installations connecting your own Python installation can be a challenge. If you are in this situation we highly recommend to work with a Python installation that supports environment management, like Conda. Create for each major GAMS version a new environment and choose the copy method or the site customization (see entry b)) (e.g. via sitecustomize.py) to connect GAMS with this environment. Here is a typical session to create two environments, one for GAMS 31 and one for GAMS 32 using Conda: conda create --name gams31 python=3.8 conda activate gams31 cd /path/to/gams31/apifiles/Python/api_38 python setup.py install conda create --name gams32 --clone gams31 conda activate gams32 cd /path/to/gams32/apifiles/Python/api_38 python setup.py install ## Specifying a GAMS System Directory There are several ways to specify which system directory should be used. On all platforms, the system directory can be specified in the GamsWorkspace constructor. If no system directory is specified by the user, The API tries to find one automatically: • Windows: Try to find a system directory in the Windows registry. • Linux: Try to find a system directory in the PATH first. If none was found, search LD_LIBRARY_PATH. • OS X: Try to find a system directory in the PATH first. If none was found, search DYLD_LIBRARY_PATH. The environment variable PATH can be set as follows on Linux and macOS: export PATH=<Path/To/GAMS>:$PATH
Note
On Linux and macOS it is recommended to specify the PATH only instead of (DY)LD_LIBRARY_PATH since this might cause problems loading the correct version of certain modules (e.g. gdx).

## Troubleshooting

AttributeError: module 'gams' has no attribute 'SuffixDLVars':

Since GAMS 31, the Object-oriented Python API is distributed as source code (*.py files) instead of binaries (*.pyd files on Windows, *.so files on Linux and macOS). Installing the API with python setup.py install copies the API files into the site-packages directory of the Python interpreter, but won't overwrite or delete old *.so/*.pyd files since those are not part of the distribution anymore. This might lead to wrongly loaded files that are not compatible with each other anymore (e.g. options.py). In order to resolve the problem it is required to delete the gams directory located in the site-packages directory of the used Python installation before installing a new version of the API. Executing python -m site will reveal the location of the site-packages directory.

# Important Classes of the API

This section provides a quick overview of some fundamental classes of the GAMS Namespace. Their usage is demonstrated by an extensive set of examples in the How to use the API section.

# How to use the API

The GAMS distribution provides several examples that illustrate the usage of the Python API. [GAMSDIR]/apifiles/Python contains multiple examples dealing with the well-known transportation problem. In further course of this tutorial we discuss these examples step by step and introduce new elements of the API in detail.

We recommend to open the aforementioned files to gain a complete overview of the examples. Down below we explain the examples with the help of selected code snippets.

## How to import packages/modules from the GAMS Python API (transport1.py)

If you followed the instructions from the Getting started section and installed the GAMS Python API there should be a directory [PYTHONDIR]\Lib\site-packages\gams that contains the required packages and modules. We simply import all of them using

from gams import *

Conventional Python packages/modules can by imported like that:

import os
import sys

## How to choose the GAMS system (transport1.py)

By default the GAMS system is determined automatically. In case of having multiple GAMS systems on your machine, the desired system can be specified via an additional argument when the workspace is created. If we type python transport1.py C:/GAMS/win64/39.2 we use the 64-bit version of GAMS 39.2 to run transport1.py even if our default GAMS system might be a different one. This is managed by the following code:

...
if len(sys.argv) > 1:
ws = GamsWorkspace(system_directory = sys.argv[1])
else:
ws = GamsWorkspace()
...

Remember that the bitness of the GAMS system has to match the bitness of your Python version.

## How to export data to GDX (transport_gdx.py)

Although the Object-oriented Python API offers much more than exchanging data between Python and GDX, a common use case is the export and import of GDX files. The central class for this purpose is GamsDatabase. We assume that the data to be exported is available in Python data structures.

...
plants = [ "Seattle", "San-Diego" ]
markets = [ "New-York", "Chicago", "Topeka" ]
capacity = { "Seattle": 350.0, "San-Diego": 600.0 }
demand = { "New-York": 325.0, "Chicago": 300.0, "Topeka": 275.0 }
distance = { ("Seattle", "New-York") : 2.5,
("Seattle", "Chicago") : 1.7,
("Seattle", "Topeka") : 1.8,
("San-Diego", "New-York") : 2.5,
("San-Diego", "Chicago") : 1.8,
("San-Diego", "Topeka") : 1.4
}
...

Different GAMS symbols are represented using different Python data structures. The data for the GAMS sets is represented using Python lists of strings (e.g. plants and markets). On the other hand, GAMS parameters are represented by Python dictionaries (e.g. capacity and demand). Note that the representation of the two dimensional parameter distance uses Python tuples for storing the keys. The choice of data structures can also be different, but the used structures in this example fit well for representing GAMS data with standard Python data structures.

A new GamsDatabase instance can be created using GamsWorkspace.add_database.

...
# create new GamsDatabase instance
...

We start adding GAMS sets using the method GamsDatabase.add_set which takes the name and the dimension as arguments. The third argument is an optional explanatory text. A for-loop iterates through plants and adds new records to the recently created GamsSet instance i using GamsSet.add_record.

...
# add 1-dimensional set 'i' with explanatory text 'canning plants' to the GamsDatabase
i = db.add_set("i", 1, "canning plants")
for p in plants:
...

GamsParameter instances can be added by using the method GamsDatabase.add_parameter. It has the same signature as GamsDatabase.add_set. Anyhow, in this example we use GamsDatabase.add_parameter_dc instead which takes a list of GamsSet instances instead of the dimension for creating a parameter with domain information.

...
# add parameter 'a' with domain 'i'
a = db.add_parameter_dc("a", [i], "capacity of plant i in cases")
for p in plants:
...

As soon as all data is prepared in the GamsDatabase, the method GamsDatabase.export can be used to create a GDX file.

...
# export the GamsDatabase to a GDX file with name 'data.gdx' located in the 'working_directory' of the GamsWorkspace
db.export("data.gdx")
...

## How to import data from GDX (transport_gdx.py)

Data can be imported from a GDX file using GamsWorkspace.add_database_from_gdx. The method takes a path to a GDX file and creates a GamsDatabase instance.

...
# add a new GamsDatabase and initialize it from the GDX file just created
...

Reading the data from the GamsSet i into a list can be done as follows:

...
# read data from symbols into Python data structures
i = [ rec.keys[0] for rec in db2["i"] ]
...

A Python list is created using list comprehensions. i is retrieved by querying the GamsDatabase db2. The returned GamsSet object can be iterated using a for-loop to access the records of the set. Each record is of type GamsSetRecord and can be asked for its keys.

You can do the same for GamsParameters. Instead of creating a Python list, we want to have the data in the form of a Python dictionary. GamsParameterRecords can not only be asked for their keys, but also for their value. The following code snippet shows how to read the one dimensional parameter a into a Python dictionary using dict comprehensions.

...
a = { rec.keys[0]:rec.value for rec in db2["a"] }
...

For multi dimensional symbols, we choose the Python dictionary keys to be tuples instead of string. We access the keys as usual, but do not address a specific key. Instead, we take the whole list of keys and turn it into a tuple.

...
d = dict( (tuple(rec.keys), rec.value) for rec in db2["d"] )
...

Scalars can be read into a Python identifier by accessing the value of the first and only record.

...
f = db["f"].first_record().value
...

## How to run a GamsJob from file (transport1.py)

At first we create our workspace using ws = GamsWorkspace(). Afterwards we load the trnsport model from the GAMS model library which puts the corresponding gms file in our working directory. Apparently you can create a GamsJob with any other gms file you might have created on your own as long as it is located in the current working directory. Then the GamsJob t1 can be defined using the add_job_from_file method and afterwards we run the job.

...
ws.gamslib("trnsport")
t1.run()
...

## How to retrieve a solution from an output database (transport1.py)

The following lines create the solution output and illustrate the usage of the GamsJob.out_db property to get access to the GamsDatabase created by the run method. To retrieve the content of variable x we use squared brackets that internally call the get_symbol method.

...
for rec in t1.out_db["x"]:
print "x(" + rec.keys[0] + "," + rec.keys[1] + "): level=" + str(rec.level) + " marginal=" + str(rec.marginal)
...

Note that instead of using the squared brackets we could also use

...
for rec in t1.out_db.get_symbol("x"):
...

## How to specify the solver using GamsOptions (transport1.py)

The solver can be specified via the GamsOptions class and the GamsWorkspace.add_options method. The GamsOptions.all_model_types property sets xpress as default solver for all model types that can be handled by the solver. Then we run our GamsJob t1 with the new GamsOption.

...
opt.all_model_types = "xpress"
t1.run(opt)
...

## How to run a job with a solver option file and capture its log output (transport1.py)

At first we create the file xpress.opt with content algorithm=barrier which will be used as solver option file and is stored in the current working directory. Afterwards we use a GamsOption just like in the preceding example and set GamsOption.optfile property to 1 to tell the solver to look for a solver option file. In addition, we specify the argument output in order to write the log of the GamsJob into the file transport1_xpress.log. Providing newline="" to the open function prevents additional newline characters in the log file.

...
file = open(os.path.join(ws.working_directory, "xpress.opt"), "w")
file.write("algorithm=barrier")
file.close()
opt.optfile = 1
with open("transport1_xpress.log", "w", newline="") as log:
t1.run(opt, output=log)
...

Instead of writing the log output to a file, any object that provides the functions write and flush can be used. In order to write the log directly to stdout, we can use the following code:

...
t1.run(opt, output=sys.stdout)
...

## How to use include files (transport2.py)

In this example, as in many succeeding, the data text and the model text are separated into two different strings. Note that these strings accessed via functions get_data_text and get_model_text are using GAMS syntax. At first we write an include file tdata.gms that contains the data but not the model text and save it in our current working directory.

...
file = open(os.path.join(ws.working_directory, "tdata.gms"), "w")
file.write(get_data_text())
file.close()
...

Afterwards we create a GamsJob using the GamsWorkspace.add_job_from_string method. GamsOptions.defines is used like the 'double dash' GAMS parameters, i.e. it corresponds to --incname=tdata on the command line where incname is used as name for the include file in get_model_text as shown below.

...
opt.defines["incname"] = "tdata"
t2.run(opt)
...

The string get_model_text contains the following lines to read in the data.

...
$if not set incname$abort 'no include file name for data file provided'
$include %incname% ... ## How to read data from string and export to GDX (transport3.py) We read the data from the string accessed via function get_data_text. Note that this contains no model but only data definition in GAMS syntax. By running the corresponding GamsJob a GamsDatabase is created that is available via the GamsJob.out_db property. We can use the GamsDatabase.export method to write the content of this database to a gdx file tdata.gdx in the current working directory. ... t3 = ws.add_job_from_string(get_data_text()) t3.run() t3.out_db.export(os.path.join(ws.working_directory, "tdata.gdx")) ... ## How to run a job using data from GDX (transport3.py) This works quite similar to the usage of an include file explained in How to use include files (transport2.py). ... t3 = ws.add_job_from_string(get_model_text()) opt = ws.add_options() opt.defines["gdxincname"] = "tdata" opt.all_model_types = "xpress" t3.run(opt) ... Note that there are some minor changes in get_model_text compared to preceding examples due to the usage of a gdx instead of an include file. ...$if not set gdxincname $abort 'no include file name for data file provided'$gdxin %gdxincname%
$load i j a b d f$gdxin
...

## How to run a job using implicit database communication (transport3.py)

This example does basically the same as the two preceding examples together. We create two GamsJobs t3a and t3b where the first one contains only the data and the second one contains only the model without data. After running t3a the corresponding out_db can be read in directly just like a gdx file. Note that the database needs to be passed to the GamsJob.run method as additional argument.

...
t3a.run()
opt.defines["gdxincname"] = t3a.out_db.name
t3b.run(opt, databases=t3a.out_db)
...

## How to define data using Python data structures (transport4.py)

We use squared brackets (Python Lists) to define the sets and curly brackets (Python Dictionaries) for the parameter definition.

...
plants = [ "Seattle", "San-Diego" ]
markets = [ "New-York", "Chicago", "Topeka" ]
capacity = { "Seattle": 350.0, "San-Diego": 600.0 }
demand = { "New-York": 325.0, "Chicago": 300.0, "Topeka": 275.0 }
distance = { ("Seattle", "New-York") : 2.5,
("Seattle", "Chicago") : 1.7,
("Seattle", "Topeka") : 1.8,
("San-Diego", "New-York") : 2.5,
("San-Diego", "Chicago") : 1.8,
("San-Diego", "Topeka") : 1.4
}
...

## How to prepare a GamsDatabase from Python data structures (transport4.py)

At first we create an empty GamsDatabase db using the GamsWorkspace.add_database method. Afterwards we prepare the database. To add a set to the database we use the GamsSet class and the GamsDatabase.add_set method with arguments describing the identifier, dimension and explanatory text. To add the records to the database we iterate over the elements of our Python data structure and add them by using the GamsSet.add_record method.

For parameters the procedure is pretty much the same. Note that the table that specifies the distances in GAMS can be treated as parameter with dimension 2.

The GamsJob can be run like explained in the preceding example How to run a job using implicit database communication (transport3.py).

...
i = db.add_set("i", 1, "canning plants")
for p in plants:
j = GamsSet(db, "j", 1, "markets")
for m in markets:
a = GamsParameter(db, "a", 1, "capacity of plant i in cases")
for p in plants:
b = GamsParameter(db, "b", 1, "demand at market j in cases")
for m in markets:
d = GamsParameter(db, "d", 2, "distance in thousands of miles")
for k, v in distance.iteritems():
f = GamsParameter(db, "f", 0, "freight in dollars per case per thousand miles")
t4 = GamsJob(ws, source=get_model_text())
opt = GamsOptions(ws)
opt.defines["gdxincname"] = db.name
opt.all_model_types = "xpress"
t4.run(opt, databases = db)
...

## How to initialize a GamsCheckpoint by running a GamsJob (transport5.py)

The following lines of code conduct several operations. While the first line simply creates a GamsCheckpoint, the second one uses the GamsWorkspace.add_job_from_string method to create a GamsJob containing the model text and data but no solve statement. Afterwards the run method gets the GamsCheckpoint as argument. That means the GamsCheckpoint cp captures the state of the GamsJob.

...
t5.run(checkpoint=cp)
...

## How to initialize a GamsJob from a GamsCheckpoint (transport5.py)

Note that the string returned from function get_model_text contains the entire model and data definition plus an additional demand multiplier and scalars for model and solve status but no solve statement:

...
Scalar bmult demand multiplier /1/;
...
demand(j) .. sum(i, x(i,j)) =g= bmult*b(j) ;
...
Scalar ms 'model status', ss 'solve status';
...

In transport5.py we create a list with eight different values for this demand multiplier.

...
bmultlist = [ 0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.3 ]
...

For each entry of that list we create a GamsJob t5 using the GamsWorkspace.add_job_from_string method. Besides another string which resets the demand multiplier bmult, specifies the solve statement and assigns values to the scalars ms and ss we pass the checkpoint cp as additional argument. This results in a GamsJob combined from the checkpoint plus the content provided by the string.

We run the GamsJob and print some interesting data from the out_db.

...
for b in bmultlist:
t5 = ws.add_job_from_string("bmult=" + str(b) + "; solve transport min z use lp; ms=transport.modelstat; ss=transport.solvestat;", cp)
t5.run()
print "Scenario bmult=" + str(b) + ":"
print " Modelstatus: " + str(t5.out_db["ms"].find_record().value)
print " Solvestatus: " + str(t5.out_db["ss"].find_record().value)
print " Obj: " + str(t5.out_db["z"].find_record().level)
...

NOTE: Some of the demand multipliers cause infeasibility. Nevertheless, GAMS keeps the incumbent objective function value. Therefore the model status and the solve status provide important information for a correct solution interpretation.

## How to run multiple GamsJobs in parallel using a GamsCheckpoint (transport6.py)

This example illustrates how to run the jobs known from transport5.py in parallel. We initialize the GamsCheckpoint cp and introduce a demand multiplier as we did before :

...
t6.run(checkpoint=cp)
bmultlist = [ 0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.3 ]
...

Furthermore, we introduce a lock object io_lock that will be used to avoid mixed up output from the parallel jobs. We create one scenario for each entry of bmultlist and cause a thread to begin execution.

...
for b in bmultlist:
for b in bmultlist:
...

In function run_scenario a GamsJob t6 is created and run just like in the preceding example of transport5.py. The output section is also the same except for the fact that it is 'locked' by the object io_lock which means that the output section cannot be executed simultaneously for multiple demand multipliers.

...
def run_scenario(workspace, checkpoint, io_lock, b):
t6 = workspace.add_job_from_string("bmult=" + str(b) + "; solve transport min z use lp; ms=transport.modelstat; ss=transport.solvestat;", checkpoint)
t6.run()
# we need to make the ouput a critical section to avoid messed up report informations
io_lock.acquire()
print "Scenario bmult=" + str(b) + ":"
print " Modelstatus: " + str(t6.out_db["ms"][()].value)
print " Solvestatus: " + str(t6.out_db["ss"][()].value)
print " Obj: " + str(t6.out_db["z"][()].level)
io_lock.release()
...

While the output in transport5.py is strictly ordered subject to the order of the elements of bmultlist in transport6.py the output blocks might change their order but the blocks describing one scenario are still appearing together due to the io_lock object.

## How to create a GamsModelInstance from a GamsCheckpoint (transport7.py)

In transport7.py the usage of GamsModelInstance is demonstrated.

At first checkpoint cp is created as in the preceding examples. Note that the GamsJob t7 again contains no solve statement and the demand multiplier is already included with default value 1. We create the GamsModelInstance mi using the GamsCheckpoint.add_modelinstance method.

...
t7.run(checkpoint=cp)
...

## How to modify a parameter of a GamsModelInstance using GamsModifier (transport7.py)

A GamsModelInstance uses a sync_db to maintain the data. We define bmult as GamsParameter using the GamsDatabase.add_parameter method and specify gurobi as solver. Afterwards the GamsModelInstance is instantiated with 3 arguments, the solve statement, GamsModifier bmult and GamsOptions opt. The GamsModifier means that bmult is modifiable while all other parameters, variables and equations of GamsModelInstance mi stay unchanged. We use the GamsParameter.add_record method to assign a value to bmult. That value can be varied afterwards using the GamsParameter.first_record method to reproduce our well-known example with different demand multipliers.

...
bmult = mi.sync_db.add_parameter("bmult", 0, "demand multiplier")
opt.all_model_types = "gurobi"
mi.instantiate("transport use lp min z", GamsModifier(bmult), opt)
bmultlist = [ 0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.3 ]
for b in bmultlist:
bmult.first_record().value = b
mi.solve()
print "Scenario bmult=" + str(b) + ":"
print " Modelstatus: " + str(mi.model_status)
print " Solvestatus: " + str(mi.solver_status)
print " Obj: " + str(mi.sync_db.get_variable("z").find_record().level)
...

## How to modify a variable of a GamsModelInstance using GamsModifier (transport7.py)

We create a GamsModelInstance using the GamsCheckpoint.add_modelinstance method. Afterwards we define x as GamsVariable and a GamsParameter xup that will be used as upper bound for x. At the following instantiate method GamsModifier has 3 arguments. The first one says that x is modifieable, the second determines which part of the variable (lower bound, upper bound or level) can be modified and the third specifies the GamsParameter that holds the new value, in this case xup.

In the following loops we set the upper bound of one link of the network to zero, which means that no transportation between the corresponding plant and market is possible, and solve the modified transportation problem.

...
x = mi.sync_db.add_variable("x", 2, VarType.Positive)
xup = mi.sync_db.add_parameter("xup", 2, "upper bound on x")
mi.instantiate("transport use lp min z", GamsModifier(x, UpdateAction.Upper, xup))
mi.solve()
for i in t7.out_db["i"]:
for j in t7.out_db["j"]:
xup.clear()
mi.solve()
print "Scenario link blocked: " + i.keys[0] + " - " + j.keys[0]
print " Modelstatus: " + str(mi.model_status)
print " Solvestatus: " + str(mi.solver_status)
print " Obj: " + str(mi.sync_db["z"].find_record().level)
...

## How to use a queue to solve multiple GamsModelInstances in parallel (transport8.py)

We initialize a GamsCheckpoint cp from a GamsJob. Then we define a list that represents the different values of the demand multiplier. That list will be used like a queue where we extract the last element first. The objects list_lock and io_lock are used later to avoid multiple reading of one demand multiplier and messed up output. Then we call function scen_solve multiple times in parallel. The number of parallel calls is specified by nr_workers.

...
t8.run(checkpoint=cp)
bmult_list = [ 1.3, 1.2, 1.1, 1.0, 0.9, 0.8, 0.7, 0.6 ]
# start 2 threads
nr_workers = 2
for i in range(nr_workers):
for i in range(nr_workers):

In function scen_solve we create and instantiate a GamsModelInstance as in the preceding examples and make parameter bmult modifiable. Note that we choose cplex as solver because it is thread safe (gurobi would also be possible).

We have two critical sections that are locked by the objects list_lock and io_lock. Note that the pop method removes and returns the last element from the list and deletes it. Once the list is empty the loop terminates.

...
def scen_solve(workspace, checkpoint, bmult_list, list_lock, io_lock):
bmult = mi.sync_db.add_parameter("bmult", 0, "demand multiplier")
opt.all_model_types = "cplex"
# instantiate the GamsModelInstance and pass a model definition and GamsModifier to declare bmult mutable
mi.instantiate("transport use lp min z", GamsModifier(bmult), opt)
while True:
# dynamically get a bmult value from the queue instead of passing it to the different threads at creation time
list_lock.acquire()
if 0 == len(bmult_list):
list_lock.release()
return
b = bmult_list.pop()
list_lock.release()
bmult.first_record().value = b
mi.solve()
# we need to make the ouput a critical section to avoid messed up report informations
io_lock.acquire()
print "Scenario bmult=" + str(b) + ":"
print " Modelstatus: " + str(mi.model_status)
print " Solvestatus: " + str(mi.solver_status)
print " Obj: " + str(mi.sync_db.get_variable("z").find_record().level)
io_lock.release()
...

## How to fill a GamsDatabase by reading from MS Access (transport9.py)

This example illustrates how to import data from Microsoft Access to a GamsDatabase. There are a few prerequisites required to run transport9.py successfully.

• We import pyodbc which can be downloaded from http://code.google.com/p/pyodbc/.
• Note that an architecture mismatch might cause problems. The bitness of your MS Access, Python, pyodbc and GAMS should be identical.

We call a function read_data_from_access that finally returns a GamsDatabase as shown below.

...
...

The function read_from_access begins with the creation of an empty database. Afterwards we set up a connection to the MS Access database transport.accdb which can be found in [GAMSDIR]\apifiles\Data. To finally read in GAMS sets and parameters we call the functions read_set and read_parameter that are explained down below.

...
# connect to database
str_access_conn = 'DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};DBQ=..\\Data\\transport.accdb'
try:
connection = pyodbc.connect(str_access_conn)
except Exception as ex:
print "Error: Failed to create a database connection. \n {0}".format(ex)
sys.exit(1)
# read GAMS sets
read_set(connection, db, "SELECT Plant FROM plant", "i", 1, "canning plants")
read_set(connection, db, "SELECT Market FROM Market", "j", 1, "markets")
# read GAMS parameters
read_parameter(connection, db, "SELECT Plant,Capacity FROM Plant", "a", 1, "capacity of plant i in cases")
read_parameter(connection, db, "SELECT Market,Demand FROM Market", "b", 1, "demand at market j in cases")
read_parameter(connection, db, "SELECT Plant,Market,Distance FROM Distance", "d", 2, "distance in thousands of miles")
connection.close()
return db
...

The function read_set adds a set to the GamsDatabase that is filled with the data from the MS Access file afterwards. The function read_parameter works quite similar.

...
def read_set(connection, db, query_string, set_name, set_dim, set_exp=""):
try:
cursor = connection.cursor()
cursor.execute(query_string)
data = cursor.fetchall()
if len(data[0]) != set_dim:
print "Number of fields in select statement does not match setDim"
sys.exit(1)
i = db.add_set(set_name, set_dim, set_exp)
for row in data:
keys = []
for key in row:
keys.append(str(key))
except Exception as ex:
print "Error: Failed to retrieve the required data from the DataBase.\n{0}".format(ex)
sys.exit(1)
finally:
cursor.close()
...

Once we read in all the data we can create a GamsJob from the GamsDatabase and run it as usual.

## How to fill a GamsDatabase by reading from MS Excel (transport10.py)

This example illustrates how to read data from Excel, or to be more specific, from [GAMSDIR]\apifiles\Data\transport.xlsx.

At first you have to download an additional package from pypi, e.g. to C:\Users\[USERNAME]\Downloads. Unzip the package and run the following code from command line:

cd C:\Users\[Username]\Downloads\xlrd-0.9.2 && python setup.py install && cd [GAMSDIR]\apifiles\Python

Now you should be able to find the directory xlrd in [PYTHONDIR]\Lib\site-packages and to run transport10.py.

In transport10.py the model is given as string without data like in many examples before and the Excel file transport.xlsx is located at [GAMSDIR]\apifiles\Data. At first we define the workbook to read from and the different sheet names. To ensure to have the same number of markets and plants in all spreadsheets, we conduct a little test that checks for the number of rows and columns. Our workspace is only created if this test yields no errors.

...
wb = open_workbook("..\\Data\\transport.xlsx")
capacity = wb.sheet_by_name("capacity")
demand = wb.sheet_by_name("demand")
distance = wb.sheet_by_name("distance")
# number of markets/plants have to be the same in all spreadsheets
assert (distance.ncols-1 == demand.ncols) and (distance.nrows-1 == capacity.ncols), \
"Size of the spreadsheets doesn't match"
...

Now we can create a GamsDatabase and read in the data contained in the different worksheets. We iterate over the columns and read in the set names and the corresponding parameter values.

...
i = db.add_set("i", 1, "Plants")
j = db.add_set("j", 1, "Markets")
capacity_param = db.add_parameter("a", 1, "Capacity")
demand_param = db.add_parameter("b", 1, "Demand")
distance_param = db.add_parameter("d", 2, "Distance")
for cx in range(capacity.ncols):
capacity_param.add_record(str(capacity.cell_value(rowx=0, colx=cx))).value = capacity.cell_value(rowx=1, colx=cx)
for cx in range(demand.ncols):
demand_param.add_record(str(demand.cell_value(rowx=0, colx=cx))).value = demand.cell_value(rowx=1, colx=cx)
for cx in range(1, distance.ncols):
for rx in range(1, distance.nrows):
keys = ( str(distance.cell_value(rowx=rx, colx=0)), str(distance.cell_value(rowx=0, colx=cx)) )
distance_param.add_record(keys).value = distance.cell_value(rowx=rx, colx=cx)
...

Note that we can name sets and parameters just like in the database but we don't have to. Now we can run our GamsJob as usual.

...
opt.defines["gdxincname"] = db.name
opt.all_model_types = "xpress"
t10.run(opt, databases=db)
for rec in t10.out_db["x"]:
print "x(" + rec.keys[0] + "," + rec.keys[1] + "): level=" + str(rec.level) + " marginal=" + str(rec.marginal)
...

## How to create and use a save/restart file (transport11.py)

In transport11.py we demonstrate how to create and use a save/restart file. Usually such a file should be supplied by an application provider but in this example we create one for demonstration purpose. Note that the restart is launched from a GamsCheckpoint.

We create a directory tmp with internal identifier w_dir in the directory we are currently in (usually [GAMSDIR]\apifiles\Python). This file will be used as working directory later. From the main function we call the function create_save_restart giving it directory tmp and the desired name for the save/restart file (tbase) as arguments.

...
w_dir = os.path.join(".", "tmp")
create_save_restart(os.path.join(w_dir, "tbase"));
...

In function create_save_restart we create a workspace with the given working directory (w_dir refers to tmp). Then we create a GamsJob from a string. Note that the string given via get_base_model_text contains the basic definitions of sets without giving them a content (that is what \$onempty is used for). Afterwards we specify a GamsOption to only compile the job but do not execute it. Then we create a checkpoint cp that is initialized by the following run of the GamsJob and stored in the file given as argument to the function, in our case tbase. This becomes possible because the add_checkpoint method accepts identifiers as well as file names as argument.

...
def create_save_restart(cp_file_name):
if len(sys.argv) > 1:
ws = GamsWorkspace(os.path.dirname(cp_file_name), sys.argv[1])
else:
ws = GamsWorkspace(os.path.dirname(cp_file_name))
opt.action = Action.CompileOnly
j1.run(opt, cp)
del opt
...

So what you should keep in mind before we return to further explanations of the main function is, that the file tbase is now in the current working directory and contains a checkpoint that will work exactly like a restart file.

In the main function we define some data using Python data structures as we already did in transport4.py before we create the GamsWorkspace and a GamsDatabase.

...
if len(sys.argv) > 1:
ws = GamsWorkspace(w_dir, sys.argv[1])
else:
ws = GamsWorkspace(w_dir)
...

Afterwards we set up the GamsDatabase like we already did in transport4.py. Once this is done we run a GamsJob using this data plus the checkpoint stored in file tbase.

...
t4 = ws.add_job_from_string(get_model_text(), cp_base)
opt.defines["gdxincname"] = db.name
opt.all_model_types = "xpress"
t4.run(opt, databases=db)
...

Note that the string from which we create job t11 is different to the one used to prepare the checkpoint stored in tbase and is only responsible for reading in the data from the GamsDatabase correctly. The entire model definition is delivered by the checkpoint cp_base which is equal to the one we saved in tbase.

ws
db
def get_model_text()
cp_base
b
t4
opt
GAMS Development Corp.
GAMS Software GmbH

General Information and Sales
U.S. (+1) 202 342-0180
Europe: (+49) 221 949-9170