### Table of Contents

This document briefly describes GDXMRW (GDX-Matlab Read/Write), a suite of utilities to exchange data between GAMS and MATLAB. The software gives MATLAB users the ability to use all the optimization capabilities of GAMS, and allows visualization of GAMS models directly within MATLAB. The most recent version of GDXMRW is included as part of the current GAMS Distribution.

- Date
- August 1, 2014

# Introduction

Optimization is becoming widely used in many application areas as can be evidenced by its appearance in software packages such as Excel and MATLAB. While the optimization tools in these packages are useful for small-scale nonlinear models (and to some extent for large linear models), the lack of a capability to compute automatic derivatives makes them impractical for large scale nonlinear optimization. In sharp contrast, modeling languages such as GAMS and AMPL have had such a capability for many years, and have been used in many practical large scale nonlinear applications.

On the other hand, while modeling languages have some capabilities for data manipulation and visualization (e.g., Rutherford's GNUPLOT), specialized software tools like Excel and MATLAB are much better at these tasks.

This paper describes a link between GAMS and MATLAB. The aim of this link is two-fold. Firstly, it is intended to provide MATLAB users with a sophisticated nonlinear optimization capability. Secondly, the visualization tools of MATLAB are made available to a GAMS modeler in an easy and extendable manner so that optimization results can be viewed using any of the wide variety of plots and imaging capabilities that exist in MATLAB.

In order to enable this link between GAMS and MATLAB, we have implemented MATLAB callable functions that can efficiently import and export data to and from GAMS through GDX files. The simplest read and write functions, `irgdx`

and `iwgdx`

, deal with indexed parameters. Without getting into their precise meaning, these parameters are essentially indexed in each dimension by the sequence of integers from 1 to the size of that dimension. As an example of the read function, the following command in MATLAB will store the matrices `A`

and `B`

into the caller workspace after reading from the file `foo1.gdx`

:

>> irgdx('foo1','A','B');

Similarly, the call

>> iwgdx('foo2','C','D');

would write the matrices `C`

and `D`

(located in the MATLAB workspace) as indexed parameters into a file named `foo2.gdx`

. Note that, many of the examples found in this manual are also part of the GAMS data utilities models library, referred to here as **datalib**. The statements above can be found in datalib example `gdxmrw_intro02_init`

. For reading and writing more complex data such as sets, variables, equations, and non-indexed parameters, we can use the more sophisticated functions `rgdx`

and `wdgx`

. Further descriptions of these functions together with advanced use of `irgdx`

and `iwgdx`

are detailed in the rest of this paper.

In Section Data Transfer, we discuss the data transfer utilities that allow importing and exporting data between MATLAB and GDX files: `irgdx`

, `iwgdx`

, `rgdx`

, `wgdx`

. In Section Extended use, we give a few examples of the MATLAB and GAMS interface. In APPENDIX A - Configuring GDXMRW, we provide information about configuring GDXMRW and testing the utilities. In APPENDIX B - Utility functions: gdxWhos and gdxInfo, we describe additional utility functions, `gdxWhos`

and `gdxInfo`

, that allow viewing contents of GDX files in the MATLAB console. Finally, in APPENDIX C - Calling GAMS model from MATLAB, we discuss the `gams`

function that with a single call initializes a GAMS model with MATLAB data, executes GAMS on the model, and returns results back into MATLAB.

# Data Transfer

This paper describes a suite of tools for exchanging data between GAMS and MATLAB. This data exchange is accomplished via the GDX (i.e. the GAMS Data eXchange) interface and API. There are many advantages to using GDX, including platform independence, space and time efficiency in storing and accessing data, and a guarantee that all GDX data contains no duplicates and is free from any logical or syntax errors that might prevent it from being read into GAMS. The GDX interface is well tested, available to the public, and is the basis for most if not all of the GAMS data utilities.

In this section we discuss four MATLAB routines. The first two, `irgdx`

and `iwgdx`

, are the subject of sections irgdx and iwgdx and are used to quickly and simply read indexed parameters from a GDX file into MATLAB and vice versa. For an example use of these functions, a generic quadratic program function that mimics MATLAB's `quadprog`

function is provided in datalib example `gdxmrw_qp3`

. The `rgdx`

and `wgdx`

routines (sections rgdx and wgdx) are more sophisticated versions of the first two that can read and write more general GDX data, e.g. sets, variables, equations, and non-indexed parameters. To understand the structure of this GDX data, the material in section Indexing with labels (UELs) is essential.

## irgdx

`irgdx`

is a specialized MATLAB function to do an efficient import of an indexed parameter from a GDX file. The `irgdx`

call will not read non-indexed parameters; it can only read data recognized as an indexed data type. If a GDX file consists of both indexed and non-indexed data, `irgdx`

can still read the indexed data. To see whether particular parameters in a GDX file are indexed, the `gdxWhos`

function described in APPENDIX B - Utility functions: gdxWhos and gdxInfo can be used.

The syntax and functionality of `irgdx`

closely resembles that of the MATLAB `load`

function to import MATLAB formatted data, i.e data stored in MAT files. In Basic syntax and Load symbols into output structure we describe the basic modes of operation where the data read from GDX are either stored in the caller workspace or into a MATLAB structure on the left-hand side of an assignment. In MATLAB sparse form, we describe an optional specification that allows read results to be stored in MATLAB sparse form. Finally, in Subsection Renaming, we describe a simple syntax that allows the renaming of variables/parameters without the need to make expensive copies.

### Basic syntax

The basic syntax for `irgdx`

is

irgdx('gdxFileName', 'sym1', 'sym2', ...);

The above call will read the indexed parameters named sym1, sym2, and so on from the GDX file whose name is specified in the first argument and store the results in the MATLAB caller workspace. Only the GDX name argument is mandatory. If only this argument is specified (i.e. no symbol names are provided) then *all* of the indexed parameters stored in the GDX file will be loaded into the workspace. The GDX name can be specified with or without the `′.gdx′`

extension. Note that all of the input arguments must be specified in string form.

As an example of the above syntax, we can read the scalar a0, the vector a1, and the matrix a2 from the file `idx1_.gdx`

with the following command:

>> irgdx('idx1_','a0','a1','a2');

The three symbols are stored as MATLAB variables a0, a1, and a2 in the caller workspace. If we want to read all the symbols from the file `idx1_.gdx`

, we simply call `irgdx`

with only the file name:

>> irgdx('idx1_');

### Load symbols into output structure

An alternative to storing the results in the MATLAB caller workspace is to return the results in a MATLAB structure. Here, the syntax is:

s = irgdx('gdxFileName','sym1','sym2',...);

This call will return the indexed parameters named `sym1`

, `sym2`

, and so on in the structure s, with fields `s.sym1`

, `s.sym2`

, and so on containing the values of the respective indexed parameters. The input arguments must all be in string form. Only the GDX name is mandatory; if only this argument is specified, all of the indexed parameters stored in the GDX file will be returned in the output structure. The GDX name can be specified with or without the `.gdx`

extension.

For example, the following call returns the values of parameter a2 in the structure s:

>> s = irgdx('idx1_','a2'); >> s.a2 ans = 50 50 50 50

To return all the indexed parameters in the fields of the structure `s`

, we would simply call `irgdx`

with only the GDX name:

>> s = irgdx('idx1_');

See datalib example `gdxmrw_irgdx01_init`

.

### MATLAB sparse form

By default, `irgdx`

will store results in MATLAB dense form. However, it is possible to use MATLAB sparse form for parameters with dimension less than or equal to 2; MATLAB's sparse storage scheme cannot handle matrices with more than 2 dimensions. Using sparse storage will allow for efficient import of especially large sparse indexed parameters. Note that in this context *sparse* does not refer to data represented in the form [i,j,...,val] but rather to MATLAB's internal sparse storage scheme. To store a particular symbol in MATLAB sparse form, one can simply append ':s' to that symbol's name in the `irgdx`

call. For example, the following call will store the parameter a2 into the caller's workspace in sparse form:

>> irgdx('idx1_','a2:s');

Note that if we want to store *all* the indexed parameters from a specified file in MATLAB sparse form, we would need to specify each symbol individually in order to append the ':s' to each symbol's name.

### Renaming

The details of this section are rather involved; it can be skipped without loss of continuity.

While an `irgdx`

call will typically store results in the caller workspace or create field names in the output structure using the symbol names from the GDX file, it is possible to rename the outputs in MATLAB without making potentially expensive copies. To rename a symbol, we can take advantage of MATLAB's default copy-on-write mechanism: when an assignment occurs, the values assigned are not actually copied until one of the values (the original or the copy) is actually changed. For example, after executing the following two lines of code, the variable `a2_new`

points to the same memory location as a2 and is *not* a separate copy of a2:

>> irgdx('idx1_','a2'); >> a2_new = a2;

If however the value of `a2_new`

were changed after executing the above lines, MATLAB would then need to make a separate copy, at the cost of memory and time. Now, the command

>> clear a2;

immediately following the above code would complete the rename of `a2`

to `a2_new`

as desired. Once `a2`

is cleared, we can change `a2_new`

without needing to make a copy. Similarly, we can do a rename when we have an output structure:

>> s = irgdx('idx1_','a2'); >> s.a2_new = s.a2; >> s = rmfield(s,'a2');

To convince ourselves that this code indeed does not require a copy, we can run the commands in the debug output format. That is, immediately after the assignment of `s.a2`

to `s.a2_new`

in the above code (before using the rmfield function), we can run the following commands:

>> format debug; >> s.a2 ans = Structure address = 8c1eda0 m = 2 n = 2 pr = 75fa88a0 pi = 0 50 50 50 50 >> s.a2_new ans = Structure address = 8c1eda0 m = 2 n = 2 pr = 75fa88a0 pi = 0 50 50 50 50

Note that the pointer pr is the same for both outputs, implying that MATLAB does not make a separate copy prior to a write. A similar test can also be done when parameters are stored in workspace variables rather than a structure.

## iwgdx

`iwgdx`

is a function that creates a GDX file of indexed parameters from MATLAB data. Since `iwgdx`

writes only indexed parameters to GDX, the input MATLAB data is simplified: no labels are required or implied. The input matrices can be stored in the usual (dense) scheme or using the MATLAB sparse scheme: `iwgdx`

detects and handles the two cases automatically. Note that matrices represented in `[i,j,...,val]`

form are not acceptable as `iwgdx`

input.

While the `iwgdx`

function is patterned after the `save`

function in MATLAB, there are important differences. Firstly, if `only`

the file name is specified, the `save`

function stores all variables in the caller workspace, while `iwgdx`

creates an empty GDX file. The behavior of `iwgdx`

in this case also differs from that of `irgdx`

, which reads all parameters when only the file name is specified. Secondly, `iwgdx`

has an additional pass by reference and renaming syntax, in which the parameter string name and values are passed as two consecutive arguments. In Subsection Basic syntax, we describe the basic syntax for `iwgdx`

. In Subsection Pass by reference syntax , we introduce the pass by reference and renaming syntax. Finally, in Subsection Dimensionality specification, we describe an optional dimensionality specification that allows symbols to be written with a larger dimensionality than the default.

### Basic syntax

The basic syntax for `iwgdx`

is:

iwgdx('fileName','sym1','sym2',...);

The above call will write the values of the variables sym1, sym2, and so on in the MATLAB caller workspace into a GDX file with the name given by the first input argument. The file name can be specified with or without the '.gdx' extension. If a GDX file with the same name already exists, it will be overwritten. Note that a call with a single input argument (the GDX name) will raise a warning and will create an empty GDX file. This contrasts to the MATLAB save function, which writes all of the workspace variables when none are specified. In the basic form described here, all of the `iwgdx`

input arguments are strings.

To illustrate, suppose the matrices a2 and a3 exist in the MATLAB caller workspace. We can write the results of these matrices into a file test.gdx with the following call:

>> iwgdx('test','a2','a3');

If the file test.gdx already exists, it will be overwritten. Specifying only the file name will create an empty GDX file:

>> iwgdx('test'); Warning: an empty gdx file was created.

### Pass by reference syntax

When using the basic `iwgdx`

syntax, a copy of the workspace variable to be written to GDX is made as part of the call. This is unavoidable: the MATLAB executable (MEX) interface only provides functions that return *copies* of workspace variables, most likely to avoid overwriting or corrupting data. However, an alternative, more efficient approach is possible by passing the numeric data as an argument. Essentially, the pointer to the data values are passed by reference and no copy is made. This could be especially useful when writing a large amount of data. An additional benefit of this syntax is the ability to write GDX parameters with different names than used in the MATLAB caller workspace. For example:

>> iwgdx('test','s1',s1Mat,'s2',s2Mat);

In the above command, `s1`

and `s2`

are strings holding the GDX parameter names, while `s1Mat`

and `s2Mat`

are matrices containing the values to store. We can also use a mix of the basic syntax with the pass by reference syntax. For example:

>> iwgdx('test','anew',a,'b','c',c);

This call would save matrix `a`

as anew in GDX without making a copy, would internally make a copy of `b`

while saving `b`

to GDX, and would save `c`

to GDX without renaming or making a copy.

We can do a timing test to see the performance difference between the basic syntax and the pass by reference syntax. The following commands create a 5000x5000 dense matrix consisting of random values in the workspace.

>> clear; >> randMat = rand(5000);

First, we do a timing test using the basic syntax:

>> delete test.gdx; % delete file if it exists >> tic; >> iwgdx('test','randMat'); >> t1 = toc t1 = 5.3346

Now, we do a timing test using the pass by reference syntax:

>> delete test.gdx; % delete file if it exists >> tic; >> iwgdx('test','randMat',randMat); >> t2 = toc t2 = 5.1636

For the above example, the improvement in time performance is about 3%. Of course, the pass by reference call will also use only the half the memory compared to the basic call. We can expect further improvement in performance time and memory use for even larger data.

### Dimensionality specification

It is possible to specify the number of dimensions (i.e. dimensionality) for the symbol written to GDX by appending `:n`

to the variable/symbol name, where `n`

is the desired dimensionality of the GDX symbol. This feature is motivated by two quirks of the MATLAB environment. Firstly, all scalars and vectors are stored in MATLAB as 2-dimensional matrices. For example:

>> s0 = 100 >> s1 = [ 100 ]

results in two variables with identical storage schemes: 1x1 matrices. In the MATLAB environment this isn't a problem: things behave as you would expect them to, and the `isscalar`

, `isvector`

, and `ismatrix`

calls are available to interrogate MATLAB about how it views variables. However, in GDX there is a distinction between a scalar (something with 0 dimensions), a parameter with 1 dimension and length 1, and a 1x1 parameter. We need to be able to create GDX files containing any or all of these.

Secondly, MATLAB "flattens" all variables by removing trailing dimensions whose size is 1. For example, after executing

>> B = ones(2,2,2) >> C = ones(2,2,2,1,1)

`B`

and `C`

will be identical 3-dimensional MATLAB arrays, yet we need to be able to create a 2x2x2x1x1 indexed parameter in GDX.

By default, `iwgdx`

creates GDX parameters that are consistent with the MATLAB view of the data passed in. For example, executing

>> d0 = ones(1,1) >> d1 = ones(3,1) >> d2 = ones(3,3) >> iwgdx('dd','d0','d1','d2')

results in indexed parameters with 0, 1, and 2 dimensions, respectively, being written to GDX. However, if we want to write d0 as a vector or matrix or higher-dimensional array, we need to use the optional syntax to specify the dimensionality of the resulting symbol in GDX. For example:

>> d0 = pi; >> p1 = d0; >> d1 = pi*ones(3,1); >> p2 = d1; >> d2 = pi*ones(3,3,1) >> iwgdx('pp','d0','p1:1','d1','p2:2','d2','p3:3',d2)

will write the following symbols to GDX:

- The 0-dimensional symbol
`d0`

. This is the default behavior, consistent with the MATLAB view that`d0`

is a scalar. - The 1-dimensional symbol
`p1`

. The scalar variable`p1`

was promoted to a 1-dimensional symbol by adding an additional singleton dimension. - The 1-dimensional symbol
`d1`

. This is the default behavior, consistent with the MATLAB view that`d1`

is a vector. - The 2-dimensional symbol
`p2`

. The vector variable`p2`

was promoted to a 2-dimensional symbol by adding an additional singleton dimension. - The 2-dimensional symbol
`d2`

. This is the default behavior, consistent with the MATLAB view that`d2`

is a matrix, i.e. the final singleton dimension is just removed. - The 3-dimensional symbol
`p3`

. The matrix variable`d2`

was promoted to a 3-dimensional symbol by adding an additional singleton dimension. Independently, call-by-reference was used to rename`d2`

to`p3`

.

Note that the dimensionality specified must always be a promotion, i.e. the dimensionality of the resulting GDX symbol is only increased from what it would be by default.

See datalib example `gdxmrw_iwgdx01_init`

.

## Indexing with labels (UELs)

To this point we have been looking at functions to read (`irgdx`

, Section irgdx) and write (`iwgdx`

, Section iwgdx) *indexed* parameters from and to GDX. Indexed parameters are convenient since their structure is essentially identical to that of MATLAB matrices. This structure can be encapsulated very simply as the number of dimensions and the size or extent of each dimension. Using indexed parameters, we do not need to be very concerned with the structure of the data since little structure exists. However, there is much more to GDX data than indexed parameters. If we want to read and write GDX data in more generality we will need to understand how labels or strings are used to reference GDX data and how these labels are organized within GDX.

In general, GAMS data is referenced with labels instead of with numbers, so that one references `demand(′chicago′)`

instead of `demand(2)`

in a GAMS model. These labels are also called Unique Element Labels, or UELs. The collection of UELs used in a model or in a GDX file is ordered internally and often referred to as the *universe* of UELs. For efficiency, it is not necessary to use labels internally in GDX or when using the GDX API. Instead, a correspondence or mapping between integers and labels is established initially and/or built up as labels are introduced, so that integers can be used in place of the UELs that reference the data. It is important to keep in mind that the labels are the key thing in referencing GDX data: when integer maps are used the integers are only used to efficiently represent the labels.

A similar scheme mapping integers to labels is used in the `rgdx`

and `wgdx`

routines for reading and writing general GDX data. In addition to the actual data values being stored, there will be a mapping passed to allow integers to be mapped to labels and vice versa. In some cases, where no mapping is passed and data are written to GDX, a default mapping (`1`

to `′1′`

, `2`

to `′2′`

, etc.) may be used. When reading, the default mapping to use when passing back data is the universe mapping from the GDX, but it is possible to apply a filter. A filter can reduce the amount of data returned and also change the order of that data.

## rgdx

`rgdx`

is a MATLAB utility to import data from a GDX file. It takes structural input and returns data back in the form of a structure. This is a very flexible routine as it gives user control over the output data structure. `rgdx`

can read a set/parameter/equation/variable from a GDX file and display results in either full/dense or sparse form. A user can also perform a filtered read to read only certain specific elements of a symbol. It can also perform compression to remove extra zeros.

This routine can take up to two arguments. The first argument is a string input containing the GDX file name. It can be with or without the `′.gdx′`

file extension. If you call this routine with only the GDX file name as an argument then the `′uels′`

field of output structure will be the global UEL of the GDX file and the rest of the fields of the output structure will be NULL. The second argument is a structure input containing information regarding the desired symbol. The syntax for this call looks like this:

x = rgdx('fileName', structure);

As an example, we read a 3D parameter, `′test3′`

from `′sample.gdx′`

. Here we display this parameter in full format but without redundant zeros:

>> s.name = 'test3'; >> s.form = 'full'; >> s.compress = true; >> x = rgdx('sample', s) x = name: 'test3' type: 'parameter' dim: 3 val: [4x2x2 double] form: 'full' uels: {{1x4 cell} {1x2 cell} {1x2 cell}} >> x.val ans(:,:,1) = 3 4 4 5 5 6 6 7 ans(:,:,2) = 4 5 5 6 6 7 7 8 >> x.uels{1} ans = '1' '2' '3' '4' >> x.uels{2} ans = 'j1' 'j2' >> x.uels{3} ans = 'k1' 'k2'

In the following subsections we will explain the input and output structures. Please note that except for the `′name′`

and `′uels′`

fields, all other string fields take case insensitive input. All boolean fields can also be entered as string values as well.

### Input structure

To read a symbol from a GDX file we just need to know its name in string format. Thus, the only mandatory field of the input structure is 'name'. e.g.

>> s.name = 'test3';

There are several other optional fields of the input structure that give user more control over the output structure. These optional fields are as follows:

- form

This field represents the form of the output data. Output data can be either in`′full′`

or`′dense′`

form or it can be in`[i, j,.., val]`

sparse form. We will label`[i, j,.., val]`

as 'sparse'. A user can enter it as string input with value`′full′`

or`′sparse′`

. e.g.>> s.form = 'full';

By default the data will be in 'sparse' format. Note that this sparse format differs from MATLAB's internal sparse storage scheme that we referred to in prior sections for the`irgdx`

and`iwgdx`

functions. - compress

By default the uels in the output structure will be a global UEL of the GDX file and the`′val′`

field data will be indexed tothis UEL. The`rgdx`

routine allows a user to remove rows and columns with all zeros from the 'val' data matrix and re-indexes the uels accordingly. This is called compression of the data. This can be achieved by setting compress as true in the input structure. Valid values for this field are true and false, either in logical form or in string form. e.g.>> s.compress = 'true';

However, we note that compressing the data can be dangerous because the size of the matrix that is read can be incorrect. Essentially, all zero rows and columns are removed, including those that might actually be part of the data values in the symbol matrix. - uels

This input field is used to perform a filtered read, i.e. output data matrix will contain values only corresponding to the entered uels. Filtered read is very useful if user just wants certain specific set of data. Uels should be entered in cell array form. It has to be in 1xN form with each column being a cell array representing the uels for that dimension. Each column can have strings, doubles or combinations of both. It also allows a user to enter double data in shorthand notation or a 1 x N matrix. For example, in the previous example we can perform a filtered read to get data corresponding to only the`′1&prime`

,`′3′`

elements of the first index of the parameter`′test3′`

. The following function is handy when one needs to generate a UEL listing for the input structure:>> guel = @(s,v) strcat(s,strsplit(num2str(v)));

Using the above function, we can create a listing of strings consisting of the string s appended to each number in the array v. Thus, instead of using the command>> s.uels = {{'1','3'},{'j1','j2'},{'k1','k2'}};

we can conveniently use the command>> s.uels = {guel('',[1,3]),guel('j',1:2),guel('k',1:2)};

The benefit of using the latter command will be more apparent when creating many elements in the listing. Now, as an example, suppose we would like to do a filtered read on the parameter called`test2`

in`sample.gdx`

. We could use the following commands:>> s.name = 'test3'; >> s.form = 'full'; >> s.compress = false; >> s.uels = {guel('',[1,3]),guel('j',1:2),guel('k',1:2)}; >> x = rgdx('sample',s) x = name: 'test3' type: 'parameter' dim: 3 val: [2x2x2 double] form: 'full' uels: {{1x2 cell} {1x2 cell} {1x2 cell}} >> x.val ans(:,:,1) = 3 4 5 6 ans(:,:,2) = 4 5 6 7

Here it should be noted that we turned off compression while performing the filtered read. This is necessary because the filtered read will give data in accordance with the entered uels and the output uels will be the same as the input uels; thus compression is not possible. - field

This field is required when variables or equations are to be read from a GDX file. Sets and parameters in the GDX file do not have any field value but variables and equations have 5 fields namely, level, marginal, lower, upper, and scale. Thus, it may be useful to enter field as an input when reading an equation or a variable. A user can enter it as a string with valid values being`′l/m/up/lo/s′`

. e.g.>> s.field = 'm';

By default, the output will be the level value of a variable or an equation. - ts

This represents the text string associated with the symbol in the GDX file. If a user sets this field to be`′true′`

, then the output structure will have one more string field`′ts′`

that contains the text string of the symbol. e.g.>> s.ts = true;

- te

GAMS allows a modeler to enter text elements for a set. Similarly to the`′ts′`

field, if a user sets`′te′`

to be true in the input structure, then the output structure will contain one more field representing the text elements for that symbol. Please note that text elements only exist for`′sets′`

. e.g.>> s.te = true;

### Output Structure

As mentioned earlier, output of the `rgdx`

routine will be in structure form. This structure is very similar to the input structure. To get information regarding any symbol, we always need to display its basic characteristics, such as its name, type, value, uels, form, etc. An output structure will always have these fields:

- name

It is same as that entered in the input structure name field, i.e., the symbol name in the GDX file. - val

It represents the the value matrix of the symbol. To save MATLAB memory by default it will be in`′sparse′`

format. e.g.>> s = rmfield(s, 'form'); >> s s = name: 'test3' compress: 0 >> x = rgdx('sample', s) x = name: 'test3' type: 'parameter' dim: 3 val: [16x4 double] form: 'sparse' uels: {{1x8 cell} {1x8 cell} {1x8 cell}}

Here`val`

is a 16x4 double matrix. As it is a parameter; thus the last column of the sparse matrix will represent the value and the rest (i.e. the first three columns) will represent its index. Please note that in the case of a`′set′`

, the number of columns in the sparse matrix will be equal to its dimension, i.e., it will not have a column representing its value. Here, the presence of each row in the output`′val′`

field corresponds to the existence of a set element at that index. When a`′full′`

matrix output is specified, a 1 represents existence of a set element and a 0 otherwise. - form

It represents the format in which the`′val′`

field is being displayed. As mentioned earlier it can be either in`′full′`

or`′sparse′`

form. - type

While reading a symbol from a GDX file it is often very useful to know its type. The`rgdx`

routine is designed to read set, parameter, variable and equation. This field will store this information as a string. - uels

This represents the unique element listing of the requested symbol in the form of a cell array. It is a 1 x N cell array, where N is the dimension of the symbol. Each column of this array consists of string elements. By default, the output uels will be the same as the global uel of the GDX file, but it can be reduced to element specific local uels if compress is set to be true in the input structure. If a user is using a filtered read, i.e. calling`rgdx`

with input uels, then the output uels will be essentially the same as the input uels. dim

It is a scalar value representing the dimension of the symbol.Apart from these necessary fields there are a few additional fields as well. They are as follows:

- field

If we are reading variables or equations, then it becomes useful to know which field we have read, i.e.,`l/m/up/lo/s`

. This information is displayed via this field in the form of a string. - ts

It display the explanatory text string associated with the symbol. This field only exists in the output structure if the`′ts′`

field is set as`′true′`

in the input structure. - te

It is an N dimensional cell array representing the text elements associated with each index of the set. This field only exists in the output structure if the`′te′`

field is set as true in the input structure and the symbol is a set.

See datalib example `gdxmrw_rgdx01_init`

.

## wgdx

`wgdx`

is a MATLAB routine to create a GDX file containing data from MATLAB. Similar to the `rgdx`

routine, it takes a structure input and can write multiple symbols into a single GDX file with one call. The first argument is the file name of the GDX file to be created in string format; it can be with or without the `′.gdx′`

file extension. The rest of the arguments are structures, each containing data for different symbols to be written into the GDX file. The syntax for the call is:

>> wgdx('fileName', s1, s2 ...);

If the GDX file already exists in the MATLAB current directory, `wgdx`

will overwrite it; otherwise a new file will be created. After a successful run, it doesn't return anything back into MATLAB. Most of the fields of its input structures are the same as those of the `rgdx`

output structure. In the example below, we use the `wgdx`

routine to create `foo.gdx`

containing a set `′l′`

and a parameter `′par′`

.

>> s.name = 'l'; >> s.uels = {{'i1', 'i2', 'i3'}, {'j1', 'j2'}}; >> c.name = 'par'; >> c.type = 'parameter'; >> c.val = eye(3); >> c.form = 'full'; >> c.ts = '3 x 3 identity'; >> wgdx('foo', s, c)

The equivalent code in GAMS to create the above set `′l′`

is:

set a /i1*i3/; set b /j1*j2/; set l(a,b); l(a,b) = yes;

In the next subsection we will explain the input fields in detail.

### Input Structure

The necessary fields in an input structure to represent a symbol are given below:

- name

It is a string representing the name of the symbol. - val

It represents the value matrix of the parameter or set. It can be entered in either full or sparse format, whichever is convenient to the user; the corresponding format must be specified in the`′form′`

field. By default the value matrix is assumed to be in sparse format. - type

It is a string input to specify the type of the symbol. The`wgdx`

routine can write a set or parameter into the GDX file. In the previous example, we didn't specify the type for structure`′s′`

because by default it is assumed to be a set. - form

This is a string input representing the format in which the val matrix has been entered. By default it is assumed that the data is specified in sparse format. uels

Similar to the`rgdx`

uels field, this represents the local unique element listing of the symbol in an 1 x N cell array form. Each column of this cell array can contain string or double or both. Again we can make use of a handy function to create a UEL listing as mentioned previously in the section about`rgdx`

:>> guel = @(s,v) strcat(s,strsplit(num2str(v)));

As an example, we create a set using this function and the

`wgdx`

routine:>> s.name = 'l'; >> s.uels = {guel('i',1:20),guel('j',1:40)}; >> wgdx('foo',s);

The above code will create

`foo.gdx`

that contains a 2 dimensional set`′l′`

with set elements for each of the specified uels. If a user would like to create a set with set elements only at specified indices, one could do so by entering the indices in the val field. e.g.>> s.val = [1, 1; 2, 1; 3, 6; 4, 7]; >> wgdx('foo',s);

Now the above would result in a set with only elements existing in the indices specified in the

`val`

field. If a user enters a structure with only two fields, name and uels, as in the example given in the introduction of this section (structure s), then`wgdx`

will create a full set corresponding to the global uels.The optional fields are:

- dim

This field is useful when a user wants to write a zero dimensional or 1 dimensional data in full format. As every data matrix in MATLAB is at least 2D, it becomes necessary to indicate its dimension for writing purposes. - ts

This is the text string that goes with the symbol. If nothing is entered then`′MATLAB data from GDXMRW′`

will be written in the GDX file.

See datalib example `gdxmrw_wgdx01_init`

.

# Extended use

In this section, we will discuss a few examples of the MATLAB and GAMS interface. We will give a simple example of a nonlinear optimization problem that would benefit from this capability and describe the steps that are needed in order to use our interface in this application.

- Special values

Following example shows how special values are handled by this interface. It can be seen that`rgdx`

can retrieve all these values from GDX file and display them appropriately in MATLAB.>> s.name = 'special'; >> s.form = 'full'; >> s.compress = true; >> x = rgdx('sample', s) x = name: 'special' type: 'parameter' dim: 1 val: [4x1 double] form: 'full' uels: {{1x4 cell}} >> x.val ans = -Inf NaN 3.141592653589793 Inf

- Variables and Equations

In an optimization problem, we are not only interested in level value of variables and equations but also in their marginal values, lower and upper bounds. This interface gives its user ability to read any of these values into MATLAB. By default`rgdx`

and`gams`

routines will read the level value of equations and variables but this can be changed very easily by using`′field′`

in input structure. In`gams`

call user can also specify this in`′$set matout′`

statement. e.g.$set matout "'matsol.gdx', x.m, dual.lo=dl ";

In this case the marginal value of variable`′x′`

will be read and lower bound of dual variable will be read and stored in`′`

dl′. - Text string and Text elements

GAMS allows its user to enter text string and explanatory text elements and all GDX file contain this information as well. Following example shows how to get these text elements in MATLAB.>> s1.name = 'el'; >> s1.te = true; >> s1.ts = true; >> s1.compress = true s1 = name: 'el' te: 1 ts: 1 compress: 1 >> z = rgdx('sample', s1) z = name: 'el' type: 'set' dim: 2 val: [3x2 double] form: 'sparse' uels: {{1x2 cell} {1x2 cell}} ts: 'This is 2D set with text elements' te: {2x2 cell} >> z.te ans = 'element1' 'element2' '2.j1' [] >> z.val ans = 1 1 1 2 2 1

- String elements

One piece of information that may be needed within MATLAB is the modelstat and solvestat values generated by GAMS for the solves that it performed. This is easy to generate, and an example is given in`do_status.m`

. This example is generated by taking the standard gamslib trnsport example, and adding the following lines to the end:$set matout "'matsol.gdx', returnStat, str "; set stat /modelstat,solvestat/; set str /'grunt', '%system.title%'/; parameter returnStat(stat); returnStat('modelstat') = transport.modelstat; returnStat('solvestat') = transport.solvestat; execute_unload %matout%;

Note that the relevant status numbers are stored in GAMS into the parameter returnStat which is then written to matsol.gdx and read back into MATLAB using the`rgdx`

call.>> gamso.output = 'std'; >> gamso.form = 'full'; >> gamso.compress = true; >> s = gams('trnsport') s = 1 1

See datalib example `gdxmrw_ext01_init`

.

- Advanced Use: Plotting

One of the key features of the GAMS/MATLAB interface is the ability to visualize optimization results obtained via GAMS within MATLAB.

Some simple examples are contained with the program distribution. For example, a simple two dimensional plot with four lines can be carried out as follows. First create the data in GAMS and export it to MATLAB using the`gams`

routine (see APPENDIX C - Calling GAMS model from MATLAB for a detailed description of this routine).

We make an assumption that the user will write the plotting routines in the MATLAB environment. To create the plot in MATLAB, the following sequence of MATLAB commands should be input (saved as`do_plot.m`

).The data is created using the following gams code.gamso.output = 'std';gamso.compress = true;gamso.form = 'full';[a,xlabels,legendset,titlestr] = gams('simple');figure(1)% Plot out the four lines contained in a;% format using the third argumentplot(a,'+-');xtick = 1:5:length(xlabels{1});xlabels{1} = xlabels{1}(xtick);set(gca,'XTick',xtick);set(gca,'XTickLabel',xlabels{1});% Add title, labels to axestitle(titlestr{1});xlabel('Year -- time step annual');ylabel('Value');% Add a legend, letting MATLAB choose positioninglegend(legendset{1},0);% match axes to data, add grid lines to plotaxis tightgrid$title Examples for plotting routines via MATLAB$set matout "'matsol.gdx', a, t, j, sys ";set sys /'%system.title%'/;set t /1990*2030/, j /a,b,c,d/;parameter a(t,j);a("1990",j) = 1;loop(t, a(t+1,j) = a(t,j) * (1 + 0.04 * uniform(0.2,1.8)); );parameter year(*); year(t) = 1989 + ord(t);Omit some data in the middle of the graph:a(t,j)$((year(t) gt 1995)*(year(t) le 2002)) = NA;execute_unload %matout%;

The figure below is an example created using this utility ( and the MATLAB command `print -djpeg`

simple).

See datalib example `GDXMRWPlotting01`

.

MATLAB supports extensive hard copy output or formats to transfer data to another application. For example, the clipboard can be used to transfer meta files in the PC enviroment, or encapsulated postscript files can be generated. The `help print`

command in MATLAB details the possibilities on the current computing platform.

Scaling of pictures is also most effectively carried out in the MATLAB environment. The following code is an example of rescaling printed out. Note that the output of this routine is saved as a jpeg file `"rescale.jpg"`

.

Other examples of uses of the utility outlined in this paper can be found in the `"m"`

files:

that are contained in the distribution.

# Acknowledgements

The authors would like to thank Alexander Meeraus and Michael R. Bussieck of GAMS corporation for constructive comments on the design and improvement of this tool. Thanks to Rishabh Jain for work on an earlier version.

# APPENDIX A - Configuring GDXMRW

## Installation

This section describes the installation procedure for all types of machines. The following section describes the testing procedure for verifying a correct installation.

First of all, you need to install both MATLAB and GAMS on your machine. For brevity, we will assume that the GAMS system (installation) directory is (for Windows)

c:GAMS

and for non-Windows systems:

/usr/local/gams

All of the utilities come as a part of the GAMS distribution, so to use them you have only to add the GAMS directory to the MATLAB path. One way to do this is from the MATLAB command prompt, as follows:

>> addpath 'C:GAMS'; savepath;

OR this can be done by following these steps:

- Start MATLAB
- Click on 'File' tab.
- Now click on 'Set Path'
- Click on 'Add Folder'
- Select GAMS directory and click 'OK'.
- Save it and then close it.

## Testing

The GAMS system comes with some tests that you should run to verify the correct configuration and operation of the GDXMRW utilities. In addition, these tests create a log file that can be useful when things don't work as expected. To run the tests, carry out the following steps.

- Create a directory to run the tests in, e.g.
% mkdir \tmp

- Extract the test models and supporting files from the GAMS test library into the test directory.
% cd \tmp % testlib gdxmrw03 % testlib gdxmrw04 % testlib gdxmrw05 % testlib gdxmrw06

- Execute the GAMS files
`gdxmrw03`

,`gdxmrw04`

, and`gdxmrw06`

. The files`gdxmrw03`

and`gdxmrw04`

test that the`rgdx`

and`wgdx`

routines are working properly, and`gdxmrw06`

test`irgdx`

and`iwgdx`

. In addition to calling MATLAB in batch mode, they verify that the data are read and written as expected and give a clear indication of success or failure. - The GAMS file
`gdxmrw05`

tests the`gams`

MATLAB routine. Like the other tests, it can be run in batch mode. You can also run it interactively by starting MATLAB, making`tmp`

the current directory, and running the script`′testinst.m′`

.>> testinst

In addition to messages indicating success or failure, this test produces a log file`testinstlog.txt`

that will be useful in troubleshooting a failed test.

# APPENDIX B - Utility functions: gdxWhos and gdxInfo

In this section, we'll describe two utilities, `gdxWhos`

and `gdxInfo`

, provided with GDXMRW that can be called from the MATLAB command prompt. The `gdxWhos`

function is patterned loosely after the MATLAB function `whos`

used to query `.mat`

files. It provides information about symbols in a specified GDX file. The only input argument is the name of a GDX file, which can be with or without the `′.gdx′`

extension. With no output argument, `gdxWhos`

lists the symbols in the specified GDX file in the MATLAB command prompt. When used with an output argument, information about the symbols in the GDX file is returned as an array of structures. This meta-data can be used in a variety of ways and is especially useful when programming.

An example GDXWhos call without an output argument is shown below:

>> gdxWhos('idx1_.gdx'); Symbol info of GDX idx1_.gdx Index Type Dim NRecs Name 1 Parameter 0 1 a0 2 Parameter 1 3 a1(5) 3 Parameter 2 4 a2(2,2) 4 Parameter 3 6 a3(3,5,2) 5 Parameter 10 256 a10(3,5,2,2,2,2,2,2,2,2)

The above output indicates that there are five indexed parameters in the file `′idx1_.gdx′`

. The way we can recognize an indexed parameter is by noticing numbered arguments of the parameters under the `′Name′`

column. For non-indexed data, the arguments will appear either as the name of a set or by a '*'. For example, the first three symbols are non-indexed and the last symbol is indexed in the output from this call:

>> gdxWhos('fake.gdx'); Symbol info of GDX fake.gdx Index Type Dim NRecs Name 1 Set 1 3 i(*) 2 Parameter 1 3 a(i) 3 Set 1 3 d_i_m__3(*) 4 Parameter 1 3 aa(3)

The GDXInfo function is patterned after the `gdxdump`

utility in GAMS. It lists and dumps all of the data values for each symbol in the specified GDX file to the MATLAB console. The only input argument is the name of a GDX file, which can be with or without the `′.gdx′`

extension. It accepts no output arguments. It is especially useful to see the entire contents of smaller GDX files. To use the function, we could simply enter a command such as `gdxInfo(′idx1_′)`

in the MATLAB command prompt, and the entire data contents would be displayed in the console. Further information about what is displayed can be found in the GAMS documentation of the `gdxdump`

utility.

# APPENDIX C - Calling GAMS model from MATLAB

Until now we have discussed the data Import/Export utility between MATLAB and GAMS. In this section, we will discuss a new MATLAB utility `′gams′`

that initializes a GAMS model with MATLAB data then executes GAMS on that model and bring the results back into MATLAB. This `′gams′`

routine is based on the same design as `rgdx`

and `wgdx`

but instead it does everything in one call. This routine can take multiple input arguments and can return multiple output arguments. Its standard syntax is as follows:

>> [x1, x2, x3] = gams('model', s1, s2.., c1, c2..);

Here note that the first argument of `gams`

is the GAMS model name plus any user specific command line settings. If a user wants to solve the given model (in this case found in `qp.gms`

) using a different solver then it can be done by adding that solver to the GAMS model name as `"qp nlp=baron"`

. This feature allows a user to change the execution time behavior of the model.

The rest of the input arguments of GAMS are structures. Their positioning is not important. These structures are of two kinds; one is similar to the input structure of `wgdx`

and the other structure has just two string fields, name and val. This latter structure is used to set or overwrite values in the model using the `"$set"`

variables syntax of GAMS. We will explain it in detail a later section.

The first step is to generate a working GAMS model. For example, we can set up a simple model file to solve a quadratic program that minimizes ½ x^{T}Qx+c^{T}x subject to Ax≥b and x≥0.

The GAMS model for this quadratic problem is as given in the following code.

This GAMS qp model can be executed directly at the command prompt using the following command

gams qp (for Unix/Linux)

or

gams.exe qp (for Windows)

or the user can simply hit the run button in the GAMSIDE. The optimal value is 0.5. In order to run the same model within MATLAB and return the solution vector x back into the MATLAB workspace, no change is required to the GAMS file. In MATLAB, all you have to do is to execute the following command:

>> x = gams('qp');

This command will first collect the input structure data and create `′matdata.gdx′`

and `′matdata.gms′`

that contains include statements for the symbols written in a file `matdata.gdx`

. In the previous example there is no structural input, so an empty `′matdata.gdx′`

file will be created and `′matdata.gms′`

will have just have a load statement for the GDX file but no load statements for any symbol. This is done to prevent any undesirable loading of data in the main model if there had already existed a `′matdata.gdx′`

or `′matdata.gms file′`

. After creating these two files then the `′gams′`

routine will execute `"gams qp"`

using a system call. When this model is executed, another file `′matsol.gdx′`

will be created because of the `execute_unload`

statement in the last line of the model. Here it should be noted that any model that you want to execute using the MATLAB `gams`

routine should contain something like

$set matout "'matsol.gdx', x, dual, obj, returnStat ";

either as the first line, or somewhere near the start of the model file. This is a standard `gams`

`$set`

statement, setting the value of the local variable `′matout′`

. The reason to have this statement near the start of the gms file is that the `gams`

routine searches the file from the beginning for `"$set matout"`

in the gms file. As these files can be very large, it is wise to have this statement near the start of the file. In this statement `′fileName′`

is the gdx file name that will be created containing symbols `′x1′`

, ′x2′`, etc. These symbols can then be exported to MATLAB. The last line of the model should always be

execute_unload %matout%;

The purpose of setting the first and last line of the model in this manner is to specify what data the user wants to export to MATLAB in a `"header"`

of the model. As MATLAB does not give any information about the output arguments except the number of expected arguments, we have to specify what data to export to MATLAB in the GAMS model with minimum modification to the existing model. In the previous example, there is only one output argument, thus the `gams`

routine will get data for its first element from the output gdx file and store it in the MATLAB output argument.

If there is more than one output argument, e.g.,

>> [x, u] = gams('qp');

then the `gams`

routine will read the output gdx file and store its first element information of the GDX file as the first output argument of MATLAB, i.e. `′x,′`

and the second element information of the GDX file in the second output argument of MATLAB, i.e. `′u′`

and so on. If the number of MATLAB output arguments is greater than the number of elements in the GDX file then `gams`

will throw an error.

See datalib example `gdxmrw_qp4`

.

## Input Structure

As mentioned earlier, the `gams`

routine takes input arguments in structured form. It allows two different types of structure input. One contains the symbol data similar to the wgdx input structure, to be exported to the GDX file. The other structure will just have two string fields 'name' and 'value'. Example use:

>> s.name = 'Q'; >> s.val = eye(3); >> s.form = 'full'; >> m = struct('name','m','val','2'); >> [x] = gams('qpmcp',s, m);

In this example both `′s′`

and `′m′`

are structures but `′m′`

has only two fields and both are strings. The `gams`

routine will use the 's' structure to create a `′matdata.gdx′1 file and`

′m′` to modify the execution command line to include "--m=2" at the end i.e. a command that will executed will be "gams qpmcp --m=2"

The structure `′s′`

is the same as the input structure for `wgdx`

but with two important differences. Firstly, it can be seen in the above example that `′s′`

doesn't have a `′type′`

field. In `wgdx`

we assume the type to be 'set' by default, but in the `gams`

routine the type is assumed to be `′parameter′`

by default. The second change is an optional additional input field (in addition to those given in Section Input Structure) called `"load"`

.

- load

It is a string input representing how the corresponding data will be loaded into the GAMS program. Depending on the value of the global option`"gamso.input"`

(see next section) the input data will be read into GAMS in different ways. Suppose the input structure`′s′`

has a`′name′`

field called`′foo′`

. By default (where`gamso.input = ′compile′`

), the file`matdata.gms`

will$loadR foo

The GAMS parameter (or set) foo will be replaced by the data that is in the`′matdata.gdx′`

container called`′foo′`

. If the data has been initialized before in the model, this will replace that intial data with the new data from `matdata.gdx'. The option can also be explicitly set usings.load = `replace'

There are two other compile time load options, namely`′initialize′1 and`

′merge′`. The first is only valid if the parameter values have not been initialized in the GAMS file, otherwise an error is thrown. It uses the GAMS syntax$load foo

The merge option is valid when the GAMS file being run has already initialized the parameter values. The new values in the MATLAB structure`′s′`

are merged into the parameter simply overwriting existing values with the new values given. Explicitly, the`′matdata.gms′`

file contains the statement$loadM foo

to direct GAMS accordingly.

Finally, if gamso.input = `exec', the loading will occur at execution time. In this case,`s.load = ′initialize′`

is not a valid input, the default setting is`s.load = ′replace′`

which carries outexecute_load "matdata.gdx" foo

and the alternative setting`s.load = ′merge′`

carries outexecute_loadpoint "matdata.gdx" foo

In this way, the data is loaded at execution time and performs an appropriate replace or merge.

## Global input to change default behavior

Until now we have seen how to specify different input to the `gams`

routine and in this section we will see how to change the default behavior of a `gams`

call. This can be done by creating a structure `′gamso′`

in the current workspace and adding different fields to that structure. There are currently nine fields that can be set in that structure that affect the behavior of the program. Except the uels field, all other string fields take case insensitive data. These are as follows:

`gamso.output`

By default, output of the`gams`

routine will be in structure form but it might be the case that a user is only interested in the data matrix, i.e., the val field of that structure. This can be done by setting`gamso.output`

as`′std′`

. This will give only the value matrix as output. If this is not set to`′std′`

then output will be in the structure form described in the`wgdx`

section.>> gamso.output = 'Std'; >> x = gams('qp nlp=baron') x = 0.5000

`gamso.input`

By default, the interface updates data at compile time. Thus, if execution time updates are made to the parameters before the line`′$include matdata.gms′`

these may override the data that is provided in`′matdata.gms′`

(i.e. from the command line). This may not be desirable. If you wish to perform execution time updates to the data, you should set`gamso.input`

to`′exec′`

.`gamso.write_data`

If this is set to`′no′`

, then all parameters on the call to`gams`

are ignored, except the program name. This is useful for dealing with large datasets. Consider the following invocation:x = gams('largedata','A'); y = gams('resolve','A');

The first call generates a file`′matdata.gms′`

containing the elements of the matrix`A`

for use in the`largedata.gms`

program. The second call rewrites a new`′matdata.gms′`

file that again contains`A`

. If we wish to save writing out`A`

the second time we can use the following invocation:x = gams('largedata','A'); gamso.write_data = 'no'; y = gams('resolve','A'); clear gamso;

or the equivalent invocation:x = gams('largedata','A'); gamso.write_data = 'no'; y = gams('resolve'); clear gamso;

`gamso.show`

This is only relevant on a Windows platform. This controls how the`′command box′`

that runs GAMS appears on the desktop. The three possible values are:`′minimized′`

(default): The command prompt appears iconified on the taskbar.`′invisible′`

: No command prompt is seen.`′normal′`

: The command prompt appears on the desktop and focus is shifted to this box.

`gamso.path`

This option is used to specify fully qualified path for the gams executable. This is very useful if you have multiple versions of GAMS installed on your system and want to make sure which version you are running for the`gams`

call. e.g.>> gamso.path = 'C:\Program Files\GAMS23.4GAMS.exe';

The output of `gams`

is similar to `rgdx`

but unlike the `rgdx`

`gams`

routine it doesn't take input specific to a particular symbol. Thus it becomes important to implement a way to change the default behavior of the output. This can be acheived by adding following field to the global structure `′gamso′`

. All these fields behave similar to that described in `rgdx`

and take the same input as of `rgdx`

.

`gamso.compress`

`gamso.form`

`gamso.uels`

`gamso.field`

This is a global option however.