GAMS Data eXchange (GDX)

This document describes the GDX (GAMS Data eXchange) facilities available in GAMS. The GDX facilities provide basic functionalities for exchanging GAMS data such as read and write. In addition to these facilities, there are a number of GDX Tools for exchanging data between GAMS and othere data sources as well as for performing specific operations on a GDX file.

A GDX file is a file that stores the values of one or more GAMS symbols such as sets, parameters variables and equations. GDX files can be used to prepare data for a GAMS model, present results of a GAMS model, store results of the same model using different parameters etc. A GDX file does not store a model formulation or executable statements. Among other usages a GDX file can also be used to prepare data for a GAMS model, pass results of a GAMS model into different programs, and pass results into GAMS from different programs.

GDX files are binary files that are portable between different platforms. They are written using the byte ordering native to the hardware platform they are created on, but can be read on a platform using a different byte ordering. See also General notes on GDX files .

Reading a GDX file

Reading a GDX file into a GAMS model can be done during the compile phase or the execution phase.

Compile Phase

During compilation, we can use Dollar Control Options to read data from a GDX file. Reading during the compile phase also allows us to define the elements of a set and the subsequent use of such a set as a domain. The following Dollar Control Options are available for reading data from a GDX file into GAMS during compilation of a GAMS model :

$gdxin

The $gdxin command is used in a sequence either to load specified items from a GDX file or to close the specified GDX file. The statement with $gdxin can be used in one of the following forms:

$gdxin file_name 
$gdxin 

where

Parameter Description
file_name Specify the name of the GDX file (with or without the extension .gdx; read from the current working directory)
(no file_name) closes the GDX file

See Example 1 - Reading a GDX File and Example 2 - Reading a GDX File on how to use $gdxin.

$load

The $load command loads specified items from a GDX file. The statement containing $load is preceded and succeeded by the $gdxin statements where

  • the preceding $gdxin specifies the GDX file name and opens the file.
  • the succeeding $gdxin closes the GDX file.

The statement with $load command can be used in one of the following forms:

$load
$load id=*
$load id1 id2 ... idn
$load id1=gdxid1 id2=gdxid2
$load id1<=gdxid1 id2<=gdxid2.dim3

where:

Parameter(s) Description
(no identifiers) Reads all symbols from the gdx file and writes to the listing file
id=* Loads all unique elements from the gdx file into set id
id1 id2 ... idn Read GAMS symbols id1, id2, … idn from the GDX file
id1=gdxid1 id2=gdxid2 Read GAMS symbols id1, id2 with corresponding names gdxid1, gdxid2 in the GDX file
id1<gdxid1 id2<gdxid2.dim3 Reads GAMS one dimensional set id1 and id2 from the GDX parameter or set gdxid1 and gdxid2. WIthout the dimN suffix, GAMS tries to match the domains from the right (<). If no domain information is available for the GDX symbol, the dimN suffix determines the index position that should be read into the GAMS set. For more details see the Example 4 - Reading a GDX File.
id1<=gdxid1 id2<=gdxid2.dim3 Reads GAMS one dimensional set id1 and id2 from the GDX parameter or set gdxid1 and gdxid2. WIthout the dimN suffix, GAMS tries to match the domains from the left (<=). If no domain information is available for the GDX symbol, the dimN suffix determines the index position that should be read into the GAMS set. For more details see the Example 4 - Reading a GDX File.
Note
  • One can load the level values of a variable into a parameter of the same dimension using the syntax parametername=var.l as follows
    parameter storexlevel(i,j);
    $gdxin tran2
    $load storexlevel=x.l
    $gdxin
    display storexlevel;
    
  • Loading the level values for a variable into the same variable (x=x.l) works but loads everything including bounds, scales, marginals and levels.

See Example 1 - Reading a GDX File and Example 2 - Reading a GDX File on how to use $load.

$loaddc

The $loaddc command lists all symbols in the GDX file and will cause a compilation error when the data read causes a domain violation. In contrast the $load command simply ignores elements that are not in the domain. See also Compile Phase Reading Data using $load.

The statement with $loaddc can be used in one of the following forms:

$loaddc
$loaddc id=*
$loaddc id1 id2 ... idn
$loaddc id1=gdxid1 id2=gdxid2
$loaddc id1<=gdxid1 id2<=gdxid2.dim3

where:

Parameter(s) Description
(no identifiers) Reads all symbols from the gdx file and writes to the listing file
id1 id2 ... idn Read GAMS symbols id1, id2, … idn from the GDX file
id1=gdxid1 id2=gdxid2 Read GAMS symbols id1, id2 with corresponding names gdxid1, gdxid2 in the GDX file
id1<gdxid1 id2<gdxid2.dim3 Reads GAMS one dimensional set id1 and id2 from the GDX parameter or set gdxid1 and gdxid2. WIthout the dimN suffix, GAMS tries to match the domains from the right (<). If no domain information is available for the GDX symbol, the dimN suffix determines the index position that should be read into the GAMS set. For more details see the Example 4 - Reading a GDX File.
id1<=gdxid1 id2<=gdxid2.dim3 Reads GAMS one dimensional set id1 and id2 from the GDX parameter or set gdxid1 and gdxid2. WIthout the dimN suffix, GAMS tries to match the domains from the left (<=). If no domain information is available for the GDX symbol, the dimN suffix determines the index position that should be read into the GAMS set. For more details see the Example 4 - Reading a GDX File.

See Example 4 - Reading a GDX File on how to use $loaddc.

$loadm, $loadr, $loaddcm, and $loaddcr

The $loadm, $loadr, $loaddcm, and $loddcr are additional forms of the $load and $loaddc directives. The M indicating a merge and the R indicating a full replacement.

The statements with $loadm can be used in one of the following forms:

$loadm 
$loadm id1 id2 ... idn
$loadm id1=gdxid1 id2=gdxid2
$loadm id1<gdxid1 id2<gdxid2.dim3
$loadm id1<=gdxid1 id2<=gdxid2.dim3

The statement with $loadr can be used in one of the following forms:

$loadr 
$loadr id1 id2 ... idn
$loadr id1=gdxid1 id2=gdxid2
$loadr id1<gdxid1 id2<gdxid2.dim3
$loadr id1<=gdxid1 id2<=gdxid2.dim3

The statement with $loaddcm can be used in one of the following forms:

$loaddcm 
$loaddcm id1 id2 ... idn
$loaddcm id1=gdxid1 id2=gdxid2
$loaddcm id1<gdxid1 id2<gdxid2.dim3
$loaddcm id1<=gdxid1 id2<=gdxid2.dim3

The statement with $loddcr can be used in one of the following forms:

$loaddcr 
$loaddcr id1 id2 ... idn
$loaddcr id1=gdxid1 id2=gdxid2
$loaddcr id1<gdxid1 id2<gdxid2.dim3
$loaddcr id1<=gdxid1 id2<=gdxid2.dim3

where:

Parameter(s) Description
(no identifiers) Reads all symbols from the gdx file and writes to the listing file
id1 id2 ... idn Read GAMS symbols id1, id2, ..., idn from the GDX file
id1=gdxid1 id2=gdxid2 Read GAMS symbols id1, id2 with corresponding names gdxid1, gdxid2 in the GDX file
id1<gdxid1 id2<gdxid2.dim3 Reads GAMS one dimensional set id1 and id2 from the GDX parameter or set gdxid1 and gdxid2. WIthout the dimN suffix, GAMS tries to match the domains from the right (<). If no domain information is available for the GDX symbol, the dimN suffix determines the index position that should be read into the GAMS set.
id1<=gdxid1 id2<=gdxid2.dim3 Reads GAMS one dimensional set id1 and id2 from the GDX parameter or set gdxid1 and gdxid2. WIthout the dimN suffix, GAMS tries to match the domains from the left (<=). If no domain information is available for the GDX symbol, the dimN suffix determines the index position that should be read into the GAMS set.

See Example 4 - Reading a GDX File and [load8] examples on how to use $loadm and $loaddcm.

$loadidx

The command $loadidx read GAMS symbols from the GDX file. Each symbol should have been written using an indexed write.

The statement with $loadidx can be used in one of the following forms:

$loadidx 
$loadidx id1 id2 ... idn
$loadidx id1=gdxid1 id2=gdxid2

where:

Parameter(s) Description
(no identifiers) Reads all symbols from the gdx file and writes to the listing file
id1 id2 ... idn Read GAMS symbols id1 id2 ... idn from the GDX file. Each symbol should have been written using an indexed write; see also execute_unloadidx
id1=gdxid1 id2=gdxid2 Read GAMS symbols id1, id2 with corresponding names gdxid1, gdxid2 in the GDX file. Each symbol should have been written using an indexed write; see also execute_unloadidx

See [ldidx01] example on how to use $loadidx.

Execution Phase

When reading data from a GDX file during execution phase the data in the GDX file will be the data present in the GDX file at the time that the statement is executed. The results of all prior calculations and the most recent solve for any model will be reflected. The statements to read data from a GDX file during execution phase are:

execute_load

The execute_load statement acts like an assignment statement, except that it does not merge the data read with the current data; it is a full replacement. The same restrictions apply as in an assignment statement: we cannot assign to a set that is used as a domain, or to a set used as a loop control.

Sets defining domains cannot be loaded. However sets that are subsets of existing sets and do not define new elements can be loaded at execution time (Domain defining sets can be loaded can at compile time using $load).

The statement with execute_load can be used in one of the following forms:

execute_load 'file_name', id1, id2, ..., idn ;
execute_load 'file_name', id1=gdxid1, id2=gdxid2, ..., id2=gdxid2 ;
execute_load 'file_name', setid=* ;

where

Parameter(s) Description
file_name Specify the name of the GDX file (with or without the extension .gdx; read from the current working directory)
id1, id2, ..., idn Read GAMS symbols id1, id2, ..., idn from the GDX file; note that commas are optional
id1=gdxid1, id2=gdxid2, ..., id2=gdxid2 Read GAMS symbols id1, id2 with corresponding names gdxid1, gdxid2, ..., gdxidn in the GDX file; note that commas are optional
setid=* Allow to load the universe of labels from a GDX file into a set. Note, that only labels known to the GAMS program will be loaded.
Note
  • items must be declared with Set, Parameter, Scalar, Variable or Equation statements before the execute_load appears.
  • when loading data domain checking is not enforced so that when an item is resident in a GDX file for set elements not present in the current file these items are ignored and do not create errors or cause generation of any messages. The execute_loaddc variant checks to see that the domains match.

See [load11] and [qp1x] from on how to use execute_load.

execute_loaddc

With execute_loaddc statement any domain violation will be reported and flagged as execution error. In contrast, the execute_load statement ignores all domain violations and loads only data that meets the domain restrictions. In addition to loading data for sets, parameters and variables, we can load a field of a variable into a parameter.

Warning: when loading a single field, all other fields are reset to their default value.

The statement with execute_loaddc can be used in one of the following forms:

execute_loaddc 'file_name', id1, id2, ..., idn ;
execute_loaddc 'file_name', id1=gdxid1, id2=gdxid2, ..., id2=gdxid2 ;

where

Parameter(s) Description
file_name Specify the name of the GDX file (with or without the extension .gdx; read from the current working directory)
id1, id2, ..., idn Read GAMS symbols id1, id2, ..., idn from the GDX file; note that commas are optional
id1=gdxid1, id2=gdxid2, ..., id2=gdxid2 Read GAMS symbols id1, id2 with corresponding names gdxid1, gdxid2, ..., gdxidn in the GDX file; note that commas are optional
setid=* Allow to load the universe of labels from a GDX file into a set. Note, that only labels known to the GAMS program will be loaded.

See [load7] example on how to use execute_loaddc.

execute_loadpoint

The execute_loadpoint is similar to execute_load, however, the new values are merged with the old values. If no arguments besides the name of the GDX file are given, all variables and equations that match variables and equations of the calling GAMS programs will be merged with the GDX level and marginal values. Bounds, scales and priorities will remain unchanged.

The statement with execute_loadpoint can be used in one of the following forms:

execute_loadpoint 'file_name', id1, id2, ..., idn ;
execute_loadpoint 'file_name', id1=gdxid1, id2=gdxid2, ..., id2=gdxid2 ;

where

Parameter(s) Description
file_name Specify the name of the GDX file (with or without the extension .gdx; read from the current working directory)
id1, id2, ..., idn Read GAMS symbols id1, id2, ..., idn from the GDX file; note that commas are optional
id1=gdxid1, id2=gdxid2 Read GAMS symbols id1, id2 with corresponding names gdxid1, gdxid2 in the GDX file; note that commas are optional

Example 1 - Reading a GDX File

The file trnsport.gms (from [TRNSPORT]) has been modified to use the demand data from an external source. Only the relevant declarations are shown.

The parameter B is read from the GDX file using the name 'demand', and only those elements that are in the domain 'jwill be used. Values for parameterBthat are outside the domainj` will be ignored without generating any error messages.

* Example 1
Set  
   j    markets  / new-york, chicago, topeka / ;
Parameter 
   B(j) demand at market j in cases ;
$gdxin demanddata.gdx  
$load  b=demand 
$gdxin

Example 2 - Reading a GDX File

In this example, the set J is also read from the GDX file, and is used as the domain for parameter B. All elements read for the set J will be used. Values for the parameter B that are outside the domain J will be ignored. Note that the dimension of set J is set to one by specifying its domain.

* Example 2
$gdxin demanddata.gdx  
Set  
   J(*)    markets;
$load j=markets
Parameter 
   B(j) demand at market j in cases ;
$load  b=demand 
$gdxin

Example 3 - Reading a GDX File

The $load command with out parameters can read a listing of all symbols in a GDX fie. The following:

* Example 3
$gdxin trnsport.gdx
$load

writes the following to the listing file:

Content of GDX C:\XLSFUN\TRNSPORT.GDX

Number Type      Dim  Count  Name
 
    1 Set         1      2  i         canning plants
    2 Set         1      3  j         markets
    3 Parameter   1      2  a         capacity of plant i in cases
    4 Parameter   1      3  b         demand at market j in cases
    5 Parameter   2      6  d         distance in thousands of miles
    6 Parameter   0      1  f         freight in dollars per case per thousand miles
    7 Parameter   2      6  c         transport cost in thousands of dollars per case
    8 Variable    2      6  x         shipment quantities in cases
    9 Variable    0      1  z         total transportation costs in thousands of dollars
   10 Equation    0      1  cost      define objective function
   11 Equation    1      2  supply    observe supply limit at plant i
   12 Equation    1      3  demand    satisfy demand at market j

which lists the items present by Type, Name, Number of sets the item is defined over(Dim), number of elements in the file for this item (Count).

Example 4 - Reading a GDX File

Sometimes, a set is implicitly given by the elements of a parameter symbol. For example,

parameter a(i) / seattle 350, san-diego 600 / ;

in trnsport.gms implicitly defines the set of plants i. GAMS does not allow us to provide domain checked data, if the data for domain sets is unknown. So this code produces a compilation error:

Set i plant;
Parameter a(i) capacity / seattle 350, san-diego 600 /;

When entering data directly in the GAMS source adding the domain sets before the actual parameter declarations is usually not a problem, but when data comes from external sources (e.g. spreadsheets, databases, etc), this often results in an additional query to the database, spreadsheet etc. Nowadays, such data exchange happens mostly via the GD facility. With the domain load capability of the compile time load instructions ($load, $loadDC, $loadR, $loadM, $loadDCM, and $loadDCR) one can project an index position from a parameter or set symbol in the GDX container and load this slice into a one dimensional set. Here is a simple example:

Set i plant;
Parameter a(i) capacity;
$gdxin data
$load i<adata a=adata

This will try to load set elements from the GDX parameter symbol adata into the set i and next load the GDX parameter adata into the GAMS parameter a. The latter one is no problem anymore, since the data for set i is known when loading symbol a. GAMS will use the domain information stored in GDX of parameter adata to identify the index position to project on. If no appropriate domain information can be found in GDX, the GAMS compiler will generate an error. In such case the user can explicitly select an index position (here first index position) from the GDX symbol:

$load i<adata.dim1 a=adata

The automatic index position matching (i.e. no .dimN) using the domain information stored in GDX matches on the name of the set to be loaded and the domain set names stored in GDX for the symbol. The domain in GDX are searched from right to left (start with n=symbol dimension, then n-1, n-2, ...) and stops at the first match. With the projection symbol <=, the domain in GDX is searched from left to right. This follows the style of the GAMS run time projection operation:

option sym1<sym2, sym1<=sym2;

Here is an example how to load. The network is defined by the capacity parameter cap contained in a GDX container net.gdx:

parameter cap(n,n) / (1*3).4 5, 4.(5*9) 3 /;

The following code loads the entire node set n of the network as well as the nodes with outgoing (out) and incoming (in) arcs and the capacity c.

set n nodes, out(n), in(n);
parameter c(n,n) capacity;
$gdxin net
$loadM n<=cap n<cap
$loadDC out<cap.dim1 in<cap.dim2 c=cap
display n, out, in;

The listing file looks as follows:

----      6 SET n  nodes
1,    2,    3,    4,    5,    6,    7,    8,    9
 
----      6 SET out  Domain loaded from cap position 1
1,    2,    3,    4
 
----      6 SET in  Domain loaded from cap position 2
4,    5,    6,    7,    8,    9

There is a potential issue with loading domains from parameters that have a zero value for some record. Since GAMS works with sparse data, it is sometime difficult to distinguish between a record with value zero (0) and the non-existence of a record. This is usually not a problem since we know the domain of a parameter and hence know all potential records. In case of using a parameter to define the domain this represents a source of confusion. Moreover, GDX has the capability of storing true zeros (most GDX tools like gdxxrw have options (Squeeze=Y or N) to either write a true 0 or squeeze the 0s when writing GDX). So in case GDX has a zero record, a domain load from such a parameter will include this record. Here is an example. The spreadsheet Book1.xlsx contains the following data:

gdxbook.jpg

The GDX utility GDXXRW with the following command line:

gdxxrw Book1.xlsx Squeeze=N par=dat rng=Sheet1!a1 rdim=1

reads the Excel data and produces a GDX container Book1.gdx with a one dimensional parameter dat(*) which can be viewed in the GDX browser in the GAMSIDE:

gdxbookide.jpg

Notice that label a4 is present while label a3 is not part of GDX symbol dat. Without the Squeeze=N (the default is Squeeze=Y) we also would not have seen a4. If we load dat to define the domain (remember we need to use $load i<dat.dim1 since gdxxrw does not write domain information to GDX), we will miss out on a3 but have a4 (assuming Squeeze=N). Please also note that the zero record disappears on regular loading and is turned into an EPS when loading under $OnEps:

set i;
parameter a(i);
$gdxin Book1
$load i<dat.dim1 a=dat
display i,a;
parameter a0(i);
$OnEps
$load a0=dat
display a0;

This results in a listing file

----      5 SET i  Domain loaded from dat position 1
a1,    a2,    a4
 
----      5 PARAMETER a
a1 5.000,    a2 1.000
 
----      9 PARAMETER a0
a1 5.000,    a2 1.000,    a4   EPS

With gdxxrw parameter Squeeze=Y the listing file would look as follows:

----      5 SET i  Domain loaded from dat position 1
a1,    a2
 
----      5 PARAMETER a
a1 5.000,    a2 1.000
 
----      9 PARAMETER a0
a1 5.000,    a2 1.000

Example 5 - Reading a GDX File

The following statement reads gams element k, d, f, a, b, and x from file trans2.gdx during execution phase:

execute_loaddc 'tran2',k=j,d,f,a=sup,b=dem,x,supply;

where k be renamed from j, abe renamed fromsup, andbbe renamed from demin the GDX filetran2.gdx`.

Suppose there is one element topeka is missing from the set definition but the element remains in the GDX file tran2.gdx. As a consequences the listing file cotains an error message like:

**** GDX ERROR AT LINE 45 - Domain violation when loading from GDX file
**** 1 Domain errors for symbol k
     topeka

**** GDX ERROR AT LINE 45 - Domain violation when loading from GDX file
**** 2 Domain errors for symbol d
     seattle.topeka
     san-diego.topeka

and the job is aborted with an execution error.

Note
  • domain errors occur whenever set element names are not spelled exactly the same as an element specified in the corresponding set in GAMS flagging alternative spellings or missing elements.
  • domain errors do not arise when items are not specified with them set to zero (no entery for a set element leaves to a corresponding value of zero)

Inspecting contents of a GDX file

In addition to reading data from a GDX file during compile phase or execution phase there are a few ways to examine the contents of a GDX file.

Inspecting contents with $load

The $load commad without any parameters will show a listing of all symbols in the GDX file. See Example 3 - Reading a GDX File on how to use $load to get a listing of all symbols.

Inspecting contents with the IDE

The GAMSIDE can be used to view the contents of a GDX file by opening a GDX file with the Open file dialogue. The IDE only recognizes the .gdx file extension of a GDX file. The resultant display gives the names of the items in the GDX file on left hand part of the screen while the right part gives the exact data entries for the item highlighted in the left hand part.

For example, opening a GDX file using the Open file dialogue and and moving the cursor to the set i6 causes the screens, it becomes:

gdxviewer_i6.png

showing the elements in the set as contained within the GDX file. Similarly choosing the object modedistance yields the screen:

gdxviewer_modelinstance.png

showing the data for this item. The data may be reordered via the mouse. For example placing the mouse on the column containing san francisco and dragging the column up yields:

gdxviewer_sanfrancisco.png

which is the slice of the matrix containing san francisco data. In turn clicking on Brussels yields:

gdxviewer_brussels.png
Note
  • the columns are sortable in the left hand portion of the display. All one needs to do is to click on the gray boxes (Symbol, Type,...) with the mouse. There is also a search dialog at the bottom that permits one to look for select items.
  • a right mouse click allows one to write the contents of any or all items to HTML format.

Inspecting contents with GDXDUMP

The GDXDUMP utility can list the symbols in the file and it also can write sets and parameters formatted as a GAMS data statement.

gdxdump gdxfile_name format=choice symb=optional choice

where

  • the gdxfile_name is the name of the GDX file to write data from.
  • the output is created to the screen not to a file. One can also write the GDX file contents into a GAMS file usign the command:
    gdxdump gdxfile_name > filetouse.gms
    
  • Data for a selected set, parameter, variable or equation (under all three of the output options when a specific item is named using the SYMB option)
  • Data for all sets, parameters, variables and equations (Under normal option when the SYMB is not used)
  • Data on solution items (variables and equations) formatted in a fashion suitable for import as a basis in another GAMS program where the marginals and levels are output. All of the scalars, sets and parameters (tables) in a GDX file to standard output formatted as a GAMS program with data statements or in CSV format. It skips information for variables and equations.
  • Under the format=CSV choice it only creates output when a symbol is selected using the SYMB syntax.
  • Under the format=CSV choice when the requested symbol is a variable or an equation one only gets the level values not the marginal, under the other formats one gets all items.
  • Under the format=gamsbas choice one gets all variables and equations when the SYMB syntax is not used.

Suppose we wish to write out he GDX files tran.gdx, then we would use the command:

gdxdump tran

See more Examples on inspecting contents with GDXDUMP.

Inspecting contents with GDXDIFF

The GDXDIFF utility can be used to compare two GDX files by creating a third GDX file containing a list of differences between all symbols. In particular for all items with the same name, type and dimension in the two GDX files the differences in numerical values are written to a third GDX file with A summary report written to standard output (ordinarily the LOG file).

This utility can be used either at command line, or by $Call, or execute command.

 gdxdiff file1 file2 {diffile} {Eps = value} {RelEps = value} {Field = FieldName} {ID=Identifier}

GDXDIFF requires the first two file name parameters,

  • File1 Name of the first GDX file
  • File2 Name of the second GDX file

The remaining parameters are optional

  • Diffile An optional third file name that is the name of the GDX file that contains the differences found in the parameters. If that parameter, is absent the file will be named 'diffile.gdx' and placed in the current directory.
  • Eps = value A tolerance that is the maximum amount that two numbers may differ by ie given a1 and a2 then abs(a1-a2) is reported as different if it exceeds this tolerance
  • RelEps = value A tolerance that is the maximum percentage amount that two numbers may differ by ie given a1 and a2 then abs(a1-a2)/max(abs(a1),abs(a2)) is reported as different if it exceeds this tolerance.
  • Field = FieldName A field that if specified limits doen between the information for variables and equations to specific attributes (Lo, L, Up, M, Scale and Prior)
  • ID=Identifier Limits the comparisons to selected items; items not specified will be ignored. Multiple items can be specified as: ID=id1 ID=id2 or ID="id1 id2"

Suppose we wish to compare the GDX files tran.gdx and tran2.gdx, then we would use the command:

gdxdiff tran tran2

In turn the output to standard output (nominally the terminal screen) appears as follows:

Summary of differences:
     d   Data is different
   dem   Keys are different
   sup   Keys are different
supply   Symbol not found in file 1
     x   Symbol not found in file 1

and summarizes the differences found. Simultaneously the file diffile.gdx when examined in the IDE contains the following

gdx_difffile_ide.png

which reports on the differences found in the two files.

Note
  • Some new coding is introduced in the difference GDX file. Namely a new dimension is added to the parameters being compared which can contain 4 entries
    • dif1 indicates that the entry occurs in both files and shows the value found in the first file.
    • dif2 indicates that the entry occurs in both files and shows the value found in the second file.
    • ins1 indicates that the entry only occurs in the first files and shows the value found.
    • ins2 indicates that the entry only occurs in the second file and shows the value found.
  • Only named items with the same name, type and dimension will be compared in the diffile.gdx output. Named items that are new or are deleted will only appear in the standard output summary report

See more Example on inspecting contents with GDXDIFF.

Writing a GDX file

Writing of GDX files in a GAMS model can be done during the compile phase or the execution phase. A GDX file can also be written after compilation and execution.

Compile Phase

During compilation, we can use a group of Dollar Control Options to write data to a GDX file. Writing during the compilation phase also allows us to define the elements of a set and the subsequent use of such a set as a domain. The following Dollar Control Options are available for writing data to a GDX file into GAMS during compilation of a GAMS model:

$gdxout

The $gdxout command is used in a sequence either to unload specified items to a GDX file or to close the specified GDX file. The statement with $gdxout can be used in one of the following forms:

$gdxout file_name 
$gdxout 

where:

Parameter(s) Description
file_name Specify the name of the GDX file (with or without the extension .gdx; written to from the current working directory)
(no file_name) closes the GDX file

See Example 1 - Writing a GDX file and [CompTimeWriteTrnsportGDX] for examples on how to use $gdxout.

$unload

The $unload command unloads specified items to the GDX file. The statement containing $unload is preceded and succeeded by the $gdxout statements where

  • the preceding $gdxout specifies the GDX file name and opens the file.
  • the succeeding $gdxout closes the GDX file.

The statement with $unload command can be used in one of the following forms:

$unload 
$unload id1 id2 ... idn
$unload id1=gdxid1 id2=gdxid2 ... idn=gdxidn

where:

Parameter(s) Description
(no identifiers) Write all symbols to the gdx file
id1 id2 ... idn Write GAMS symbols id1, id2, ..., idn to the GDX file
id1=gdxid1 id2=gdxid2 ... idn=gdxidn Write GAMS symbols id1, id2, ..., idn with corresponding names gdxid1, gdxid2, ..., gdxidn into the GDX file

See Example 1 - Writing a GDX file and [unload1] - [unload9] for examples on how to use $unload. See also [CompTimeWriteTrnsportGDX] example.

Execution Phase

When writing data to a GDX file during execution phase the data in the GDX file will be the data present in the GDX file at the time that the statement is executed. The results of all prior calculations and the most recent solve for any model will be reflected. The statements to read data from a GDX file during execution phase are

execute_unload, execute_unload, and execute_unloadidx

The execute_unload statement replaces an existing file with that name; it does not add symbols to or replace symbols in an existing GDX file. Without specifying any identifier, all sets, parameters, variables and equations will be written to the GDX file.

The execute_unloaddi statement replaces an existing file with that name; it does not add symbols to or replace symbols in an existing GDX file similar to execute_unload, but also writes the domains of all unloaded symbols to the same file.

The execute_unloadidx statement requires that each symbol written is a parameter; each parameter must have a domain specified for each index position. These domains have the requirement that they are formed using an integer sequence for the UELs that starts at 1 (one). The domain names are changed to indicate the size of each domain. This information is used when reading the data back from the GDX file using $LoadIDX during compilation. Using the special domain names, the UELs for the domains can be recovered without writing the domains to the GDX file; see example below.

The statement with execute_unload can be used in one of the following forms:

execute_unload  'file_name' , id1, id2, ..., idn ;
execute_unload  'file_name' , id1=gdxid1, id2=gdxid2, ... ;
execute_unload  'file_name' , setid=* ;

The statement with execute_unloaddi can be used in one of the following forms:

execute_unloaddi 'file_name' , id1, id2, ..., idn ;
execute_unloaddi 'file_name' , id1=gdxid1, id2=gdxid2, ... ;
execute_unloaddi 'file_name' , setid=* ;

The statement with execute_unloaddidx can be used in one of the following forms:

execute_unloadidx 'file_name' , id1, id2, ..., idn ;
execute_unloadidx 'file_name' , id1=gdxid1, id2=gdxid2, ... ;
execute_unloadidx 'file_name' , setid=* ;

where

Parameter(s) Description
file_name Specify the name of the GDX file (with or without the extension .gdx; written to from the current working directory)
id1, id2, ..., idn Write GAMS symbols id1, id2, ..., idn into the GDX file
id1=gdxid1, id2=gdxid2 Write GAMS symbols id1, id2 with corresponding names gdxid1, gdxid2 into the GDX file
Note
  • when only file_name is specified without other parameters all GAMS Symbols will be written into the GDX file file_name.
  • The GAMS option gdxUELs controls which UELs are registered in file_name. With option gdxUELs = squeezed; (default) only the UELs that are required by the exported symbols are registered while all known UELs are registered if we set option gdxUELs = full;. See also [unload10].

See Example 2 - Writing a GDX file and [qp1x] on how to use execute_unload. See [unload10] on how to use execute_unloaddi. See Example 3 - Writing a GDX file on how to use execute_unloadidx.

Savepoint

A GDX file containing the marginals and levels for all variables and equations at the end of a solve will be created with the command line parameter, model attribute or option statement Savepoint. One can save the solution information from the last solve or from every solve. The points that are saved can be used to provide an advanced basis, integer program starting point or NLP starting point.

The basic command line form is:

gams mymodelname Savepoint=number

the model attribute form is

modelname.savepoint=number;

and the option statement form is

option savepoint=number

where

  • when number equals 1 a point gdx file is saved from the last solve in the GAMS model and the file name will be modelname_p.gdx where model name is the name of the model identified in the solve statement.
  • when number equals 2 a point gdx file is saved from the every solve in the GAMS model and the file name will be modelname_pnn.gdx where model name is the name of the model identified in the solve statement and nn is the internal number of the solve. Thus if 10 solves occur one will get 10 files named modelname_p1.gdx through modelname_p10.gdx.

The following example:

model firm / all /;
firm.savepoint=1;;
solve firm using LP maximizing objfun;

saved a point gdx file firm_p.gdx.

and:

model transport /all/ ;
option savepoint=2;
set newseattle /s1,s2/;
parameter datador(newseattle) /s1 350, s2 450/;

loop(newseattle,
    a("seattle")=datador(newseattle);
    Solve transport using lp minimizing z ;
);
Display x.l, x.m ;

saved two point gdx files transport_p1.gdx and transport_p2.gdx.

Note
  • the GDX point file contains numerical records for all variables and equations giving just their levels and marginals. In a non point GDX file information on bounds and scales are also present, if defined.

Example 1 - Writing a GDX file

This example has modified the file trnsport.gms from [TRNSPORT] by adding the following statements after the last line.

...[TRNSPORT]...

d(i,j)=d(i,j)*10;
 
$gdxout tran
$unload i j
$unload d
$unload f
$unload b=dem a=sup
$gdxout

This example creates the GDX file tran.gdx containing the data for the sets i and j as well as the parameters d, f, a and b during the compile time, when a and b have been renamed in the GDX file to dem and sup. Also note the parameter d will not have been multiplied by 10 but rather take on their compile time value.

An $unload statement above can specify more than one symbol to be writtten to a GDX file and the similar result could also be accomplished using:

...[TRNSPORT]...

d(i,j)=d(i,j)*10;
 
$gdxout tran
$unload i j d f b=dem a=sup
$gdxout

Example 2 - Writing a GDX file

This example has modified from the file trnsport.gms (from [TRNSPORT]) by adding the following statement right after the solve statement.

...[TRNSPORT]...

Solve trnsport using LP minimizing Z;
execute_unload 'results.gdx', i, j, z, x;

After solving the model, the sets i and j and the variables z and x with all the data available after the sole.

Example 3 - Writing a GDX file

This example shows the use of the indexed write and read data during execution phase:

Set I /1*100/,
    J /1*50 /;
parameter A(I,J) /1.1=11, 1.9=19, 10.1=101/;
 
execute_unloadidx 'data.gdx', A;

Viewing the file data.gdx in the gamside shows the modified domain information:

gamsidegdx.gif
gamside showing data.gdx note the modified domains

To read from data.gdx, we use the indexed read:

Set I,J;
parameter A(I,J);
* load the data
$gdxin data.gdx
$loadidx A
$gdxin
* write all symbols so we can inspect in the gamside
$gdxout test.gdx
$unload
$gdxout
 
execute_unloadidx 'data.gdx', A;

Viewing the file test.gdx in the gamside shows that the domains have been populated:

gamsidegdx2.gif
View test.gdx in the gamside

Writing a GDX file after compilation or execution

A GDX file containing all data items resident at the end of the run of a GAMS code can be created using the gdx command line option either via GAMS call at the command line or via the command line parameter box in the IDE. This will cause all sets, parameters, variables and equations to be written to the GDX file.

For example:

gams mymodelname gdx=gdxfile_name

Or

gams mymodelname action=c gdx=gdxfile_name

where

  • mymodelname specifies the name of the GAMS file
  • gdxfile_name gives the file name and possible path where the GDX file is to be retained. When no path is specified the default directory is the current working directory or project directory in the IDE as seen below.
  • action=c indicates request to write a GDX file after compilation only
  • setting GDX to the string "default" (ie gdx=default) causes GAMS to create a GDX file with the GAMS file root name and a GDX extension. Thus
    gams trnsport gdx=default 
    will cause GAMS to write the gdx file trnsport.gdx.

When the IDE is used, the GDX file creation is invoked by an entry in the upper right hand corner of the IDE screen as illustrated below:

gdx_cmd_ide.png
Note
  • When this option is used the GDX file is created just at the end of the GAMS execution so the data written will contain the current values for all sets, parameters, variables and equations that are on hand at the end of the GAMS job.
  • The GDX data for the variables and equations contains the levels, marginals, lower bounds, upper bounds and scales for each item.
  • This yields a file that may be automatically opened in the IDE by doing a mouse click on the highlighted line in the IDE process window.

Using the gdx command line parameter when running the model using the IDE, the process window will show the GDX file_name in blue indicating that the file can be opened using a double-click with the mouse. See also Inspecting contents with IDE.

General notes on GDX files

There are several things worth noting about GDX files:

  • When working with GDX only one GDX file can be open at a time.
  • When reading data from a GDX file, the symbol to be read must be declared before the reading statement appears.
  • When the GDX file to be written has the same name as an existing GDX file the existing file will be overwritten. The resultant file will only contain the new data; there is no merge or append option.
  • The $unload command to write a GDX during compile time using the will only write out data defined in the compilation at the point where the command appears. No results of any solves or calculations done within the current GAMS program will be reported with ` $unload. This is not true when using the execute_unload or execute_unloaddi commands.
  • Both execute_unload and execute_unloaddi will write out data defined in the execution sequence at the point where the command appears. The results of the most recent solve command and any parameter calculations occurring before the GDX write will be reported.
  • Any subsequent execute_unload or execute_unloaddi to a GDX file written earlier will totally overwrite that file so care must be taken to write all wanted information in the last appearing execute_unload or execute_unloaddi
  • A command line GDX write using the gdx=file_name command line parameter will write out data defined at the end of the execution sequence. The results of the most recent solve and any parameter calculations will be reported.
  • When loading data note that domain checking will not be enforced so that when items are resident in the GDX file for set elements not present in the current file these items will be ignored. GAMS will not generate any message to tell you items are ignored.
  • Option Savepoint and execute_Loadpoint provide a GDX way of saving and loading a basis.
  • The contents as they differ between GDX files can be examined with GDXMERGE or GDXDIFF/
  • GDX files as of version 22.3 are written in compressed form unless the environment variable GDXCOMPRESS is set to zero.
  • Starting with GAMS version 22.3, gdx files can be written in a compressed format unless the environment variable GDXCOMPRESS is set to zero. A value of 1 indicates compression.
  • Compressed GDX files are not readable by older GAMS versions but the utility gdxcopy allows one to transform to older versions.
  • GDX files from diffent GAMS verion can possibly be incompatible due to compression among other changes. A current GAMS system can read all older GDX file formats. GDX files can be converted to a compressed format or an older format using GDXCOPY.
  • Users can also write their own programs accessing data in a GDX file via the expert-level GDX API using gdxdclib library. The interface and usage for this library is described in a separate document; see gdxioapi.chm or gdxioapi.pdf.

GAMS Data eXchange Tools

A number of GDX based tools and GDX related tools are included in GAMS distribution and maintained by GAMS. See also Tools Manuals.

  • The data exchange tools provide functionality to exchange data between GAMS and other data sources.
  • The GDX service tools operate directly on GDX containers.
  • Some of Data transfomration tools perform very specific data transformation tasks that are either awkward or inefficient to implement in GAMS directly.

Some utilities are avaiable only on specific platform. See Supported Platforms for more details.