GDXMERGE

The program GDXMERGE combines multiple GDX files into a single GDX file. Symbols with the same name, dimension and type are combined into a single symbol of a higher dimension. The added dimension has the file name of the combined file as its unique element.


Usage

gdxmerge filepattern1 filepattern2 .... filepatternN {options}

Each file pattern represents a file name or a wildcard representation using ? and *. A parameter of the form @filename will process the commands from the text file specified. The result of the GDXMERGE execution will be written to a file called merged.gdx, unless this default is overwritten by the output option.

The .gdx file extension can be omitted. Files without a full path name are assumed to be in the current directory when using a command prompt. When using the GAMS IDE, these files are assumed to be in the current project directory.


Options

The following options can be specified:

id = <ident1>, <ident2>... (default = none)

Only merge the symbols ident1, ident2, ...

exclude = <ident1>, <ident2>... (default = none)

Merge all symbols except for ident1, ident2`, ...

big = <integer>

The size for big symbols.

output = fileName (default = merged.gdx)

The optional output file name.

All symbols with matching type and dimension will be merged. By specifying the parameter id=ident1 the merge process will only be performed for the identifier(s) specified, while exclude=ident1 indicates that all symbols should be merged except for the ones specified in the exclude list. Note that the two options id and exclude are mutually exclusive.

By default, the program reads all GDX once and stores all data in memory before writing the merged.gdx file. The big parameter is used to specify a cutoff for symbols that will be written one at a time. Each symbol that exceeds the size will be processed by reading each GDX file and only process the data for that symbol. This can lead to reading the same GDX file many times, but it allows the merging of large data sets. The formula used to calculate the cutoff is:

dimension * totalNumberOfElements.

The calculated value is doubled for variables and equations.

In addition to the symbols written, a set is added to the GDX file representing all the files processed during the merge operation. The name of the set is Merged_set_1, and is made unique by changing the number. The explanatory text for each set element contains the date and time of the GDX file processed.

Note
  • The file merged.gdx, or the file specified with the output parameter, will never be used in a merge operation even if the name matches a file pattern.
  • Symbols with dimension 20 cannot be merged, because the resulting symbol will have dimension 21 which exceeds the maximum dimension allowed by GAMS.


Examples

Merging several GDX Files

In this example, we solve the [trnsport] model from the GAMS Model Library using different LP solvers. After each run, we write all symbols to a GDX file and merge the files into one file. The variable X is read from the merged file and displayed.

$call gamslib trnsport
$call gams trnsport lp=bdmlp gdx=bdmlp
$call gams trnsport lp=cplex gdx=cplex
$call gams trnsport lp=xpress gdx=xpress
$call gams trnsport lp=conopt gdx=conopt
$call gams trnsport lp=minos gdx=minos
$call gams trnsport lp=snopt gdx=snopt
$call gdxmerge *.gdx

Variable AllX(*,*,*);
$gdxIn merged.gdx
$load AllX=X
$gdxIn

option  AllX:5:1:2;
display AllX.L;

The display statement generates the following output in the listing file:

----     22 VARIABLE AllX.L  shipment quantities in cases

           seattle     seattle   san-diego   san-diego
          new-york     chicago    new-york      topeka

bdmlp     50.00000   300.00000   275.00000   275.00000
conopt               300.00000   325.00000   275.00000
cplex     50.00000   300.00000   275.00000   275.00000
minos     50.00000   300.00000   275.00000   275.00000
snopt     50.00000   300.00000   275.00000   275.00000
xpress               300.00000   325.00000   275.00000

Note that the different solutions are combined into a single symbol of a higher dimension. The filenames (resp. the solver used) are added as unique elements.
Instead of using the display statement, we can also use the GAMS IDE or GAMS Studio to display the merged.gdx file. The following figure shows the contents of merged.gdx after selecting the level subfield of the variable to be displayed and arranging the display:

gdxmerge1.gif
Output file merge.gdx displayed in GAMS Studio

This example is also part of the GAMS Data Utilities Library, see model [GDXMERGEExample17] for reference.