[ Home | Support | Sales | Solvers | Documentation | Model Libraries | Search | Contact Us ]

seders.gms : ERS Data Manipulations with SED

Data files are often not in the form that can be read by another
program and/or transformations have to be made. The example below
shows how to manipulate an input output matrix in comma delimited
form. A sample file is shown below:

BAS1,1 Dairyfarmprd,2 PoultryEggs,3 Meatanimals,4 livestckmisc, ... 512 Wat2,513 Air2,Total,
1 Dairyfarmprd,0,0,265.7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, ... 0,0,0,0,0,0,0,0,0,26452.6,
2 PoultryEggs,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,523.8,0,0, ... 0,0,0,0,0,0,,17623.5,
...
503 Air2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1.2,0.7,1.7,0.3, ... 8,18859.6,470.4,0,0.5,5.4,33699.3,
Total,18728.2,14814.9,50839.5,2670.9,2693.7,3421.0,1831, ... ,3593.4,12052.2,6520145.5,

This file is the result of a routine process to take a raw file from a
web site and attach custom labels to rows and columns
( http://www.bea.doc.gov/bea/dn2/i-o_benchmark.htm ).  The rows and
columns have the form '<number> space <text>'. Some of the rows have
been dropped and the <number> on the row labels does not match the
corresponding column label.  Note that the row named Air2 has number
503, whereas the column Air2 has number 513.  Since the aggregations
are done by column numbers, the row numbers need to be reassigned to
use the column order. Furthermore, there are three row labels that
have no matching column labels.

Finally, the matrix needs to be aggregated from a size of 513 to a
much smaller size.

We use the unix tool SED to manipulate the text files into a format
suitable to be read by GAMS. For more details on SED please an SED
book.

• Somwaru, A, Problem contribution, ERS, US Department of Agriculture, Washington, DC.
• Doufherty, D, and Robbins, A, SED and AWK. O'Reilly & Associates, Sebastobol, 1997.