Essence of the small to large approach

The small to large approach is motivated by what we did not have to do in the above example.  One can develop a model with a small data set and once the structure is right it can be used without change with larger data sets.  Thus we have built the last transport model algebra we ever need to!

GAMS allows the same model structure, calculations and report writing to be developed, implemented, tested and debugged using a small data set instead of having to put up with the size, cumbersomeness, and speed degradation inherent in working with the large data set.  Thus, the golden rule of GAMS model development and debugging is

work from small to large

One should to the extent possible develop, debug and augment model codes using a representative, but purposefully small, data set.

The larger the model the longer everything takes.  This includes solution, compilation, generation, editing etc.  Generally, time expands exponentially.  Often frustration will result even when one is trying to find some relatively small data problems.  Using a reduced data set permits intimate examination of model structure and function.  After such verification, later expand to full problem context.  But now what about the argument "I can't go back to the small data set".

Using a small model is psychologically difficult to do.  Once a model is developed it is tempting to use the large data set rather than simplify.

Consider one of my mistakes.  I have a linked agriculture - forest sector model used to do a greenhouse gas offset studies.  Once I had to make small modifications to the constraints and report writing.  After each modification the full model study was rerun covering 100+ scenarios taking 2½ days.  I assured myself I didn't need to use a small model to test modifications (figuring I was a perfect modeler).  Well I ended up running the 2½ day analysis 6 times, each time fixing a bug.  Finally I went back to a small model and got things right in 2 hours.  Sheer laziness and arrogance (I knew the model was right - 5 times over) that led me to not initially check things out using a small model, and lengthened the project by at least ten days.