Runing the Test Cases Using Scripts

3.2. Runing the Test Cases Using Scripts

Scripts are provided for both Windows and Unix users to exercise most of the major functions of the software. In the scripts directory, the Windows versions of the scripts end in .bat and the Unix versions end in .csh. If you cd into the scripts directory, all of the scripts should run without any editing. However, if you wish to run any of the scripts without first cd-ing into the scripts directory, edit the BASDIR setting for all of the scripts in the directory (e.g., generate_surrogates, compare_surrogates) to specify the full path to your installation folder.

A set of example scripts is provided with the software to assist users with getting started using the system. These example scripts are provided only to serve as examples - the user is responsible for providing actual input data for his/her application and customizing the scripts accordingly. The example scripts generate a set of surrogates for an 8-km grid over Nashville, Tennessee. The input files for the examples were derived from files available from the EPA at ftp://ftp.epa.gov/EmisInventory/emiss_shp2003/. Some of the files are used as is, while others are extracted for Tennessee from the larger national data sets due to their size. The only file that was not from this web site is the cnty_tn shapefile that contains the polygons for the counties of Tennessee. This file was obtained from an independent source of county data.

The following surrogates are generated in the examples:

Airports
Land Area
Ports
Navigable Water
Major Highways
Housing
Population

NOTE that these are examples only and should not be considered a complete set of surrogates with which to do emissions modeling! However, a complete set of input files to reproduce EPA's 66 surrogates using the Surrogate Tool is available with the Surrogate Tool distribution.

The following are a few test scripts included in the release:

To test the basic operation of the surrogate creation program (i.e., generating surrogates for point, line, and polygon datasets), run the generate_surrogates script (.csh for UNIX, .bat for Windows). This will generate a set of test surrogates in the output directory, in addition to some gridded shapefiles that can be viewed in a GIS tool. Note that some warnings will be generated for counties on the edge of the grid since some of their surrogates will not sum to 1.
Run the compare_surrogates script (.csh for UNIX, .bat for Windows) to compare your surrogates with the reference surrogates. Note that compare_surrogates uses the diffsurr utility to compare two sets of surrogates with respect to a specified tolerance. You may find the diffsurr utility useful, for example, when you are creating new surrogates and would like to compare them to the previous versions. Diffsurr takes its arguments from the command line. An example of its usage follows; note that 3 is the numeric ID for the surrogates in the two files (i.e., both surrogates are numbered 3 in this case), although that may not always be true.
```
    diffsurr ..\data\srg_nash_ref.txt 3 ..\output\my_srgs.txt 3 0.00001
    
```
Examine the outputs of the program in the output directory.
- The surrogate file (srg_M_08_99NASH.txt) includes comments preceded by a ! on each line that contains a surrogate fraction. Each comment shows the surrogate numerator and denominator that were used to compute the fraction, and a running sum of the surrogate fractions for that county. For counties entirely within the grid, the sum should be 1; for other counties, the sum should not exceed 1.
- Since the OUTPUT_FILE_NAME variable was specified for each surrogate created with the generate_surrogates script, shapefiles were created that contain the sum of the surrogate fraction numerators for each grid cell. The names of these files start with grid (e.g., grid_airpt_M_08_99NASH.shp), and they are in the map projection of the output grid. A corresponding .csv file is also created that just contains the row, column, and sum for the grid cell. There will be quite a few zeros in these files because only data for Tennessee was processed. For the airports surrogate, many additional grid cells will be zero because many cells do not contain airports.
To see how to change the map projection of a Shapefile using the CONVERT_SHAPE mode of the allocator program run the convert_shape script as follows:
```
convert_shape ..\data\tn_ports ..\output\tn_ports_ll
```
The new tn_ports_ll file will have coordinates in a latitude-longitude map projection. The tn_ports_ll file should be easily visualizable by GIS tools. Note that the ports gridded shapefile will not have a nonzero value in each grid cell that has a port because BERTHS was used as the weight and many ports in the weight file have a value of 0 for the BERTHS attribute. To make all grid cells with ports nonzero, use NONE as the value for the WEIGHT_ATTR_LIST when creating the ports surrogate.

(Optional) If you require a mapping tool for shapefiles (e.g., for quality assurance of surrogates), you may want to consider ArcExplorer by ESRI, which is downloadable for free from the ESRI web site (http://www.esri.com).
To see how to filter a shapefile using the FILTER_SHAPE mode of the allocator program, run the filter_shapefile script. Examine the outputs that start with filtered_ in the output directory to see the files that it created. Also, to understand what filter was applied, examine the file county_filter.txt in the data directory. For additional examples of filters, look in the testsuite/filters directory.

To see how to generate a filtered surrogate, run the filtered_surrogate script. Examine the outputs that start with filtered_pop and filtered_srg in the output directory to see the files that it created. Also, to understand what filter was applied, examine the file filter_tn_pop.txt in the data directory. For additional examples of filters, look in the testsuite/filters directory.

To see how to generate a surrogate using a weight function, run the weighted_surrogate script. Examine the outputs that start with weighted_ in the output directory to see the files that it created. Also, to understand what weight was applied, look for the WEIGHT_FUNCTION line in the weighted_surrogates script. Also note that the ATTR_WEIGHT is set to USE_FUNCTION.

To see how to generate new surrogates as functions of existing surrogates, cd to the top installation folder, which should contain srgmerge.exe. Now type the following:
- If on Windows, type srgmerge data\halfPopHalfHousWin.txt;
- If on Unix, type srgmerge.exe data/halfPopHalfHousUnix.txt.
(Note the only difference between these halfPopHalfHous files is the direction of the slashes for the file names.) A file named merged_surrogates.txt should appear in the output directory. The fractions in these file should be 50% of the value for the population surrogate plus 50% of the value for the housing surrogate.

To see how to gap fill surrogates with other surrogates, cd to the top installation folder, which should contain srgmerge.exe. Now type the following:
- If on Windows: srgmerge data\gapfillAirportsWin.txt
- If on Unix: srgmerge.exe data/gapfillAirportsUnix.txt
Note that the only difference between the gapfillAirports files is the direction of the slashes for the file names. A file named gapfilled_surrogates.txt should appear in the output directory. It has a set of new surrogates that are the values from the airports surrogate where available, and values from the population surrogate where there were no values for the airport surrogate. In addition, the population and airports surrogates should be in the output file unchanged from their original state.

Examples of how to use the ALLOCATE and OVERLAY modes of the allocator program are given in the General Spatial Allocation and Overlays section.

To convert any DBF file to a .csv file, go to the main installation directory and type:

dbf2asc input_DBF_file output_csv_file

To see how to create inputs to SMOKE biogenic emissions processing run the script convert_beld.csh. This script runs the program beld3smk. Prior to running the script, you need to download and uncompress the input BELD3 data for the Tennessee area (88MB compressed, 1GB uncompressed). This is available from After running convert_beld.csh, the script compare_beld.csh. This script runs the diffioapi utility to compare the outputs to a reference data set and confirm that the results are the same. Note: The permanent location for the 24 BELD3 tiles in netCDf format has not been determined as of July 9, 2005 due to the large size of the data. This page will be updated once it has been determined.

A more comprehensive testsuite with additional tests and examples can be optionally downloaded.

To Section 3.3: Environment Variables Used by the Software