Peer Reviews: Status and Reports

Status of TMIP Peer Reviews — Summary Report: East-West Gateway Council of Governments Travel Demand Model Peer Review

Location:: St. Louis, Missouri
Date:: December 7-8, 2006
Peer Review Host Agency:: East-West Gateway Council of Governments (EWGCG)
Contact:: Lubna Shoaib, Division Manager, Systems Evaluation; lubna.shoaib@ewgateway.org; (314) 421-4220
Peer Review Panelists:: Chandra Bhat, University of Texas at Austin
David Boyce, Northwestern University
Ken Cervenka, North Central Texas Council of Governments
Guy Rousseau, Atlanta Regional Commission
Frank Spielberg, Vanasse Hangen Brustlin, Inc.

List of Abbreviations and Acronyms

AADT -: average annual daily traffic
EWGCG -: East-West Gateway Council of Governments
FTA -: Federal Transit Administration
LEAM -: Land Use Evolution and Assessment Model
LUAM -: land-use allocation model
MPO -: metropolitan planning organization
TAZ -: traffic analysis zone
TMIP -: Travel Model Improvement Program
VDF -: volume-delay function
VMT -: vehicle miles traveled

Executive Summary

On December 7 and 8, 2007, the East-West Gateway Council of Governments (EWGCG), the metropolitan planning organization for the St. Louis area, hosted a travel demand model peer review meeting. The two-day peer review was held as part of the Travel Model Improvement Program, sponsored by the Federal Highway Administration. The peer review panel consisted of five travel demand modeling experts from around the country. The primary purpose of the peer review was to advise the EWGCG on model fixes and enhancements so that it can be used for its spring 2007 compliance review.

EWGCG asked the panelists to address the following aspects of the model:

Assess the sufficiency of the model, especially validation criteria and mode choice constants.
Identify probable causes of problems and potential solutions, especially for transit constants.
Comment on the use of K factors.
Comment on the volume-delay function.
Suggest short- and long-term model enhancements.

Based on a day-long presentation and discussion of EWGCG's model, and on material that EWGCG sent to panelists before the meeting, the peer review panel's main recommendations were:

To help diagnose model validation problems, validation should be done for individual model components and for the entire model solved with feedback.
The number of traffic analysis zones (mentioned by one panelist) and market segments adds a lot of complexity and run time to the model and probably does not enhance its accuracy very much.
Validation criteria should be:
- Root mean square error less than 40 percent
- R² greater than 90 percent
Avoid overuse of K factors in model calibration.
Carefully examine transit mode shares, assignment, trip lengths, and path choices to try to diagnose the problem causing the unacceptably high rail constant.
Continue to pursue enhanced land-use model.

1. Background

The East-West Gateway Council of Governments (EWGCG) is the federally designated metropolitan planning organization (MPO) for the St. Louis, Missouri metropolitan area. Its planning area comprises Madison, Monroe, and St. Claire Counties in Illinois, and Franklin, Jefferson, St. Charles, St. Louis, and the City of St. Louis Counties in Missouri. The Mississippi and Missouri Rivers both run through the metropolitan area, meeting at the northern tip of the City of St. Louis—the historic "Gateway to the West." With its population of approximately 2.4 million people, the St. Louis metropolitan area is the 18th largest in the United States, slightly smaller than San Diego and slightly larger than Baltimore. Table 1 shows some of its important demographic and transportation-related characteristics.

Table 1 Characteristics of St. Louis Metropolitan Area
Characteristic	Number
Households	968,533
People	2,428,730
Persons/household (average)	2.51
Workers	1,173,772
Workers/household (average)	1.22
Vehicles	1,637,553
Vehicles/household (average)	1.69

Figure 1 shows a map of the EWGCG planning area.

Map of the EWGCG planning area which comprises Madison, Monroe, and St. Claire Counties in Illinois, and Franklin, Jefferson, St. Charles, St. Louis, and the City of St. Louis Counties in Missouri. The Mississippi and Missouri Rivers both run through the metropolitan area, meeting at the northern tip of the City of St. Louis.

Figure 1 EWGCG planning area

Among EWGCG's responsibilities are the development and maintenance of a regional travel demand model. In addition to satisfying federal mandates, the model is used for project selection, alternatives analyses, truck-traffic models, and the study of land-use and investment impacts. The current travel demand model is a traditional four-step model that was initially developed with use of the MINUTP comprehensive travel demand modeling system based on travel surveys conducted in the mid-1960s. In 1997, the model was revalidated for the base year 1990. Trip rates were based on a small travel survey undertaken that year, supplemented by earlier data on attraction rates and 1990 U.S. Census journey-to-work data.

New transportation legislation and improved computational power have resulted in the need and opportunity to apply new travel demand forecasting techniques to transportation-related issues both regionally and locally. To fulfill these expected obligations, EWGCG decided to develop a new travel demand model. In 2002, EWGCG conducted a household survey that formed the basis for the new model. This effort was paralleled by an on-board passenger survey, which was conducted to observe and document transit travel patterns.

Upon completion of the household survey, PB Consult, Inc., was hired to develop the new model. EWGCG and PB Consult chose a singly constrained destination-choice model running on a Cube platform. The model uses TRNBUILD for the multimodal transit system.

EWGCG's highest priority is model validation. Validation must be complete May 2007 so that the model can be used in the upcoming Transportation Improvement Program and Long-Range Plan analyses and in air-quality conformance determinations. As of this writing, model development was largely complete. However, validation revealed some important discrepancies between modeled and actual volumes. Modeled volumes on interstate screenlines and on the bridges crossing the Missouri and Mississippi Rivers are higher than actual volumes, whereas modeled volumes on other freeways are generally low. Overall, the modeled vehicle miles traveled (VMT) are low compared with actual VMT.

EWGCG is also concerned that the transit model might not meet Federal Transit Administration (FTA) New Starts criteria. After the model is validated, EWGCG wants to develop an improved truck model.

As part of its effort to develop and implement a new travel demand model, EWGCG invited a panel of travel demand modeling experts from around the country to evaluate the model. The peer review panel was sponsored by the Federal Highway Administration Travel Model Improvement Program.

EWGCG requested that the peer-review panel:

Assess the sufficiency of the model, especially validation criteria and mode choice constants.
Identify probable causes of problems and potential solutions.
Comment on the use of K factors.
Comment on the volume-delay function (VDF).
Suggest short- and long-term model enhancements.

2. Model Summary

2.1 Population and Land-Use Forecasts

Most of the changes in population and land use in the St. Louis area represent a redistribution of existing resources rather than net growth. Population growth in the region is approximately 4 percent, which is approximately the 36th percentile relative to growth in other U.S. metropolitan areas. The most notable population shift is from the urban core to the suburbs west of St. Louis.

EWGCG employs a land-use allocation model (LUAM) to generate population forecasts for each traffic analysis zone (TAZ). First, the model generates county-level forecasts using a cohort-survival method for natural growth. Since the net in- and out-migration in the region is essentially zero, the model ignores migration. Estimates are then adjusted on the basis of development plans and expert opinion.

Next, each TAZ is assigned an "attractiveness score" on the basis of its developable land and its proximity to "attractions" such as employment centers and major intersections. Finally, the model adjusts the estimates generated in the first step on the basis of attractiveness scores to allocate anticipated population and employment growth from the county level to the TAZ level. The allocation algorithm assigns population and employment growth in proportion to the relative attractiveness of each TAZ, with the constraint that neither variable can exceed the available land.

EWGCG is working with the University of Illinois at Urbana-Champaign to develop a new, integrated, land use-transportation model, the Land Use Evolution and Assessment Model (LEAM). This model disaggregates land into cells of 30 x 30 meters. The cells form the basis for land-use estimates, which are made according to characteristics of the cell such as development plans, distance to an expressway, and major population or employment centers. LEAM's premise is that the probability of development decreases in proportion to congestion, which in turn drives development to adjacent areas through a feedback loop between the land-use and travel models.

2.2 Data

In 2002, EWCGC conducted a travel survey of 5,094 households. The survey included household residents' personal 24-hour travel journals. These surveys generated data on the following variables:

Trip purpose
Mode
Household automobile ownership
Household size
Household income group
Number of household residents employed
Origin and destination zones
Time of day (peak or off-peak)

In the same year, a transit-on-board survey was conducted. Passengers over 16 years old on all fixed-route transit services were given a survey form to mail in. The 68 percent response rate comprised 13,535 bus-service and 1,786 rail-service respondents.

Data on these variables were then fed into the model. Peer review panelists were impressed with the amount and detail of the available data. They thought that EWGCG could take better advantage of such data to diagnose problems and enhance the model.

2.3 Highway Network

The highway network encompasses 8,144 center-lane and 18,509 lane miles. The network generally goes down to the collector level, although it contains a few smaller roads to accommodate the transit network. The highway network has 25,565 links, 14,916 centroid connectors, and 68 external stations.

The network includes posted speed limits, number of lanes, distance, functional class, and average annual daily traffic (AADT). The distances for all centroid connectors for a zone represent the average distance required for a person to travel in or out of a zone. Coding the centroid connector's distance is a manual process.

The free-flow speed is equal to the posted speed limit. Panelists expressed some reservations about this approach, suggesting it might lead to underestimation of free-flow speeds on freeways and overestimation on arterials.

The model estimates lane capacity is based on level-of-service E, using design criteria from the 2000 edition of the Highway Capacity Manual. Capacity estimates are based on functional class, area type, posted speed, and number of lanes. Lane capacity estimates ranged from 700 vehicles per hour per lane for a principal arterial in the downtown area to 1,300 vehicles per hour per lane for a principal arterial in an urban area outside the central business district. Panelists thought that the estimated capacities were too high compared with the state of the practice.

2.4 Transit Network

The St. Louis area transit network comprises three modes: local buses, express buses, and MetroLink light rail. There is also a placeholder for a future transit mode. The network has 480 one-way lines: 268 for morning peak travel (6 to 9 a.m.) and 212 for off-peak travel (9 a.m. to 2 p.m.). Transit skims are grouped by period, access mode, and mode group. The mode groups are:

Local bus
Local bus → express bus
Local bus → express bus → light rail

Bus headway is calculated as:

One panelist expressed concern that this headway calculation, which is not used for MetroLink, might not comply with FTA New Starts requirements.

Bus speeds are a function of link speed adjusted for dwell time at stops. MetroLink speed is hardcoded. When generating skims, walk access and egress are limited to one-half mile and walk transfers to one-tenth of a mile per transfer. Total transfers cannot exceed one-quarter of a mile or five minutes. Automobile travel to Park and Ride Lots is limited to 15 miles or 30 minutes.

2.5 Traffic Analysis Zones

The model has 2,527 TAZs, up from 1,398 in the previous model. One panelist thought that this might be too many TAZs, as they add run time to the model without substantially increasing its accuracy. Other panelists disagreed and thought that the number of TAZs was appropriate for the model.

About two-thirds of TAZs are less than 500 acres in area and have less than 1,000 residents, based on data from the 2000 U.S. Census. The TAZs are aggregated into 35 districts for the purpose of summarizing model outputs and conducting reasonableness checks. The model has six area types—rural, suburban, urban, core, business and entertainment, and central business district—that are used for calibration.

2.6 Trip Generation¹

The model uses a cross-classification trip production technique that calculates productions using household size and automobile availability. There are a total of 17 trip purposes. For home-based work trips, the model also uses the number of workers in a household and household income group. In addition to the core calculations of productions, several submodels are employed to provide information necessary to support the trip production calculations. These submodels are:

Area type
Automobile ownership
Household size distribution
Household worker distribution
Household income distribution
Joint distribution

This results in 61 market segments. Some panelists thought that the high number of market segments might overcomplicate the model without sufficiently enhancing its accuracy.

The trip attraction model is based on a set of linear equations using aggregated zone-based socioeconomic data, which generate independent estimates of attractions. Employment and household data are used as attractor variables.

There are also three asserted models: airport trips, based on the model used in Minneapolis-St. Paul; university trips, using trip-generation rates based on data from North Carolina State University; and a truck model, based on a model used in Tampa Bay. External trips are fixed percentages for truck trips, through-traffic trips, work trips, and non-work-based trips.

2.7 Trip Distribution²

The trip distribution model uses a singly constrained, multinomial logit destination-choice formulation. The logsum from mode choice is used as the primary variable to determine impedance. The model also uses a series of standard gravity distribution models to estimate the distribution of special-purpose trips, including airport trips, truck trips, on-campus university trips, and external trips.

To better predict destination choice behavior, the model includes a distance variable and transformations of the distance variable (2nd and 3rd power, natural log). Other variables in the distribution model are dummy variables and associated constants for intrazonal trips, river crossings, intercounty movements, inter-state travel, and movements between specific destination and production area types. The destination choice model includes the following variables:

Relative attractions based on employment
Mode choice logsums
Distance impedance
Area type at production and attraction ends
Intrazonal factors
Illinois-Missouri crossing
County crossing
Income group (for home-based work trips)

2.8 Mode Choice

The mode choice model uses a nested logit structure comprising 13 mode alternatives and a future-mode alternative, as well as a joint mode choice/destination choice algorithm. Productions are distributed simultaneously to zones and are split into modes. Mode choice variables include:

In-vehicle time
Egress and access times
Transfer time
Wait time
Fare (stratified by income)
Operating cost (stratified by income)
Parking cost (stratified by income)
Bus travel time as a fraction of rail travel time (for trips involving transfers from bus to rail)
Automobile travel time as a fraction of transit travel time (for drive access to transit trips)

The mode choice nesting structure is shown in Figure 2 below.

Currently, the model generates very large mode-specific constants; light rail constants are high positive values, and local bus constants are high negative values. One of EWGCG's major concerns is that the transit constants might not satisfy FTA New Starts requirements. The panelists believed that more detailed analyses of data and estimates are needed to try to diagnose the problem.

2.9 Model Validation

The model validation checks have uncovered some consistent and important inaccuracies:

Interstate river crossings and screenline volumes are high.
Other freeway volumes are generally slightly low, although some isolated areas show substantial discrepancies.
Overall VMT is low.

Alternative-specific transit coefficients overestimate bus trips, reducing rail attractiveness.

Figure 2 Mode choice nesting structure

SR - shared ride
DA - drive alone
2P - 2 people in car
3+P - 3 or more people in car

3. Panel Recommendations

The peer review panelists found EWGCG's staff working on the travel demand model to be very skilled and motivated. This observation extended to the agency's senior management, who have shown strong interest in and support for improving modeling in the region. Additionally, the panelists noted that PB Consult's dedication to completing the model is impressive. Panelists were particularly impressed by the MPO's eagerness to pursue advanced modeling techniques, as demonstrated by the existing land-use model and the progress toward a new model that incorporates the potential land-use effects of congestion. Panelists' recommendations for improving the model are presented below.

3.1 Overall Issues

With regard to the overall state of the model, panelists offered the following comments:

There might be too many market segments, which, because of data limitations, add complexity to the model but probably do not increase its accuracy.
One panelist felt that the model has too many TAZs, which make it unnecessarily complicated and increases run time.
Appropriate levels of assignment and model feedback convergence will increase model run time.

3.2 Validation

EWGCG wants to complete model validation as quickly as possible so that it can roll out the new model for its spring 2007 compliance review. Model validation was addressed in several of the panel's recommendations:

Model validation should more carefully examine individual model components to better understand weaknesses and anomalies.
Relaxing unit constraint on log of size measure could be a "quick fix" for validation.

With regard to longer-term model validation, the panel suggested that:

Possible causes of overestimation of volume on river crossings should be examined.
- Look for problems with feedback convergence.
- Verify that vehicle occupancies on the crossings are correct.
- Determine the effect of truck traffic on bridges.
Validation criteria should be established.
- Weekday root mean square error (RMSE) should be less than 40 percent; however, "overfitting" with K factors should be avoided.
- R² for observed versus estimated volumes should be greater than 90 percent.

3.3 Volume-Delay Function

The panel thought that using posted speed limits as free-flow speed might lead to underestimation of freeway speeds and overestimation of arterial speeds. It considered the conical shape of VDF function to be consistent with the state of the practice.

3.4 Rail Constants

Panelists thought that the current calibrated rail constants were very high and could indicate a serious problem. They recommended that observed versus modeled transit shares be analyzed by geographic area and rail-availability market. Additionally, they suggested that observed versus modeled transit trip lengths and path choices be analyzed. Finally, they thought it might be helpful to revisit assignment of the transit on-board survey.

3.5 Highway Network

The panel noted that quality assurance and quality control checks should be performed on the highway network coding.

3.6 Truck Model

The panel recommended that the current truck model be checked against vehicle classification counts. Panelists thought that, moving forward, EWGCG should closely coordinate its truck model with statewide commodity flow modeling and that it should support a locally developed commercial vehicle model.

3.7 Long-Term Improvements to the Model

The panelists encouraged EWGCG to continue its work on a new land-use model. However, they stressed the importance of ensuring consistency between the travel and land-use models. In the panelists' view, a good land-use model can help the MPO to be more proactive in its land-use planning rather than to simply react to changes in land use.

Appendix A: List of Participants


Name	Affiliation*
Marty Altman	EWGCG
Chandra Bhat†	University of Texas at Austin
Jerry Blair	EWGCG
David Boyce†	Northwestern University
Ken Cervenka†	North Central Texas Council of Governments
Bob Donnelly	PB Consult, Inc.
Mike Henderson	Missouri Department of Transportation
Aaron Keegan	EWGCG
Brad McMahon	Federal Highway Administration, Missouri Division Office
Birat Pandey	EWGCG
Steve Ruegg	PB Consult, Inc.
Guy Rousseau†	Atlanta Regional Commission
Lubna Shoaib	EWGCG
Frank Spielberg†	Vanasse Hangen Brustlin, Inc.
Ann Steffes	U.S. Department of Transportation, Volpe Center

* EWCGC = East-West Gateway Council of Governments.
† Panelist.

Appendix B: Agenda

Agenda

Group: TMIP Model Review
Date: December 7 and 8, 2006
Subject: TMIP St. Louis Model Review
Location: East-West Gateway Council of Governments, Board Room

December 7, 2006
8:15 a.m.	Continental Breakfast
8:45 a.m.	Welcome and Introductions
9:00 a.m.	Purpose of the Review
9:15 a.m.	Intended Uses for the Model
9:30 a.m.	Land Use and Demographic Forecasting
9:45 a.m.	Household Interview Survey
10:00 a.m.	Transportation Network Development
10:30 a.m.	Break
10:45 a.m.	Big-Picture Issues: Validation (Traffic and Transit)
12:00 p.m.	Lunch
1:00 p.m.	Model Overview and Structure Trip Generation Distribution—Singly Constrained Transit Skims Mode Choice Traffic and Transit Assignment Feedback Criteria
5:00 p.m.	Adjourn
December 8, 2006
8:15 a.m.	Continental Breakfast
8:45 a.m.	Follow-up on Model Discussion
10:00 a.m.	Closed-Door Panel Discussion
12:00 p.m.	Lunch
1:00 p.m.	Panel Recommendations and Open Discussion
3:00 p.m.	Adjourn

Presentation

St.Louis TMIP Model Review (HTML or PDF )

¹ See PB Consult, Inc. (June 2005). St. Louis Regional Model, TranEval Trip Generation Model, Model Development Documentation, prepared for the East-West Gateway Council of Governments.

² See PB Consult, Inc. (2006). St. Louis Regional Model, TranEval Model Evaluation, prepared for the East-West Gateway Council of Governments.