Skip to Main Content
Peer Review Program
HomeUnited States Department of Transportation, HomeContact UsSite Map
Travel Model Improvement Program - TMIP
About TMIPTMIP ServicesClearinghouse, selectedConferences and CoursesContactsTravel Model DiscussionsTRANSIMSLinks

BROWSE

SEARCH

ORDER DOCUMENTS

NEWSLETTER


DOCUMENT NAVIGATION

Previous Section
Table of Contents
Next Section

Model Validation and Reasonableness Checking Manual

7.0 Assignment Procedures

Introduction

Assignment is the fourth and last major step of the traditional four-step process. This includes both highway and transit assignment of vehicle and person trips respectively. The assignment of trips to the network is the final output of the modeling process and becomes the basis for validating the model set's ability to replicate observed travel in the base year as well as to evaluate the transportation improvements in the future year(s). Depending on the level of analysis being done, the assignment can be to a regional highway and transit network for system-wide planning, or to a detailed network for a subarea or corridor study.

Historically, highway and transit assignment procedures were used primarily for systems analysis of large scale transportation improvements. A single volume-delay function for all facility type of roadways, the Bureau of Public Roads (BPR) curve, was used to estimate link travel times resulting from the assigned volumes. In recent years, a number of enhancements have been made to the process, due in part to increases in computing power. Volume-delay functions have been developed for different facility types (freeway versus arterial for example). The detail of the coding of the networks has increased dramatically, along with the associated reduction in the size of the traffic analysis zones. Better assignment algorithms (such as equilibrium assignment) and parameters have produced improved results.

The inputs for highway and transit assignments include the coded networks and the vehicle and person trip tables produced in earlier steps. The conversion of auto person trips to vehicle trips may be performed in the mode choice model or with simple auto occupancy factors. Time-of-day/direction split factors are typically used to convert the daily production-attraction trip tables into time-specific origin-destination trip tables.

In addition to assigning traffic by time-of-day, the traffic assignment process makes it possible to directly model the effects of tolls and other user costs on traffic volumes. Specifically, travel cost can be included in the calculation of travel impedance on roadways. The travel cost can be the cost to traverse a specified distance on a roadway (the vehicle operating costs), and/or it can be the cost of a toll. In both cases, unlike travel time and delay, the travel cost is relatively independent of the traffic volume.

An alternative to evaluating the impacts of tolls on highway demand using the assignment model is to incorporate a toll "path" in the mode choice model. The use of the toll path is a choice similar to the choice made to use transit or take competing transit paths.

The validation of the highway assignment is the final validation of the complete travel model set. Most assignment validation efforts have focused on obtaining accurate link volumes, because that has traditionally been viewed as the primary output of the assignment process. However, with the strengthening connection between travel models and air quality models, there has been a renewed interest in the congested speeds produced by the final iteration of the assignment procedure.

7.1 Highway Assignment

7.1.1 Impedance Calculations

Traffic assignments are dependent on the calculation of travel impedances. At the simplest level, the impedance is the travel time. As noted above, a more refined procedure is to incorporate both time and cost into the impedance calculation. Many trip distribution and traffic assignment models are based on this combined impedance measure. A common impedance unit is generalized costs. On non-toll links, the following equation is often used:

Equation 7-1

where:

Cost Total = total link impedance
Cost Distance = travel cost due to link distance
Cost Link Time = travel cost due to the time required to traverse the link

The cost of travel distance for traffic assignments has been calculated in other studies as roughly $0.10 per mile, accounting for gas and maintenance. However, this value can vary depending on geographical location and may need to be adjusted.

In order to implement the generalized cost function, the value of time from the mode choice model can be used as a basis to convert travel time to travel cost. Unlike mode choice, all trip purposes are combined in traffic assignment. As a result, weighted average values of time that considered the varying mixes of trip purposes by time-of-day are used in the time-of-day traffic assignments.

For toll facilities, the travel impedance for the toll link can be calculated as follows:

Equation 7-2

where:

Cost Total = total link impedance
Cost Toll = travel cost due to the toll
Cost Servicetime = travel cost due to the delay at the toll booth

The cost of tolls for traffic assignments will be calculated as the actual toll paid in dollars. The travel cost associated with the time spent paying the toll (deceleration, queuing, and acceleration) is computed by applying the same value of time described above to the "toll" time.

A primary method for calibrating and the subsequent validation of the highway assignment model is the adjustment of these generalized cost impedance calculations.

7.1.2 Volume-Delay Relationships

The traffic assignment process is driven by volume-delay relationships. As traffic volumes increase, travel speeds decrease due to increased congestion.

The state-of-the-practice in traffic assignment uses link-based volume-delay functions. The variables that control the final assigned travel speeds, the beginning or free-flow speed, and the link capacity are link based. Typically, free-flow speeds and link capacities are determined via a look-up table that relates these variables to the facility type or functional class of the link and the area type surrounding the link. As an example a look-up table of free-flow speeds and per lane link capacities is shown in Table 7-1. Such a look-up table approach was used in the Urban Transportation Planning Software (UTPS) distributed by the Urban Mass Transportation Administration in the 1970s and 1980s and, as a result, has become a commonly used approach to estimating link-specific, free-flow speeds and capacities.

Table 7-1
Look-up Table of Free-Flow Speeds and Link Capacities

Area Type Functional Class
Freeway Class 1 Arterial Class 2 Arterial Class 3 Arterial Collector
Urban Capacity
FF Speed		 2000
50		 1000
35		 870
25		 670
20		 470
15
Suburban Capacity
FF Speed		 2000
55		 1000
40		 870
35		 670
25		 470
20
Rural Capacity
FF Speed		 2000
60		 1000
45		 870
40		 870
35		 470
25

In addition to the use of look-up tables to estimate link-specific, free-flow speeds and capacities, the Bureau of Public Roads (BPR) function is the most commonly-used function for relating changes in travel speed to increases in travel volume. The BPR function is specified as follows:

Equation 7-3

where:

Tf = final link travel time
To = original (free-flow) link travel time
alpha = coefficient (often set at 0.15)
V = assigned traffic volume
C = the link capacity
beta = exponent (often set at 4.0)

Figure 7-1 shows the effect of the BPR function on travel time and travel speed with the "a" coefficient set at values of 0.15 and 1.0, and the "b" exponent set at 4.0. A one-mile long freeway link was used for the example. As can be seen, if the "a" coefficient is set at 1.0, the congested speed at a volume / capacity ratio of 1.0 is one-half of the free-flow speed. In addition, as can be seen in the figure, the travel times increase very slowly at volume / capacity ratios less than 1.0 and very rapidly (actually, exponentially) at volume / capacity ratios greater than 1.0.

Figure 7-1
Example Travel Times and Speeds Using BPR Function

Figure 7-1

The BPR function is not "well behaved" in equilibrium traffic assignments. At low volume/capacity ratios (i.e., less than 1.0), additional traffic assigned to a link has very little affect on the travel speed. However, at volume/capacity ratios greater than 1.0, additional traffic has an exponential effect on travel times. Thus, the BPR function can cause an equilibrium assignment to iterate to closure more slowly due to oscillation of travel times on highly congested links.

The parameters used with the BPR formulation of volume-delay should be updated to correct some of the weaknesses. Alan Horowitz's 1991 report for FHWA, "Delay-Volume Relations for Travel Forecasting Based on the 1985 Highway Capacity Manual", contains parameters which were fit to the speed/volume relationships contained in the Highway Capacity Software, Version 1.5. The coefficient of the BPR function was determined by forcing the curve to fit the speed/volume data at zero volumes (free-flow speed) and at capacity (LOS E). The second parameter was found by nonlinear regression. The updated BPR parameters are shown in Table 7-2.

Table 7-2
Updated BPR Parameters Using HCM Procedures

Coefficient Freeways Multilane
70 mph 60 mph 50 mph 70 mph 60 mph 50 mph
alpha 0.88 0.83 0.56 1.00 0.83 0.71
beta 9.80 5.50 3.60 5.40 2.70 2.10

The speeds shown in the above table are design speeds of the facility, not the free flow speeds. Capacities used in the v/c ratio are ultimate capacity, not a design capacity as used in the standard BPR curve. The curves based on the HCM exhibit a speed of about 35 mph at a v/c ratio of 1.0. This is consistent with standard capacity rules that the denser traffic flows occur at this speed. Note that the BPR curve has a much higher speed at a v/c equal to 1.0 than does the HCM curves.

The ultimate capacity used for these curves was 1800 vehicles per hour per lane for a one mile section. This value is the ultimate capacity for typical prevailing conditions, not those under ideal conditions which would have a capacity of 2000 vehicles per hour per lane (and even higher based on recent changes to the Highway Capacity Manual). The curves extend beyond the point where the v/c ratio is 1.0, or where the flow has reached capacity. In capacity analysis, this portion of the curve is considered unstable. However, for travel demand modeling, the curve must extend beyond 1.0 to account for the theoretical assignment of the traffic.

The calibration and validation of the assignment model includes both the systematic adjustment of any lookup speed and capacity tables as well as the adjustment of the coefficients of the volume-delay function, by facility type.

7.1.3 Validation Tests

The validation tests for highway assignment are presented at three levels; systemwide, corridor, and link specific. This increasing detail of validation tests is correlated to the step(s) in the model chain that could be the cause of the possible error(s).

There are several systemwide or aggregate validation checks of the auto assignment process. The checks are generally made on daily volumes, but it is prudent to make the checks on volumes by time-of-day as well. Systemwide checks include Vehicle Miles of Travel (VMT), Vehicle Hours of Travel (VHT), cordon volume summaries and screenline summaries. In addition to checking summations of VMT, VHT, and volumes, the average VMT and VHT per household and person should be checked.

Vehicle Miles of Travel (VMT)
Validation of the model using VMT addresses all major steps in the travel demand models including trip generation (the number of trips), trip distribution (the trip lengths), and assignment (the paths taken).

VMT validation is particularly important in urban areas that are designated by the Environmental Protection Agency (EPA) as non-attainment for moderate and serious carbon monoxide (CO). The EPA has published guidance for the forecasting and tracking of VMT as required by Section 187(a) of the Clean Air Act Amendments of 1990 (CAAA). This guidance should be read and understood by those developing travel demand models for these urban areas. The document can be found on the Internet at http://www.bts.gov/smart/cat/vmt.html. The Bureau of Transportation Statistics has an Internet home page at www.bts.gov and this is an excellent resource for all information relating to transportation statistics.

The first check is observed versus modeled Vehicle Miles of Travel (VMT). VMT is simply the product of the link volume and the link distance, summed over the desired geographic area and facility types. The observed VMT is a product of a comprehensive traffic count program. Since not every link in the network will be counted for the validation year, estimates of observed VMT must be developed.

The primary source of observed VMT is the Highway Performance Monitoring System (HPMS) data. The VMT tracking and forecasting guidance issued by the EPA requires that the HPMS be used for tracking VMT in urban areas that are in violation of the air quality standards. The HPMS estimate for VMT is calculated from samples of observed traffic counts in a region and updated regularly. It is part of the reporting requirements to the Federal Highway Administration. The FHWA publishes a report, Highway Performance Monitor System (HPMS) Field Manual that should be referred to when comparing HPMS VMT with modeled VMT.

When using the HPMS estimate of VMT, is it important to account for the basic differences in the highway system covered by HPMS and that included in the typical highway network for the travel demand model. The HPMS data includes VMT estimates for all functional classifications of roadways within the Federal Aid Urbanized Area (FAUA), including local streets. Most regional model networks do not include local streets. The lowest level of roadway in most models is the collectors. The local streets are typically represented by the centroid connectors. Recognizing this difference, the direct estimates of VMT from the model should be lower than the HPMS estimate of VMT.

In addition to the differences in the functional classification of the highway system, the different geographic areas covered by each estimate of VMT must be recognized. The HPMS is designed primarily for the area within the FHWA's designated Federal Aid Urbanized Area (FAUA). On the other hand, when the EPA designates an area as being in non-attainment, the area usually includes all counties within the nonconforming area. This non-attainment area is typically larger than the FAUA. The EPA's guidance for VMT forecasting and tracking allows for non-HPMS methods to be used in the non-attainment areas that are outside of the FAUA. Therefore, it is important to reconcile the various geographic areas of the modeled area, the HPMS area, and the non-attainment area.

While the EPA requires the HPMS method be used for tracking VMT, the network based travel demand model is the preferred method for forecasting VMT in non-attainment areas. In order to simplify the forecasting of VMT for air quality purposes, many urban areas have elected to include the entire non-attainment area in the travel demand model. This has the added advantage of not only covering the entire FAUA as required by the FHWA, but also allows for forecasting travel demand in areas that are likely to become urbanized in the future, as required by the Intermodal Surface Transportation Efficiency Act (ISTEA) of 1991.

The absolute difference is the simple difference between observed and modeled VMT. The difference is typically large for high-volume links and low for low-volume links, so the size of the numerical difference does not reliably reflect the true significance of error.

Percent difference is often preferred to absolute difference since its magnitude indicates the relative significance of error. Modeled regional VMT should generally be within five percent of observed regional VMT. This five percent difference is particularly important in light of the accepted error that EPA allows for VMT tracking using the HPMS data. The EPA has allowed margins of error in VMT estimates as high as five percent in 1994 to a new margin of three percent in 1996 and afterwards.

Table 7-3 is an example of a VMT validation summary.

Table 7-3
Example VMT Validation Summary

Facility Type VMT Error VMT Distribution
Estimated1 Observed2 Difference Percent Estimated Observed
Freeways            
Principal Arterials            
Minor Arterials            
Collectors            
Total         1 1
Notes:
1 Estimated is the VMT produced by the model
2 Observed is based on either traffic counts or the HPMS estimates of VMT

Typical distributions of VMT by facility type are presented in Table 7-4.

Table 7-4
Urban Area VMT by Facility Type

Facility Type Urban Area Population
Small
(50-200K)		 Medium
(200K-1M)		 Large
(>1M)
Freeways/Expressways 18-23% 33-38% 40%
Principal Arterials 37-43% 27-33% 27%
Minor Arterials 25-28% 18-22% 18-22%
Collectors 12-15% 8-12% 8-12%

Source: Christopher Fleet and Patrick De Corla-Souza, Increasing the Capacity of Urban Highways - The Role of Freeways, presented at the 69th Annual Meeting of the TRB, January 1990

As noted, VMT per household and VMT per person are useful measures to determine if the modelled estimates of VMT are within reasonable limits. These unit measures of VMT are also useful in determining the source of modelling error. A model that underestimates regional VMT, yet has reasonable VMT per household may have errors in the household data (underestimation of the number of households). All of these pieces of data assist the analyst in determining the cause of the modelling error and the associated adjustment or correction.

Reasonable ranges of VMT per household are 40-60 miles per day for large urban areas and 30-40 miles per day for small urban areas. The 1990 NPTS reported an average of 41.37 vehicle miles traveled per household daily. Reasonable ranges of VMT per person are 17-24 miles per day for large urban areas and 10-16 miles per day for small urban areas.

When models are originally calibrated from survey data (or transferred from other regions), the modeled regional VMT will frequently be substantially lower than the observed regional VMT. An initial response to this occurrence is often to increase trip generation rates, especially for home-based non-work and non-home-based trips, under the justification that these trips are the most commonly under-reported trips in a household travel survey. Frequently, increases in modeled trip rates of 10 to 20 percent produce modeled results that reasonably match the observed regional VMT. However, some regions have increased trip rates by as much as 60 to 70 percent.

Traffic Volumes
After validation of the VMT, the next level of validation of the highway assignment is the comparison of observed versus estimated traffic volume on the highway network. The observed count data are derived from the ongoing traffic counting and monitoring program in the urban area as described in section 2.3. This data may be developed primarily for the HPMS requirements and supplemented as required. Traffic volumes are validated at the systemwide level by comparing summations of volumes at both cordons and screenlines. While the comparison of volumes on cutlines can be used as a systemwide measure, it will be treated as a localized measure in this document.

Figure 7-2
Maximum Desirable Deviation in Total Screenline Volumes
Figure 7-2

Source: NCHRP 255 p.41 (cited in FHWA, Calibration and Adjustment of System Planning Models, Dec. 1990)

Figure 7-3
Assigned versus Observed Average Daily Traffic Volumes
Figure 7-3

Equation 7-4

The Montana Department of Transportation (MDT) suggests that an appropriate aggregate %RMSE is less than 30%. The %RMSE can be calculated for all links with counts or by facility type and area type as shown in Tables 7-5 and 7-6.

Table 7-5
Percent Root Mean Square Error Comparisons

  Reno Phoenix Concord
Facility PM ADT ADT PM AM
Freeway 18.3 18.6 25.4 NA NA
Arterial 39.2 36.8 38.5 NA NA
Collector 76.1 77.5 62.7 NA NA
Total 39.9 36.8 40.6 31.1 36.8

Table 7-6
Percent Root Mean Square Error - 24-Hour Assignment (Reno)

Facility Type Area Type
1 2 3 4 5 All
Freeway 11.649 18.092 21.891 0.000 11.271 18.334
Major Art 22.547 37.778 42.209 43.162 43.283 36.768
Collector 0.000 52.953 88.920 115.326 70.148 77.482
Minor Art 25.874 44.072 52.353 28.367 60.121 43.895
Ramp 24.237 63.524 47.574 80.649 131.009 74.846
Total 21.303 37.210 37.793 43.742 38.694 36.767

Assigned Speeds
If actual observed speed data are available, this should be summarized in highway segments consisting of a number of links and intersections so that intersection-based delay is averaged into highway travel time. Speed observations should be classified by facility type and area type to compare with modeled speeds for the same categories. Checks of highway skims include the following:

Model Parameters
Once the cordon lines and screenlines are validated and the trip distribution model is judged to be producing acceptable results, the assignment volume-delay functions can be modified systematically to produce the desired assignments. It has been the practice in some urban areas to adjust individual link attributes to get an assignment that matches the link counts. In many cases, these adjustments have produced unrealistic values of link speeds and capacities (free-flow speeds of 5mph for example) that only worked to get the desired assignment results. The adjustment of link attributes should be limited to minor systematic adjustments to speeds and capacities for groups of links that have the same facility and area type.

There are a number of parameters in highway assignment that are potential sources of error. While the actual parameters and calculation options involved depend on the modeling software and assignment methodology being used, possibilities include:

Other validation tests include:

7.2 Transit Assignment

The primary validation check of the transit assignment process is of observed versus modeled boardings. These should be checked for the region, by mode and possibly sub-mode, and by trip length. In addition, a check of observed versus modeled boardings per trip (transfer rates) is a more detailed check that tests reasonability of the number of transfers made per trip.

Model Calibration
The first step of the validation of a transit assignment occurs during the mode-choice model calibration. In the calibration step, the mode-specific constants for a region are derived so that the mode-choice model produces the appropriate share of transit trips for the region. The structure of the mode-choice model will affect the order in which the bias constants are derived. In a multinomial logit model, the bias constants for all transit modes can be derived simultaneously. If a nested logit model is employed, the bias constants for the lower levels of the nest should be derived first, then the next higher level, until the top level of the nest is reached. Several iterations of this process are normally required before an acceptable set of bias constants are derived. Note: care should be used to avoid bias constants that have an absolute value greater than 2.0 or 3.0 at the top level of the nest. If the constants are too large, the model will lose its sensitivity to level of service changes.

Validation
The amount of time and effort required to validate a transit assignment is directly correlated with the level of precision demanded. For highway planning purposes, it is generally sufficient to validate to the regional number of boardings, so that the appropriate number of person trips are removed from the highway network. For transit planning purposes, however, it may be necessary to validate to the mode, corridor, route, segment, or even station level of detail. Such precision is very difficult to attain with a fully synthetic model. (One option available when a finer level of detail is required is to utilize a pivot-point model.)

A few of the common problems that occur when validating a transit assignment are discussed in the following paragraphs, along with suggested solutions.

Number of Transfers - It is very common for a transit assignment to produce more transfers than are occurring in the actual transit systems. This problem can sometimes be solved by adjusting the transfer penalty.

However, the problem of assigning too many transfers may also result from having a shortage of walk access links to serve the transit system. The walk access links should be checked to make sure that each transit route has walk access to each TAZ within the accepted walking distance, especially when an automated access coding routine is employed. This can be difficult in CBD areas, where numerous transit routes often serve even more numerous TAZs. In order to avoid the problem of having to code too many walk access links, a CBD walk network should be employed.

Trip Length Frequency Distribution - If the average trip length for the assigned transit trips is not right, check the trip length frequency distribution for the person trip table used to create it. If the person trip distribution reflects the same pattern as the transit trip, the problem may be attributed to the trip distribution model.

Otherwise, the district-to-district transit trip summaries should be examined. The problem of an erroneous trip length frequency distribution may result from trips associated with a specific zone or district in the region. If the comparison of an observed transit trip table vs. an estimated trip table shows a large imbalance for a specific area, the route and access coding for that area should be checked first. If that network coding is reasonable and consistent with the rest of the model, you may wish to derive and apply a bias constant specific to that district.

Express or Limited Service - During the transit validation process it is often helpful to examine the relative assignments of different types of transit service. For example, it may be helpful to compare the assignments of local bus service and express bus service to determine whether or not a pattern can be found.

If the express service is being under-assigned the cause could be insufficient drive access, since express bus riders are more likely than local bus riders to drive to either a formal or an informal park-and-ride lot along the route. Alternately, the under-assignment could be due to an excess of wait time, since express bus riders who know the schedule of their service would not need to wait as long as the infrequent level of service would tend to indicate.

On the other hand, if express bus ridership is overestimated in comparison to local service, you may wish to check the transit route coding to make sure that the route is not allowed to collect passengers on the limited- or non-stop portions of the journeys.

Corridor Analysis - Most transit systems have corridors, of varying lengths, that are served by more than one transit route. These corridors have the benefit of improving the perceived, or composite, frequency of service for some of the potential transit riders in that corridor. However, with most transportation planning software, care must be taken when coding the transit lines in these corridors to ensure that the stop sequence is identical, or else a composite headway will not be calculated for that trip.

Another aspect of corridors served by multiple routes is the assignment of trips to competing transit routes. The most common practice is to have the software distribute the trips to the competing routes based on the relative frequency of service. However, this practice is only valid if certain assumptions are true: 1) the potential riders must be aware of all routes that serve their particular trip; and 2) the transit service must be spaced evenly between the competing routes. Since these assumptions are usually not true in real life, it is unlikely that the assignment of transit trips to competing routes in a corridor will be consistent with reality. Therefore, it is appropriate in the validation phase to analyze competing routes as a group, and to ignore the assignments to the individual routes.

Summary
In summary, the transit validation can include analysis of the following comparisons:

Most modeling software platforms can generate a number of reports useful in the validation process, both at the regional and local levels. Typical reports provide information relating to:

Data Sources
The primary data source for transit ridership data is from the transit operator(s) within the region. Transit ridership data that can be obtained from transit operators include:

Additional ridership data can be obtained with the use of field surveys. The most common forms of transit survey include on-board surveys, ride-check surveys, and load-check surveys. These transit surveys can be conducted separately or in concert with each other.

Ride-check surveys are conducted by placing an observer on a transit vehicle to collect on/off count data at each stop. The observer is trained to record the stop location, time, the number of passengers boarding and alighting at each stop, and the passenger load following the stop. The observer can also be trained to collect other information about the passengers, such as gender, age, or the method of fare payment. The ride-check data can be used to calculate the peak load-point along a route.

On-board transit surveys involve the use of questionnaires which ask transit riders to provide information such as the origin and destination of their trip, modes of access and egress, trip purpose, and personal information such as gender, age, income level, and automobile availability. When conducted in conjunction with a ride-check survey, information from an on-board survey can be geo-coded and expanded to build trip tables describing the zone-to-zone trips made by the riders on a specific route.

Load-check transit surveys are used to count the number of passengers boarding and alighting at a transit stop, and the number of passengers on the transit vehicles travelling through that stop. Load-check surveys are used for two main purposes, to count the transit traffic at a major terminal or transfer location, and to count the number of passengers passing through a peak load point.

7.3 Validation Targets

Although absolute criteria for assessing the validity of all model systems cannot be precisely defined, a number of target values have been developed. These commonly-used values provide excellent guidance for evaluating the relative performance of particular models.

As noted earlier, observed versus estimated volumes should be checked by facility type and geographic area. The Federal Highway Administration (FHWA) and Michigan Department of Transportation (MDOT) define targets for daily volumes by facility type as shown in Table 7-7.

Table 7-7
Percent Difference Targets for Daily Traffic Volumes by Facility Type

Facility Type FHWA Targets MDOT Targets
Freeway +/- 7% +/- 6%
Major Arterial 10% 7%
Minor Arterial 15% 10%
Collector 25% 20%
Sources: FHWA, Calibration and Adjustment of System Planning Models, 1990; Michigan Department of Transportation (MDOT), Urban Model Calibration Targets, June 10, 1993

The Contra Costa Transit Authority (CCTA) in the San Francisco Bay Area has developed the following targets for peak-hour model validation:

For the CCTA, a major arterial is defined as one that carries over 10,000 vehicles per day, a major turning movement is defined as over 1,000 vehicles per hour, and a secondary turning movement is defined as 500-1,000 vehicles per hour.

R2 and %RMSE values for VMT can be calculated for subsets of links, such as by facility type, volume range, or district.

Standards also exist for comparing observed versus modeled volumes for individual links. Table 7-8 shows percent difference targets for individual links as defined by FHWA and MDOT.

Table 7-8
Percent Difference Targets for Daily Volumes for Individual Links

Average Annual Daily Traffic Desirable Percent Deviation
MDOT FHWA
<1,000 200 60
1,000-2,500 100 47
2,500-5,000 50 36
5,000-10,000 25 29
10,000-25,000 20 25
25,000-50,000 15 22
>50,000 10 21
Source: MDOT, Urban Model Calibration Targets, June 10, 1993

The FHWA targets are displayed graphically in Figure 7-3.

Additional checks should be made of observed versus modeled VHT and observed versus average speeds by facility type, area type, and district.

Figure 7-4
Maximum Desirable Error for Link Volumes
Figure 7-4

7.4 Troubleshooting Strategies

The recommended approach to model validation discussed in this manual is to carefully check each component of the travel modeling process before the complete chain of models is applied. However, even the best structured model will contain errors and show a difference between the observed data and the model results. The assignment validation measures discussed in Section 7.1, such as screenline volumes and VMT, are typically the "bottom-line" check of how well the model performs on a systemwide basis. Section 7.3 presents typical accuracy targets for these overall measures, although many regions may have their own targets. The next step in the validation process is to evaluate the extent to which the model achieves accuracy targets, determine whether the problems are regional or local, and identify the likely causes of error.

The strategies are grouped according to the level of comparison and the likely source of error including:

These levels of comparison are described in detail below. They are listed roughly in the order in which validation should be performed, i.e. from regional to local. In many areas, transit modes account for a very small portion of regional travel and transit validation is typically a low priority. However, for areas with more significant transit facilities or where transit investments are expected in the future, the transit checks can become more important than the local highway checks.

Systemwide
Systemwide problems are identified using the aggregate highway measures such as screenline volumes and total VMT. If volumes are consistently high or low across all screenlines, then adjustments are probably needed in the following areas:

Corridor
Corridor-level problems are identified by cutline volumes or link volumes on major facilities. A comparison of capacity-restrained assignment with all-or-nothing results can reveal the difference between the desired interchanges and the modeled interchanges. Check typical paths in corridor for reasonableness. Areas to investigate include:

Local
Local highway problems are identified by looking at specific links for critical roadways. In areas with parallel facilities, traffic assignment may shift trips to the wrong facilities under congested conditions. The following should be reviewed:

Link attributes: Check any values which are specific to a particular link or class of links, such as posted speeds and capacities.

Transit
Transit validation typically focuses on the path-building characteristics and assignment of transit trips to specific routes. The total number of transit person trips should be verified first.

If regional transit trips by mode are high or low, check the following:

Socioeconomic inputs or parking costs.

Transit trips are not always assigned to the correct route, particularly if the assignment algorithm does not account for competing transit service in the same corridor. When transit trips are not being assigned to the correct routes, check the following:

Route itineraries

Future Year Application
If the step-by-step process outlined in this manual has been followed, and the validation targets have been achieved as best as possible, then the application of the model set to future year forecasts should produce reasonable results. The problem with evaluating the reasonableness of future-year forecasts is that no observed data are available for comparison. Therefore, the analyst must compare projected changes in travel demand with historical trends, forecasts for similar urban areas, and assumptions about changes in model inputs, such as socioeconomic conditions and transportation network improvements. In addition, the model may be used to evaluate transportation policy changes, such as the introduction of pricing mechanisms, which were not present in the validation year.