Nonresponse in Household Travel Surveys

2. REDUCING NONRESPONSE

2.1 CHAPTER SUMMARY

This chapter describes the potential consequences of nonresponse in household travel surveys and recommends methods for reducing the level of nonresponse. It covers three main topics:

Respondent and interviewer characteristics associated with nonresponse;
Procedures that can reduce the level of nonresponse; and
Procedures that can reduce the level of missing data in otherwise complete questionnaires.

As is customary, we distinguish between two forms of nonresponse- unit nonresponse and item nonresponse. Unit nonresponse refers to the failure to obtain questionnaires or data collection forms (such as the travel diaries used in many household travel surveys) for a member of the sample. Item nonresponse refers to the failure to obtain a specific piece of information from a responding member of the sample. Item nonresponse is often used interchangeably with the term missing data. This chapter contains recommendations for reducing the levels of both forms of nonresponse. Because most household travel surveys use some combination of telephone and mail data collection, it focuses on nonresponse issues for those modes of data collection.

MAJOR RECOMMENDATIONS

The chapter includes a large number of major recommendations based on research findings in the survey and transportation literatures, and on expert opinion in those fields. We summarize them here, grouping them by topic.

Interviewer Recruitment and Training

Identify and utilize those interviewers and field supervisors who are especially good at converting reluctant participants.
Use bilingual interviewers when the sample includes a substantial portion of non-English speakers.

Questionnaire Design

Make the flow of questions in a self-administered questionnaire or diary clear by following the natural inclination of the respondents (e. g., to read from left to right) and by using graphical devices to emphasize the intended path.

Field Work

Whenever possible, send out an advance letter to members of the sample, explaining the purpose of the study.
Carry out multiple follow-ups- at least eight calls per number in a telephone survey and at least three follow-ups in a mail survey.
Select a schedule that optimizes the timing of the contact calls given the length of the field period and the characteristics of the sample. Whenever the length of the field period is relatively short, select a schedule that attempts to produce an early contact through an intensive calling effort.
Vary the appeals in a mail survey, using a special appeal on the third follow-up attempt.
Use small prepaid monetary incentives, unless participation is especially burdensome, in which case use larger monetary incentives conditional on participation and commensurate with the level of burden.

Sampling

Allow proxy respondents if they are likely to have the information sought.
Subsample the remaining cases if the response rate is unsatisfactory and the budget is running out.

In addition, the report mentions several valuable procedures that are often considered standard survey practice. These include:

Training interviewers about the purpose and sponsorship of the study, and providing prescripted answers to common questions about the survey;
Including a brief description of the study in introduction to the questionnaire;
Translating the questionnaire and all other survey materials when a substantial portion of the sample speaks a language besides English;
Pretesting the field procedures;
Pretesting the questionnaire to improve its clarity and estimate its length; and
Placing sensitive questions at the end of the questionnaire.

Although all these procedures are effective for improving response rates, the question arises as to whether they are cost-effective. In some cases, studies have examined the impact of a particular procedure- such as the use of incentives- on the total costs of the study. The results suggest that incentives reduce costs by increasing the proportion of the sample responding to early contacts. In other cases, cost data are not available. Still, many of the procedures we recommend are quite inexpensive. An advance letter, for instance, is unlikely to add much to the total cost of data collection, but can have a substantial effect on response rates. Moreover, if the goal is to gather data for a fixed number of persons or households, it may be cheaper to follow up more extensively with fewer sample members than to start with a larger sample but accept a lower response rate. The recruitment of bilingual interviewers, the translation of questionnaires when a large proportion of the target population does not speak English, the design of user-friendly data collection forms, multiple follow-ups by mail or telephone, and the use of proxies (when they are likely to have the information being sought) are all relatively inexpensive steps that are likely to produce a reasonable payoff in terms of reduced nonresponse rates. Adopting special methods to convert hard-core nonrespondents, on the other hand, will also improve response rates, but it is likely to be costly if the number of nonrespondents in the sample is large. In such situations, a more cost-effective approach is to subsample nonrespondents and use special methods to obtain a high response rate in the subsample.

ORGANIZATION OF THE CHAPTER

The chapter begins by describing the characteristics of sample members who tend to be nonrespondents in travel and other surveys. As part of that discussion, it offers specific recommendations for improving their participation rates. The next section discusses characteristics of effective interviewers and strategies for selecting interviewers with desirable characteristics. The remainder of the chapter is devoted to a more general discussion of the various methods for reducing unit and item nonresponse.

Throughout the chapter recommended procedures for reducing nonresponse are enclosed within boxes following the text that discusses them. Whether a procedure applies to face-to-face, mail, or telephone surveys is indicated by the symbols, , , or , respectively.

2.2 RESPONDENT CHARACTERISTICS

Certain sample members are less likely than others to take part in a survey. In travel and other surveys, these individuals tend to include:

the mentally and physically handicapped,
individuals with language barriers,
individuals with limited literacy skills,
the less well educated,
the elderly, and
urban dwellers.

The reasons for the lower participation rates of some of these groups are clear. Individuals with visual difficulties, for example, are unlikely to complete a self-administered questionnaire if the print is too small for them to see. Individuals with language barriers, on the other hand, are unlikely to participate in a survey unless it is conducted in their native language. But apart from such obvious relationships, very little is known about the ways in which, and the extent to which, membership in these groups affects participation largely because direct information on nonrespondents is difficult to obtain.

Information on nonrespondents is important for three reasons. First, it can be used to assess the level of bias in the data due to the exclusion or underrepresentation of certain segments of the population. Second, it can be used to reduce nonresponse to the extent that the survey can be tailored to address the needs and concerns of the nonrespondent groups. Enlarging the print size of a self-administered questionnaire, for example, is likely to increase the participation rate of elderly individuals who often fail to complete questionnaires because they have difficulty reading small print. Third, it can be used to assess the potential payoff of tailoring the survey to specific groups. If the group represents a small portion of the population, the cost of the tailoring effort may outweigh the benefits. However, if the group includes a significant portion of the population, the benefits are likely to be substantial.

The sections below discuss what is currently known about nonrespondents and, on the basis of that information, offer specific recommendations for improving their response rates.

THE MENTALLY AND PHYSICALLY HANDICAPPED

A significant portion of the U. S. population have mental or physical handicaps that may prevent them from participating in a survey depending on what it requires of them [1]. About 7 percent of the adult population report trouble seeing standard newspaper print even with corrective lenses, a factor that is likely to affect their participation in surveys involving self-administered questionnaires. Roughly the same percent have trouble hearing what another person says in a normal conversation, and, as a result, may have difficulty participating in surveys conducted by telephone. Another 3 percent indicate they have a learning disability. As discussed below, these individuals are likely to have difficulty completing a self-administered questionnaire.

Recommendation: REDUCING NONRESPONSE DUE TO VISUAL DIFFICULTIES - Use of a font size of 12 points or more for all written respondent materials if the sample is likely to include elderly individuals or persons with visual difficulties.

INDIVIDUALS WITH LIMITED LITERACY SKILLS

According to the National Adult Literacy Survey, the English literacy skills of nearly one-fourth of the U. S. adult population are extremely limited [1]. This portion of the population is able, at best, to read and understand written information when it is brief and uncomplicated. About 19 percent of these individuals who are at the lowest level of literacy report visual difficulties that interfere with their ability to read printed material. Nearly 25 percent of individuals with poor literacy skills are immigrants who may have limited skills in English. Almost two-thirds have less than a high school education. Another 9 percent report having a learning disability. As a whole, individuals at this level of literacy are likely to have difficulty completing all but the simplest questionnaires and, in some cases, even that will be beyond their ability to read and process material written in English.

Recommendation: REDUCING NONRESPONSE DUE TO LIMITED LITERACY SKILLS IN ENGLISH- Prepare all written respondent materials at the lowest reading grade level possible. Ask all questions as simply as possible, make instructions easy to understand, and follow recommended procedures for questionnaire design (page 2-14). Have a group of experts evaluate the readability of all documents, including self-administered questionnaires, travel diaries, advance letters, correspondence, and all other materials read by respondents.

INDIVIDUALS WITH LANGUAGE BARRIERS

The ethnic and racial composition of the U. S. population has changed rather markedly in recent years [1]. The portion of the population who are of Hispanic origin, for example, grew from about 6 to 9 percent over the past decade, while the percent of individuals of Asian or Pacific Islander descent rose from about 1 to 2 percent in the same period. Recent data also indicate that nearly 10 percent of adults in the U. S. were born in other countries or in U. S. territories, and that roughly 17 percent speak a language other than English. With these changes in the composition of the population, surveys are likely to encounter even larger numbers of individuals with limited proficiencies in English who are unable to participate unless the survey is conducted in their native language.

Recommendation: REDUCING NONRESPONSE DUE TO LANGUAGE BARRIERS- Whenever the size of a non-English-speaking subgroup is expected to exceed 5 percent of the population under study and the sample is expected to include 150 or more members of the subgroup, translate the survey instruments and materials into the appropriate language. Whether the interview is conducted in person or by telephone, use bilingual interviewers who are fluent in English and in the other language.

THE LESS WELL EDUCATED

Individuals with fewer years of formal schooling tend to participate in surveys at lower rates than the rest of the population [2]. That the less well educated tend to be less proficient in English is likely to contribute to their lower participation rates.

Recommendation: IMPROVING THE RESPONSE RATE OF THE LESS WELL EDUCATED- Follow the recommended guidelines for persons with limited literacy skills in English.

THE ELDERLY

There is considerable evidence in the literature that elderly individuals participate in surveys at lower rates than individuals in other age groups [2,3]. There is some support in the literature that visual and hearing problems, which tend to be more prevalent in the elderly, account for the lower response rates of this group. The lower participation rates may also be related to years of formal education and literacy level, both of which tend to be lowest in the elderly population.

In transportation studies, the lower completion rates of the elderly are probably due, at least in part, to their lower rates of travel. Anecdotal information suggests that the elderly frequently fail to fill out or return travel diaries because they either have no trips to report or feel that the survey is not relevant to them.

Recommendation: IMPROVING THE RESPONSE RATE OF THE ELDERLY - Follow the guidelines for persons with limited literacy skills and persons with visual difficulties. In contacts with respondents, stress the importance of collecting information from everyone in the sample, including persons with no or few trips to report.

URBAN DWELLERS

Individuals living in densely populated areas are less likely to participate in surveys than individuals living in rural and other less populated areas regardless of whether the survey is conducted by mail or telephone [2]. This pattern is clearly reflected in data from the 1995 Nationwide Personal Transportation Survey shown in Table 2.1. In that survey, which was conducted by telephone, response rates in smaller, less densely populated areas- such as Utica and Elmira, New York- were about three times as high as rates in densely populated areas- such as the New York City boroughs of Queens and Manhattan. As a whole, response rates exhibited a linear relationship with population density. As density decreased, the relative proportion of sample members taking part in the survey increased.

Table 2.1 Response Rate by Population Density for Selected Areas in the States of New York, Massachusetts, and Oklahoma

DENSITY	RESPONSE RATE
20239 12594 11058 6887 2496 1733 780 629 560 332 106 103 90 87 83 57 54 46 38	14.5 23.3 25.0 21.9 24.7 27.5 30.5 30.8 33.6 32.6 44.4 35.1 50.9 42.1 59.0 38.8 43.0 51.1 44.4

*Number of persons per square kilometer

Recommendation: IMPROVING THE RESPONSE RATE OF URBAN DWELLERS - In addition to following the general guidelines for interviewer training and selection, instrument design, and field procedures: (1) delay rostering household members until the end of the screening interview (page 2-14), (2) accept proxy reports of basic background information (page 2- 31), (3) use interviewers especially good at converting the reluctant, (4) optimize timing of contact calls, and (5) allow for multiple follow-up attempts (page 2-17).

OTHER GROUPS OF NONRESPONDENTS IN TRAVEL SURVEYS

In addition to the groups above, nonrespondents in travel surveys tend to include [4]:

couples with young children, single parents, individuals who hold multiple jobs, and other busy or hard-to-reach individuals, such as 18 to 24 year-olds, and
individuals who lack the community or civic ties (e. g., renters), which often motivate participation in travel surveys.

The participation of these and the other nonrespondent groups can often be improved through use of special methods, such as increased incentive amounts and special appeals. Such methods are discussed in "Special Methods for Reducing Nonresponse," pages 32-33 of this chapter.

2.3 INTERVIEWER CHARACTERISTICS

In most surveys, some interviewers are more successful than others in securing the cooperation of sample members. The extent of these differences varies from survey to survey depending on the group of interviewers included in the effort. In one telephone survey, for example, the most successful interviewer achieved a response rate of 94 percent while the least successful interviewer achieved a rate of 58 percent. In two other telephone surveys, response rates for individual interviewers ranged from about 72 to 90 percent and from about 63 to 95 percent. While response rates vary considerably among interviewers, rates achieved by individual interviewers tend to be remarkably consistent across surveys; interviewers with certain characteristics tend to be more effective than others at recruiting sample members.

This section summarizes what is known about the relationship between interviewer characteristics and response rates. This information is important for purposes of reducing nonresponse to the extent that interviewers can be selected or trained to have characteristics associated with high response rates.

INTERVIEWER EXPERIENCE

One of the most consistent findings in the literature is that more experienced interviewers achieve higher response rates than less experienced interviewers. Since better interviewers typically stay at their jobs longer than poorer interviewers, part of this effect is due to self-selection. However, novice interviewers tend to improve over time as they gain more experience in recruiting sample members, suggesting that effective strategies for recruiting sample members can be learned [2,5].

Recommendation: INTERVIEWER EXPERIENCE - To the extent possible, use experienced interviewers with strong track records in recruiting sample members.

INTERVIEWER GENDER

A number of studies in the survey research literature report a gender difference in response rates favoring female interviewers. A variety of explanations have been offered to account for this result. The explanation receiving the most support is that males attain lower response rates because they have less interviewing experience, on average, than their female counterparts; male and female interviewers with the same level of experience tend to be equally effective in securing the cooperation of sample members [6].

INTERVIEWER EXPECTATIONS

Interviewer expectations appear to contribute to an interviewer's level of success in securing the cooperation of sample members. Interviewers who perceive the task of recruiting sample members as "moderately easy" are more successful at gaining cooperation than interviewers who believe the task is "moderately difficult " [7]. That the task is probably easier for better interviewers could, however, account for this relationship between expectations and response rates.

INTERVIEWER STYLE OF INTERACTION

Other research indicates that successful interviewers tend to be more responsive and adaptive than less successful interviewers. They are likely to address a respondent's reluctance to participate in a survey in a confident and professional manner, and to adapt their approach to the situation as unexpected issues and problems arise [8,9].

Recommendation: STYLE OF INTERACTION - Train interviewers in effective procedures for persuading reluctant participants and averting nonresponse. Stress the importance of using an adaptive approach that addresses the concerns and objections of individual sample members.

INTERVIEWER VOCAL CHARACTERISTICS

The vocal characteristics of a telephone interviewer can have an impact on response rates. Research examining this relationship reveals the following patterns:

Interviewers who speak loudly, at a fast pace, and with a standard American accent are likely to achieve higher response rates than interviewers without those vocal characteristics [10].
Interviewers who use falling intonations (that is, a deeper pitch) on important words in the introductory statement are likely to achieve higher cooperation rates than interviewers who use rising intonations (a higher pitch) [10].
Interviewers who speak loudly and at a rapid pace with a standard accent and falling intonations are perceived as being more competent and confident than interviewers with less effective vocal attributes [10].

These results are in accord with psycholinguistic research indicating that individuals who talk loudly and rapidly are thought to be more persuasive, confident, and credible than individuals who speak more slowly and softly. A standard American accent may not be preferable for every sample; if the local population has a special accent, interviewers who have the same accent may achieve the highest levels of cooperation.

Recommendation: VOCAL CHARACTERISTICS - Consider the vocal characteristics of the candidate when recruiting interviewers for a telephone survey.

OTHER INTERVIEWER CHARACTERISTICS

Personal characteristics such as motivation, tenacity, and self-confidence are also likely to contribute to an interviewer's level of success in securing sample members, but little research has been done in these areas.

2.4 METHODS FOR REDUCING UNIT NONRESPONSE

This section of the report examines different steps in the conduct of a survey that will effect the final response rate obtained. It covers:

interviewer selection and training,
the design of the survey questionnaire,
the number of attempts made to contact members of the sample,
the timing of contact attempts,
pretesting the questionnaire and field procedures,
rules for selecting the respondent,
the use of respondent incentives,
proxy respondents,
special methods for converting nonrespondents, and
subsampling of nonrespondents.

Each section discusses practical measures that can be taken at each step to reduce the level of nonresponse.

INTERVIEWER SELECTION AND TRAINING

Interviewer selection. In both telephone and face-to-face surveys, the interviewer plays a crucial role in contacting and enlisting the cooperation of sample members. As a result, most survey organizations attempt to recruit as interviewers persons who are likely to achieve high response rates. It is often thought to help, for example, if the interviewers are drawn from the same population as persons from whom they will be trying to collect information. In some surveys, the interviewers assigned to a specific member of the sample may match that member on race, sex, or other readily apparent demographic characteristics. The rationale for matching the interviewer and sample persons is not always clear, but at least in one instance matching is likely to yield a high payoff- when the sample includes a large proportion of members who are not native speakers of English. In these cases, it is often essential to recruit bilingual interviewers. Even when a translation of the questionnaire is available, a bilingual interviewer is likely to be more successful at contacting and winning the cooperation of sample members who do not speak English well than an interviewer who speaks only English.

Aside from the demographic characteristics of the interviewers, survey organizations may seek to recruit experienced interviewers or interviewers with a proven track record for achieving high response rates. In fact, some survey organizations maintain rosters of experienced interviewers who are especially successful at persuading reluctant sample members to take part in surveys. Such specialists are sometimes referred to as "converters;" they can play an important part in achieving a high overall response rate.

Recommendation: INTERVIEWER SELECTION - Identify and utilize those interviewers and field supervisors who are especially good at converting reluctant participants.

Interviewer training. Interviewers must be thoroughly conversant with the study's sponsor, aims, and purposes. They must be trained on these topics in order to respond knowledgeably and confidently to the questions and objections of reluctant respondents. For this reason, most survey organizations train both new and experienced interviewers for each survey. The survey-specific training is likely to cover several topics related to nonresponse such as the following:

the purpose and sponsorship of the study,
the provisions for maintaining the confidentiality of the data,
relevant statutes authorizing the study,
answers to commonly-raised questions about the study and commonly-cited objections to participation (see Appendix 1 for an example of such questions and answers).

Sample members are more likely to agree to be interviewed when the topic is interesting to them, the study is important, the sponsor is prestigious or is seen as a legitimate authority [10,11]. Aside from covering these basic topics, the training may explicitly describe techniques for averting nonresponse or for persuading reluctant sample members to cooperate.

Recommendation: INTERVIEWER TRAINING - Train interviewers about the purpose and sponsorship of the study. Provide them with prescripted answers to common questions about the survey and to common objections to participating in a survey.

DESIGN OF SURVEY INSTRUMENTS

The design of the questionnaires can have a large impact on the final response rates. In mail surveys, the questionnaire may be crucial in shaping sample members' impressions of the study and thus play a major role in determining whether they take part. Even in a telephone interview, where the questionnaire is not visible to the respondent, the design of the questionnaire may still affect the response rates.

Screening and recruitment scripts. In telephone screening and recruitment interviews, nonresponse often occurs within the first few seconds of the interaction. Thus, the key questionnaire design variables may involve the introduction to the questionnaire, the description of the study (including its purpose and sponsorship), and whether a translation of the questionnaire is available for sample members who have difficulty with English. Most survey organizations use brief introductions that cover such information as the name of the survey interviewer, the company for which he or she works, the name of the agency sponsoring the survey, and the study's purpose. In screening interviews (intended to identify persons eligible for more detailed data collection later on), it may be necessary to obtain a roster listing all household members and to gather information about each person. Respondents may be reluctant to give out detailed information about each family member in the first few minutes of the interview. In fact, some respondents may be reluctant even to list each person by name [13]. It may, therefore, be helpful to defer the rostering of household members to the end of the screening interview, after a modicum of trust has been established.

Recommendation: ROSTER OF HOUSEHOLD MEMBERS - Postpone the rostering of individual household members until the end of the screening interview.

Self-Administered questionnaires and diaries. Additional design features are likely to become important with self-administered questionnaires and travel diaries. Among these are the apparent length and difficulty of the questionnaires [11,14]. Aside from the actual number and content of the questions, several factors may affect the potential respondent's estimate of the level of effort needed to complete the instrument:

the use of photo reduction or other methods for reducing the apparent length of the questionnaire;
whether or not the questionnaire is attractively designed;
the flow and ease of following the instructions in the questionnaire;
the logical grouping and sequencing of questions by topic or chronology.

According to many survey experts, sample members are more likely to complete the questionnaire if the topic is interesting and if the questionnaire is short, attractive, and easy to follow [15]. The findings on the impact of the length of the questionnaire are, however, not completely consistent. According to one study, adding 20 questions to a questionnaire reduces response rates only by an average of 1 percent [11]; according to another study, questionnaires longer than four pages are likely to produce worse response rates- about 8 percent lower- than questions under four pages in length [14]. Another study comparing travel diaries of different lengths found inconclusive results regarding the length of the diary [16]. In addition, efforts to make the questionnaire appear short may create problems. Reducing the type size used in the questionnaire may make it difficult for some members of the sample to read; a substantial portion of the adult population reports difficulty in reading standard news print.

These and other results suggest that respondents are sensitive to a number of features of self-administered questionnaires, all of which contribute to their sense of the overall level of effort needed. If the questions are simple and the instructions are easy to follow, respondents may be more willing to complete a long questionnaire than a short, but difficult one [17]. Unfortunately, what design features make it easy for respondents to complete a questionnaire is not well-established. The design of survey questionnaires is still more of an art than a science. Survey designers have proposed a number of principles for the design of self-administered questionnaires [17]:

The questions should follow the natural reading order of the respondents (that is, in English, questions should flow from top to bottom and from left to right);
Questionnaires should use familiar and easily-understood graphical conventions (such as arrows indicating the next question), in which the same design element always cues the same action by the respondent;
The questionnaire should call attention to key information (via boldfacing and similar methods);
There should be a clear path for respondents to follow through the questionnaire, and the graphical features should emphasize this path.

Appendix 2 contains contrasting examples of sections from two versions of a questionnaire. One version illustrates the application of the recommended principles. The other version does not follow those principles.

Once again, if language barriers are likely to be an obstacle to participation, questionnaires in the appropriate languages should be used.

Recommendation: DESIGN OF SELF -ADMINISTERED QUESTIONNAIRES - Ask the questions as simply as possible and make the instructions easy to follow. Use boldfacing and similar methods to highlight key information. Use easily-understood graphic conventions to direct respondents, adopting a reading path that follows established customs for the language in which the questionnaire is written. In English, for example, make questions flow from top to bottom.

NUMBER AND TYPE OF CONTACTS

One of the clearest findings in the literature on nonresponse to surveys is the importance of repeated contacts with sample members. In general, fewer than half of all sample members return a questionnaire after the first mailing in a mail survey. Each additional follow-up contact is likely to yield additional returns [18]. Similarly, in the best telephone surveys, interviewers may attempt 8 to 10 callbacks before giving up on a sample telephone number. One study estimated that more than 10 percent of the telephone sample was reached only after the eighth call [19]. We recommend allowing for at least eight calls in planning a telephone survey.

Recommendation: CALLBACK LIMIT - Allow for at least eight (8) callbacks per sample telephone number when conducting a telephone survey.

Advance letters. Many face-to-face and telephone surveys begin by sending members of the sample prenotification, or advance, letters. Such letters are typically signed by a non-controversial government official or some other prestigious source; they explain the purpose of the study, describe the confidentiality measures to be employed, and alert the sample member that he or she will be contacted shortly by an interviewer. Advance letters accomplish several useful purposes. They help establish the legitimacy of the survey, inform potential respondents about the study's purposes, allay concerns about confidentiality, and serve as an introduction to the interviewer. Advance letters have also proved valuable in both mail and telephone studies. With mail surveys, studies suggest that an advance letter can be quite useful in obtaining cooperation, increasing response rates by an average of about 17 percent [14]. Even with a telephone sample, it is possible to use advance letters to boost response rates. If the sample is selected from residential lists of numbers, address information usually accompanies the sample telephone numbers. If the telephone numbers are randomly generated, the sample of numbers can be matched to data bases with address information and advance letters can be sent to numbers for which addresses are found. 1

Because of their proven success, advance letters have become common practice in household travel surveys in recent years.

Recommendation: ADVANCE LETTERS - Send out advance letters to sample members explaining the sponsorship, purpose, and relevance of the study. Stress the importance of participation and the confidentiality of the results.

Follow-up contacts. For mail surveys, some experts have recommended specific regimes of follow-up contact efforts. Dillman's Total Design Method, for example, calls for an initial mailing and three follow-ups spread over a seven-week period [20]:

a reminder postcard after one week has elapsed;
a replacement questionnaire after four weeks; and
a second replacement questionnaire sent by certified mail.

Although it is not clear that this exact schedule must be followed, the literature does indicate that multiple follow-ups can have a dramatic impact on the final response rates obtained. One study concluded that each contact in a mail survey (including an advance letter) increased response rates by an average of about 11 percent [11]. In line with Dillman's prescriptions, the literature also suggests that a special third follow-up contact- such as a mailing by certified mail or a telephone contact- does boost the final response rates, with an estimated effect of about 7 percent [11]. In addition, first class mail and prepaid return postage may increase response rates relative to bulk mail.

Recommendation: FOLLOW-UP CONTACTS IN MAIL SURVEYS - Allow for at least three follow-up contact attempts in a mail survey. Change the appeal on each contract. Use a special procedure, such as a certified mailing, on the last follow-up attempt.

Length of the field period. One implication of the findings on the number of callbacks is that a high response rate will take time to achieve. Multiple callbacks are only possible (and will only be productive) if the data collection period is long enough. For example, the Total Design Method requires a field period of at least eight or nine weeks. Telephone surveys with eight or more callbacks require a field period of similar length. Surveys that must be completed too quickly to allow sufficient follow-up work are likely to have low response rates.

Recommendation: LENGTH OF THE FIELD PERIOD - Whenever possible, make the field period long enough to permit multiple contact attempts for each member in the sample.

Respondent-initiated contacts. During the course of any survey, some respondents inevitably have questions about the data collection instruments or other aspects of the survey. It is therefore good survey practice to provide a telephone hot line, toll free if possible, that respondents can call at most anytime to obtain timely answers to their questions. The line should be answered by persons who are fully conversant with the purpose and design of the survey, and especially knowledgeable about the content of the questionnaires. A call-in service of this type helps improve response rates and data quality by eliminating misunderstandings, resolving ambiguities, and allaying respondents' concerns. It also helps establish the legitimacy of the survey. A website or e-mail address can serve a similar function for respondents with on-line capabilities.

Recommendation: Recommendation RESOURCES FOR RESPONDENTS - Establish a telephone hot line that respondents can call almost anytime to obtain timely answers to their questions about the survey and its instruments.

TIMING OF THE CONTACT CALLS

Rationale. In telephone screening efforts, attempts to contact and determine the eligibility of sample members usually reach a point of diminishing returns after which additional callbacks no longer produce significant gains in the number of sample members screened. The previous section offers some guidelines for determining the optimal number of callbacks for contacting sample members in travel surveys conducted by telephone. More often than not, other considerations, such as the length of the field period and the availability of resources, determine the callback limit adopted by a survey.

Whenever resources are limited, the survey's level of success in screening the entire sample will largely depend on the timing of the contact calls. The most effective timing for a particular survey will depend on such factors as the sampling frame, the population under study, the length of the field period, and whether the call is the first or a subsequent attempt to make contact with the sample member [21,22]. In most situations, and especially when resources are very limited, the strategy that will lead to the highest response rate for the screening phase is one in which the timing of the contact calls is designed to minimize the overall level of effort required to successfully screen sample members [23,24,25]. In calling protocols of this type, the times are selected to 2 maximize the chances of a successful outcome such that the overall number of calls required to complete the screening effort is kept at a minimum. Examples of such protocols are described below.

In situations where the length of the field period is especially short and there is not enough time to contact sample members at optimal times, the calling protocols must rely on other methods to maximize the response rate for the screening phase. The most effective protocols for situations of this type attempt to produce an early contact through intensive calling efforts [26]. Usually the initial contact call is scheduled for the first available time slot and followed up as quickly as possible with as many callbacks as time and resources allow. The most efficient protocols of this type attempt to optimize the timing of the first and subsequent calls to the extent possible within the time constraints of the survey. At a minimum, they vary the times at which callbacks are attempted for a given case. An example of such a protocol is presented below.

Recommendation: TIMING OF THE CONTACT CALLS - Optimize the timing of the contact calls to the extent possible, given the length of the field period and the characteristics of the sample.

Optimizing the Timing of Calls. Protocols that optimize the timing of the contact calls vary in their complexity. Some are based on mathematical models or complex rules that require computer algorithms for their use. Others are designed for manual tracking and scheduling of calls. All are based on rules intended to increase the efficiency of the screening phase by minimizing the number of calls required to successfully screen sample members.

Table 2.2 summarizes some of the most effective rules for optimizing the timing of contact calls in household telephone surveys. The extent to which these rules can be applied in practice will depend on the resources and staffing capabilities of the survey organization, and on the amount of time available for the screening phase of the survey.

The first entry in the table refers to situations where the sample is drawn from a frame that consists mostly or entirely of household telephone numbers. Since all or most of the units in the sample will be residential units, the optimal times to initiate contact in samples of this type are days and hours when adult household members are likely to be at home. Research findings confirm the commonly held belief that the likelihood of reaching an adult member at home is highest on weekday evenings and weekends, and lowest on weekdays during daytime hours. Overall, weekday evenings tend to be more productive than weekends. The relative productivity of the individual weekday evenings tends to vary from study to study as does the relative productivity of weekend mornings, afternoons, and evenings.

Table 2.2 General Rules for Optimizing the Timing of Contact Calls

SAMPLE FRAME	MODE OF DATA COLLECTION	TIMING OF INITIAL CONTACT	TIMING OF CALLBACKS
Listed residential telephone numbers	Telephone	First available weekday evening or weekend time slot (defined as a weekend morning, afternoon, or evening)	First callback on a weekday evening or during a weekend time slot, preferably on a different evening or during a different weekend time slot than the initial call. Other callbacks may occur at any time.
Randomly generated telephone numbers	Telephone	First available weekday evening or weekend time slot	First callback on a weekday evening or during a weekend time slot, preferably on a different evening or during a different weekend time slot than the initial call. Other callbacks may occur at any time. At least one callback is scheduled during weekday business hours to screen out business numbers remaining in the sample
Randomly generated telephone numbers	Telephone Two-week field	First available weekday business hour time slot (defined as a morning or afternoon)	First callback during evening hours on the day following the initial call. period Second callback during business hours on the day following the first contact. Subsequent callbacks rotate between evening and daytime shifts. Each unresolved case is called daily until the field period is almost over. Field period ends in an intensified effort to screen all remaining noncontacts.

Recommendation
TIMING OF THE CONTACT CALLS FOR RESIDENTIAL TELEPHONE NUMBERS - When the sample consists largely of residential telephone numbers, schedule the contact calls for evenings and weekends.

The best time to schedule the second contact attempt in samples of this type is also during evening and weekend hours, preferably in a different time slot (a different weekday evening or weekend morning, afternoon, or evening) than the first attempt. Subsequent callbacks can be scheduled for any available time slot since the timing of calls appears to have little effect on the chances of a successful contact at this point in the screening process. When differences are detected they favor evenings and weekends as the most productive calling times, and methods that vary the timing of the calls as the most productive protocols.

The productivity of these protocols, and those discussed below, can be improved by using information collected during previous calls to optimize the timing of subsequent calls. In telephone surveys, for example, the person who answers the phone may be able to provide information about when adult members of the household are likely to be at home. To maximize the chances of a successful outcome, the next call should be scheduled accordingly.

Recommendation
USING INFORMATION COLLECTED DURING PREVIOUS CALLS - Use information collected during previous calls to optimize the timing of subsequent calls.

Determining the optimal times for initial calls and callbacks for samples drawn from frames that include a relatively large number of nonresidential and other out-of-scope units is more complicated since the optimal times for screening the various types of units in the sample will usually differ, and the effectiveness of any given strategy will depend in part on the relative proportion of residential and out-of-scope units in the sample. A variety of protocols have been developed to handle samples of this type. Some protocols are clearly more effective than others, but research in this area is rather limited and the most effective methods for such samples have yet to be identified.

To illustrate the type of rules associated with the more effective methods in the literature, Table 2.2 shows two protocols developed for screening samples of randomly generated telephone numbers. Both protocols were developed for the same type of sample, but the rules they apply to control the timing of the calls are quite different. The first protocol gives priority to optimizing the times for household units in the sample. It assumes that the field period is of sufficient length to successfully apply this procedure. The second protocol gives priority to identifying out-of-scope numbers. It attempts to optimize the timing of all calls to the extent possible in an intensive effort to screen all cases within a two-week time frame. The protocol assumes that all telephone numbers identified as nonresidential numbers will be replaced with other randomly selected numbers, which must then be screened before the field period ends.

As mentioned above, the first protocol gives priority to screening the households in the sample. For the most part it follows the rules for screening samples of household units. It schedules the first and second calls for times when household members are likely to be at home. At least one of the remaining callbacks is scheduled during business hours to screen out any business numbers remaining in the sample.

The rationale behind this approach is threefold. First, priority should be given to contacting households in the sample since they are the units of primary interest. Second, most business and nonresidential numbers can be screened out by matching sample numbers against published business directories or through recorded messages or operator intercepts when calls are placed during nonbusiness hours. Third, screening out nonresidential and nonworking numbers during weekend and evening hours reduces the cost of the survey since telephone tolls tend to be lowest during those hours.

The second protocol, on the other hand, schedules initial calls during business hours to expedite the identification of out-of-scope units so that interviewers will have as much time as possible to screen replacement numbers. It schedules the first callback for the evening shift of the day following the initial contact to increase the chances of finding household members at home during the second call. Subsequent callbacks rotate between daytime and evening shifts. Each case is called once daily until the end of the field period approaches. At that time the field effort is intensified. Noncontacts are called more than once daily, sometimes within the same shift.

Recommendation
TIMING OF CONTACT CALLS FOR RANDOMLY GENERATED TELEPHONE NUMBERS - Select a calling protocol that is likely to lead to the highest screener response rate given the characteristics of the sample and the length of the field period.

If resources are available, response rates obtained with other protocols for randomly generated numbers can be improved by ending the field period with an intensive calling effort of this type. Some methods used for this purpose include: 1) rotating the case through all available shifts (weekday and weekend, daytime and evening shifts) until a successful contact is made or the callback limit is reached, and 2) rotating the case through various time slots (morning, afternoon, and evening hours) within a particular shift, and continuing the procedure in other shifts until the callback limit is reached or the eligibility of the case is resolved [22]. In some protocols, the case may be called as often as every hour.

Recommendation
ENDING THE FIELD PERIOD - When resources are available, end the field period with an intensive calling effort.

Special rules for telephone samples. To increase the efficiency and productivity of the calling effort, protocols for telephone samples often adopt special rules for certain types of numbers answered by machine or voice mail, and numbers resulting in a busy signal.

Since the majority of numbers answered by machine or voice-mail eventually result in a contact after repeated call attempts(*), most protocols treat such calls 3 as "no answers" and call the numbers again until the callback limit is reached. The exception to this rule is when the recorded message indicates that they are businesses, other non-residences, or non-working telephone numbers, in which case they are classified as such and no further attempts are made [27,28].

* Even though 20 to 30 percent of households within the United States have answering machines or 3 voice mail attached to their telephone numbers, only a small percent of these households consistently use the devices to screen unwanted calls.

The optimal number of callbacks for numbers answered by recorded message tends to be higher than that for other numbers and, as a result, some calling protocols adopt a higher callback limit for these numbers. The optimal limit is about ten calls, after which the chances of reaching a person are very small. Some protocols also adopt a special callback schedule that eliminates Saturday hours from the calling routine since Saturdays are particularly poor days to reach households with answering machines [27,28].

Recommendation
CALLBACK LIMIT FOR NUMBERS ANSWERED BY MACHINE OR VOICE MAIL - Extend the callback limit to 10 when a number is consistently answered by machine or voice mail. Schedule callbacks to those numbers for days other than Saturdays.

Some protocols incorporate special rules for handling calls resulting in busy signals. The rules differ depending on whether a fast or regular busy signal is encountered. Callbacks for regular busy signals- tones that are interrupted at a rate of 60 times per minute and indicate that the line is in use- are usually scheduled for 15 to 60 minutes after the signal is encountered since the chances of reaching someone at home are typically highest during that period [29]. Callbacks for numbers resulting in a fast busy signal- tones that are interrupted at a rate of 120 times per minute and indicate difficulties due to storms or other problems, busy circuits in the calling area, or nonworking telephone numbers- are usually scheduled at varying times during business and evening hours over a period of days or weeks to determine the working or nonworking status of the number [22,29].

Recommendation
CALLBACK PROCEDURES FOR BUSY NUMBERS - Whenever possible, call back numbers with regular busy signals within 15 to 60 minutes after the signal is encountered.

PRETESTING FIELD PROCEDURES AND INSTRUMENTS

Whether the survey is to be done over the telephone or by mail, pretesting the questionnaire itself and the field procedures to be used in the main data collection will almost always improve the response rates ultimately obtained. These pretests can take several forms. Many survey firms carry out a small number of interviews (10 to 20) using the proposed questionnaire and attempt to probe for specific problems in the questions, such as comprehension difficulties. In our discussion of techniques for reducing item nonresponse, we describe a variation on this procedure called the cognitive interview. Apart from leading to improvement in the questionnaire, a small-scale pretest can serve a second function, that of providing a realistic estimate of the time needed to complete the questionnaire.

In addition to small-scale pretests of the questionnaire, other types of pretests are useful as well. It is often worthwhile to conduct a dress rehearsal for the main study. Such dress rehearsals are useful for detecting problems in the planned procedures, for estimating the likely response rate in the main data collection, and for experimenting with procedural variations (such as the format of the questionnaire or the amount of incentive to offer). The size of the samples for such field tests often vary widely, depending on such factors as whether multiple procedures are being compared. A review of 55 recent transportation surveys indicated that more than 90 percent used pretest samples of 100 or fewer cases [30].

Recommendation
PRETESTING FIELD PROCEDURES AND INSTRUMENTS- Conduct a small-scale pretest to test the survey instruments and the field procedures.

INCENTIVES

In many surveys, members of a survey sample are offered some form of incentive in an effort to gain their cooperation. In some surveys, the incentive accompanies the request for cooperation and consists of a small nonmonetary gift, such as a pen or a poster, or a small monetary gift of a dollar or two. In others, the incentive is substantially larger, conditional on participation, and offered as compensation for the respondent's time and trouble. The use of incentives is a widespread practice within survey research, especially for mail surveys (where incentives of a dollar or two are commonly employed) and in face-to-face surveys that make particularly onerous demands on the respondents, such as providing a biological specimen (where much larger incentives are the rule). This section provides guidelines for selecting effective incentives for travel surveys based on research in the survey and transportation literatures.

Offering incentives to sample members is one of the most effective procedures for reducing nonresponse in surveys. The effectiveness of a particular offering will depend on such factors as: 1) the monetary value of the incentive, 2) whether the incentive is paid in advance or upon completion of the survey, 3) whether the incentive is monetary or nonmonetary, 4) the mode of data collection- mail, telephone, or personal interview, and 5) the burden the survey places on the respondent [31, 33]. In situations where sample members are recruited by mail or telephone and asked to complete relatively brief self-administered questionnaires- as is the case in many travel surveys, the most effective incentives are usually:

small in value,
paid in advance, and
monetary.

The sections below discuss these research findings in greater detail.

Recommendation
INCENTIVES- Provide sample members with small prepaid monetary incentives of $2 or less, unless participation is especially burdensome. In that case, offer larger monetary incentives of $10 or more conditional on participation and commensurate with the level of burden placed on the respondent.

Effect of incentives on participation rates. Most research on incentives has focused on evaluating the effectiveness of small monetary offerings, typically $2 or less, or small gifts, such as a pen or a lottery voucher, in surveys where sample members are recruited by mail and asked to complete a self-administered questionnaire, or recruited by telephone and asked to complete a self-administered questionnaire at some later point in time. Surveys of this type generally place a low to moderate level of burden on the respondents; the amount of time respondents are asked to commit to the survey is relatively small, and their participation does not involve threatening or demanding tasks.

The effects of offering small incentives in such surveys have been evaluated under a variety of conditions, including situations where the incentive is paid in advance and sent to the respondent along with the survey materials, or conditional on participation and sent to the respondent after completion of the survey. In cases where respondents are recruited by mail, advance incentives are typically enclosed in the initial packet along with the request for cooperation and the survey questionnaire. In cases where respondents are recruited by telephone and asked to complete a self-administered survey as part of the study, advance incentives are usually sent along with the survey packet after respondents have agreed to participate, typically without knowledge of the incentive. Under these conditions:

incentives paid in advance are generally more effective in gaining the cooperation of sample members than conditional incentives paid after completion of a survey, even when the promised amount is larger than the prepaid amount.
incentives paid in advance typically increase response rates about 15 to 20 percentage points, even when the value of the incentive is 50 cents or less.
monetary incentives are usually more effective than small gifts in gaining the cooperation of sample members.
incentives conditional on participation often have a negligible or negative effect on response rates.

In situations where the survey places a higher burden on the respondent, larger incentives are often effective, if not required, for gaining the cooperation of certain sample members. Surveys belonging in this category typically place unusual or time-consuming demands on the respondent. They may, for instance, require the respondent to travel to a specific location, to submit to a physical examination, or to complete a battery of tests. In most surveys of this type, the incentives are offered as compensation in return for the respondent's time and effort. In the National Assessment of Educational Progress household survey, for example, youth were asked to complete a battery of four tests and offered $0, $10, $15, or $20 for finishing one, two, three, or four tests. The incentives increased response rates from about 71 percent (no incentive) to 83 percent [33]. Other surveys placing similar demands on respondents show comparable gains in response rates associated with incentives conditional on survey participation. In the 1971 National Health and Nutrition Examination Survey, where respondents were asked to submit to a physical examination, a conditional incentive of $10 increased the response rate from 70 to 82 percent.

Effect of incentives on data quality. Most studies examining the impact of incentives on data quality report a positive effect. Respondents who are offered incentives tend to provide more complete data than other respondents; item nonresponse is usually lower and answers to open-ended questions more complete. Moreover, respondents receiving a prepaid incentive are also more likely to provide complete data than respondents receiving a conditional incentive [32].

Effect of incentives on survey costs. The few studies in the literature evaluating the impact of incentives on survey costs report reductions in the average cost per case when incentives are used [32]. In the field test for the National Adult Literacy Survey, for example, the average cost per case for sample members offered conditional incentives was less than that for members in the no incentive treatment condition (see Table 2.3). Participants who were offered incentives also required fewer interviewer contacts on average than participants in the no incentive condition [34]. Other studies report similar reductions in the average number of contacts due to incentives.

Table 2.3 Incentive Amount, Number of Contacts, and Cost Per Case in the Field Test for the National Adult Literacy Survey

INCENTIVE AMOUNT	AVERAGE NUMBER OF CONTACTS PER CASE	AVERAGE COST PER CASE (INCLUDING INCENTIVE)
$ 0	6.7	$130
$20	5.3	$119
$35	5.0	$129

Incentives in transportation studies. Table 2.3, adopted in part from a report by Melissa Tooley [35], summarizes the results of some recent studies designed to evaluate the impact of incentives on response rates in household travel surveys. Overall, the results of this work support the conclusions drawn above, which were based on findings in the survey literature as a whole.

In accord with the survey literature, Table 2.3 shows that promised incentives of larger value tend to yield mixed results in travel surveys. In the San Francisco Study, for example, a conditional incentive of $10 reduced the response rate by about 6 percentage points. In the Houston-Galveston Study, on the other hand, a conditional incentive of $10 increased response rates, but only by 6 percentage points.

Travel studies also support the general finding that small prepaid incentives are usually more effective than promised incentives. In the Puget Sound Transportation Panel, for example, a prepaid incentive of $1 yielded a higher response than a promised incentive of greater value ($ 10). The prepaid incentive increased response rates by about 15 percentage points, while the conditional incentive of $10 increased participation by 11 percentage points.

The results from the Pretest of the Dallas-Fort Worth Household Travel Survey, also shown in Table 2.4, support the general finding that monetary incentives are usually more effective than nonmonetary gifts in improving response rates. In that study, a prepaid incentive of $2 led to higher response rates than a gift of a pen in three out of four cases. The differences in participation favoring the $2 incentive ranged from about 8 to 18 percentage points.

Table 2.4 Response Rates by Incentive Condition Comparisons in Five Transportation Studies

STUDY	YEAR	ADVANCE LETTER	FOLLOW-UP	RETRIEVAL	INCENTIVES	RESPONSE RATES
Puget Sound Transportation Panel	1989	Y	-	Mailback	None $1 (pre) $10 (post)	49% 64% 60%
San Francisco Study	1990	Y	-	Telephone	None $10 (post)	47% 41%
San Antonio Bexar County, Amarillo, and Brownsville	1990 1990 1991	Y	Y	Mixed	None $5 (post)	28-39% 40%
Houston/ Galveston	1984	Y	Y (Multiple)	Mailback	None $10 (post)	32% 38%
Dallas-Fort Worth Pretest	1995	N	Y	Telephone	Pen (pre) $2 (pre)	35%-48% 45%-60%

Note: "Pre" incentives are those included with the initial survey packet; "post" incentives are those offered upon completion of the questionnaires. Response rates may not conform to the recommended procedure for reporting response rates and may vary across the studies. However, comparisons among incentive conditions within a study are valid indicators of the potential effect of incentives on response rates.

PROXY RESPONDENTS

Different surveys impose different rules regarding whom is allowed to supply survey data for a sample member. In some surveys, any adult member of the household is permitted to provide information about other individuals in the household. In other surveys, the acceptability of data from proxy respondents depends on the information sought. In any case, the rules adopted by a particular survey will have an impact on the response rate obtained.

Proxy reporting of basic background information. When the survey requires only basic demographic information about each member of a household, it is common practice to gather this information from any adult member in the household. Proxy reporting of this type is frequently used in travel and other surveys to collect demographic and other background data on household members during the screening interview. This information is usually used to determine the eligibility of household members and to roster individuals within the household. Obtaining screener information in this way usually increases the efficiency and completeness of the screening effort without significantly reducing the accuracy of the information obtained. It enables interviewers to determine the eligibility of household members who are not home at the time of the contact and to collect information on members who are unable to respond for themselves because of mental or physical handicaps, or language barriers. In most cases, the gains in efficiency and response rates outweigh the small losses in accuracy associated with collecting screener information by proxy.

Recommendation
PROXY REPORTING OF BASIC BACKGROUND INFORMATION- During the screening interview, allow for proxy reporting of basic demographic and background information. Whenever possible, verify the accuracy of the information during the main interview. Resolve any discrepancies that occur.

Proxy reporting of travel behavior and attitudes. The practice of accepting proxy reports of attitudes and behaviors, on the other hand, is less common largely because the gains in response rates are frequently offset by losses in reporting accuracy. In most situations, attitudinal and behavioral information provided by proxy respondents tends to be less accurate than information supplied by the actual respondents. In the 1990 Nationwide Personal Transportation Survey (NPTS), for example, proxy respondents reported, on average, about 25 percent fewer trips per travel day, and about 20 percent fewer miles traveled and driven than persons responding for themselves, suggesting that proxy reports underestimate actual travel behavior [36]. Data from the 1995 NPTS Pretest exhibited a similar pattern; proxy respondents reported fewer trips and fewer miles traveled than self respondents. They were also more likely to respond with "I don't knows."

PROXY REPORTING OF TRAVEL BEHAVIOR AND ATTITUDES - Avoid using proxy reports, unless the accuracy of the information can be assured in one of two ways: 1) the proxy accompanied the actual respondent on the trip or in the activity, or 2) the respondent recorded the information in a travel diary or similar instrument. Always attempt to collect information from the actual respondent. When these attempts fail and it is more than three days since the travel behavior occurred, accept proxy datum when it satisfies at least one of the conditions above.

SPECIAL METHODS FOR REDUCING NONRESPONSE

In any survey, no matter how well it has been conducted, there is usually a group of hard-core nonrespondents who are very difficult to contact, very reluctant to participate, or simply unable to take part in the survey. In travel surveys this group of sample members tends to include [36]:

the elderly,
the less well-educated,
urban dwellers,
individuals with physical limitations or language barriers,
couples with young children, single parents, individuals who hold multiple jobs, and other busy or hard-to-reach individuals, such as 18 to 24 year-olds, and
individuals who lack community or civic ties that often motivate participation in travel surveys.

The number of nonrespondents remaining in these groups at the end of the survey can often be reduced by tailoring the survey materials and methods to overcome the obstacles standing in the way of their participation. In Section 2.2, we discussed some general methods for improving the response rates of the elderly, the less-well educated, individuals with poor literacy skills, and individuals with physical limitations or language barriers. The response rates of these and other nonrespondent groups can often be further improved through use of special methods that make participation particularly attractive (or more difficult to avoid), address specific concerns or objections, or call attention to the survey. Some examples of special methods of this type are:

large incentives conditional on participation,
persuasive calls from the most successful interviewers or field supervisors,
special appeals and mailings,
artfully constructed conversion letters, and
in-home visits, if the addresses of the sample members are known.

The method selected for a particular group should be tailored to that group. Most 18 to 24 year-olds, for example, are unlikely to be moved by appeals to civic duty, but will often take the time to complete a survey if they are offered a large incentive. Single parent families and families with young children, on the other hand, are often more willing to take part in a study if the sponsor offers to cover babysitter costs while they complete the survey. While some of these methods for improving response rates can be relatively inexpensive- special mailings and artfully constructed conversion letters, for example, others- such as in-home visits and large incentives, may be too expensive to apply on a frequent basis.

Recommendation
TAILORING THE SURVEY - To the extent possible, tailor the survey to address the special needs, concerns, and objections of subgroups of individuals unwilling to take part.

SUBSAMPLING NONRESPONDENTS

When it is not possible to adopt special methods for all reluctant sample members, one alternative to simply writing off the final group of nonrespondents is to select a subsample and use special methods to achieve a high response rate among the subsample.

Although formulas have been derived for determining the best point at which to begin subsampling the remaining nonrespondents, subsampling is typically undertaken only when all sample members have been contacted repeatedly, the response rate remains unsatisfactory, and the data collection budget is nearing exhaustion. To illustrate the use of subsampling, consider a hypothetical survey which has achieved only a 50 percent response rate after spending 75 percent of the data collection budget. If a subsample of one in five of the remaining nonrespondents is chosen, the budget level of effort per case can be increased substantially, making it possible to achieve a relatively high response rate among the subsample. For example, if even 40 percent of the subsample provide the required data, the weighted response rate will be 70 percent- that is, within the first half of the sample, 100 percent of the cases are respondents and, within the subsample representing the second half of the cases, 40 percent are respondents. The important point is that the sample respondents include some representatives of the hard-to-complete group.

Recommendation
SUBSAMPLING OF NONRESPONDENTS - When the data collection budget is almost exhausted, response rates are unacceptably low, and all cases in the sample have been fielded, subsample nonrespondents to boost response rates and to improve the representativeness of the sample.

2.5 ITEM NONRESPONSE

Even if the overall response rate is high, many respondents may be missing data for specific items. The statistical consequences of unit and item nonresponse are identical. Fortunately, the level of item nonresponse tends to much lower than the level of unit nonresponse. In this section, we describe some methods for lowering the rate of item nonresponse.

QUESTIONNAIRE DESIGN

The same questionnaire design features that help reduce unit nonresponse tend to reduce item nonresponse as well. Short questionnaires with clear instructions are easier to complete and more likely to be completed correctly than long questionnaires with difficult to follow instructions. The clarity and appearance of the questionnaire is likely to be particularly important with self-administered questions. Respondents are about twice as likely to incorrectly skip items when they complete the questionnaire themselves than when the questionnaire is administered by an interviewer [37]. This differential can be reduced if the instructions for filling out the questionnaire are clear and simple and if graphics are used to indicate the flow of questions [16].

With a self-administered questionnaire sent to the respondent by mail, the respondent must be able to figure out how to complete the questions without the help of an interviewer. It is, therefore, especially important that the respondents understand the questions and any accompanying instructions. The questions and the instructions must be clearly stated and easily understood.

Techniques for developing questionnaires. There are several commonly-used techniques for developing questionnaires, including focus groups, expert reviews, cognitive interviews, and small pilot studies (typically with fewer than 100 respondents). We especially recommend expert reviews and cognitive interviews as inexpensive but effective means for improving draft questionnaires.

In an expert review, three or four specialists in questionnaire design review the draft, point out problems with it, and suggest improvements. The questionnaire and background information about the study (describing the study's purposes and the analyses in which the data are likely to be used) are typically sent out to the experts prior to a face-to-face or telephone review session. The review session may last 2 to 8 hours depending on the length of the questionnaire.

It is also important to try out the questions with persons drawn from the same population as the sample for the main study. Cognitive interviews may be a particularly valuable tool for detecting respondent problems in understanding the questions or in performing the other cognitive operations (such as retrieval and judgment) that are required to answer accurately. In a cognitive interview, respondents are encouraged to think aloud as they attempt to answer the questions [38]. For example, they may be instructed to:

Please tell me what you are thinking about as you answer the next few questions. This will help us understand whether the questions we are asking are being understood as intended and whether they have any other problems. Remember to think out loud as you answer. Let's try an example before we turn to the main questions. Tell me what you are thinking as you answer the question How many windows are there in your home?

In addition, the respondents in a cognitive interview may be asked to:

point out unfamiliar terms (Are you familiar with [the term]?);

paraphrase a question or set of instructions (Can you say in your own words what that last question was asking?);

describe their thought processes after having answered a question (Please tell me how you arrived at that last answer); and

rate their confidence in their answers (Are you very sure, somewhat sure, or not too sure about that last answer?).

All these techniques are intended to reveal the cognitive processes through which respondents arrive at their answers and to identify any problems with the questions. Typically, interviews are completed with 10 to 15 respondents and a short report is produced that highlights any problems detected with specific items and suggests modifications.

Recommendation
DEVELOPING QUESTIONNAIRES - Whenever possible, use expert review and cognitive interviews to improve the quality of items in the questionnaire.

Sensitive questions. Aside from comprehension errors or other cognitive problems, other factors may contribute to high levels of missing data. For example, respondents may refuse to answer questions they regard as too personal. Many surveys ask for the annual income of the respondent or the respondent's family, and often this item has higher levels of missing data than any question in the survey. The best that can be done is to ask such sensitive questions toward the end of the questionnaire after the respondent has completed the bulk of the questions. Training interviewers to explain why the question is needed and how the information will be used may also help to reduce nonresponse to sensitive questions.

Recommendation
SENSITIVE QUESTIONS - Place sensitive questions towards the end of the questionnaire. Train interviewers to explain the importance of the question and how the information will be used.

RETRIEVAL OF MISSING ITEMS

The most direct method for reducing the number of items that are missing is to recontact the respondent and obtain any information omitted from the questionnaire. This procedure is called "retrieval" by many survey researchers. Generally, retrieval is reserved for a small number of especially important items, which are predesignated beforehand. Often the "retrievable" items are the same ones used to define whether a questionnaire is regarded as complete. (See the discussion of "critical items" in Chapter 1.)

The recontact is usually made by telephone (if a telephone number is available). The retrieval interviewer explains that an important piece of information has been omitted from the questionnaire and asks the respondent to provide it. If the initial interview was conducted by telephone and the respondent refused to provide the information in the first place, the retrieval interviewer explains the importance of the missing data and points out that the rest of the information that the respondent did provide cannot be used without the missing items.

Retrieval is generally a cost-effective method for salvaging cases that would otherwise be difficult or impossible to include in the analysis.

Recommendation
RETRIEVAL OF MISSING ITEMS - Recontact respondents who failed to provide a few pieces of key data. This recontact can be made through a less expensive method of contact, such as telephone.

2.6 CURRENT PRACTICE FOR REDUCING NONRESPONSE IN TRAVEL SURVEYS

Current practice for reducing nonresponse in household transportation surveys differs widely across survey firms and particular studies. Some studies incorporate many of the recommended procedures for reducing nonresponse. Others rely on just one or two procedures or employ less than optimal methods.

A recent review of household travel surveys by Peter Stopher and Helen Metcalf [29] reveals the extent to which current practice varies among travel surveys. Their analysis, based on a review of 55 household travel surveys conducted from 1988 to 1994, shows how recent surveys made use of three methods for reducing nonresponse: pretests, incentives, and follow-up reminders. The results of the review are described below.

PRETESTS

About 74 percent of the surveys included in the analysis carried out a pretest prior to the conduct of the main survey. The samples in the studies ranged in size from fewer than 25 to over 500 respondents; the most common size was between 26 and 50 respondents. The objectives of the pretests varied from study to study. All were designed to test the survey instruments. Some were full dress rehearsals for the main survey. Others evaluated particular aspects of the survey's design or its procedures. Table 2.5 shows the percent of pretests evaluating various aspects of the survey.

Table 2.5 A Summary of Pretest Objectives

ASPECT	PERCENT OF PRETESTS EVALUATING EACH ASPECT
Instrument	100
Management	58
Training of Survey Personnel	48
Sampling Procedures	40
Data Keying Procedures	28
Geocoding Procedures	25
Analysis	23
Effectiveness of Incentives	10

INCENTIVES

Only about twenty percent of the surveys included in the review used some form of incentive to encourage participation. Of the studies employing incentives, nearly half relied on a cash incentive. Of the surveys using cash or lotteries, about half distributed the incentive in advance, while the other half used conditional incentives.

FOLLOW-UP REMINDERS

According to the review, the use of follow-up reminders, that is, contacts to remind sample members to complete their questionnaires after they had received them in the mail, has become common practice in household travel surveys. Nearly 80 percent of the surveys employed this method for reducing nonresponse. Nonetheless, the studies differed in the type, and especially in the number, of contacts used for this purpose (see Table 2.6). About 93 percent of the studies contacted the respondents by telephone to remind them to complete the questionnaires or travel diaries. More than half relied on a single reminder, while a good portion, 20 percent, used as many as four or more follow-up reminders.

Table 2.6 Type and Number of Follow-up Reminders

FOLLOW-UP REMINDERS		PERCENT
TYPE	Telephone call	93
TYPE	Other	7
NUMBER	One	60
	Two	17
	Three	3
	Four or more	20

References

1. Kirsh, I. S., Jungeblut, A., Jenkins, L., & Kolstad, A. (1993). Adult Literacy in America: A First Look at the Results of the National Adult Literacy Survey. Washington, D. C.: National Center for Education Statistics.

2. Groves, R. M. & Lyberg, L. E. (1988). An overview of nonresponse issues in telephone surveys. In R. M. Groves, P. P. Biemer, L. E. Lyberg, J. T. Massey, W. L. Nicholls II, and J. Waksberg (Eds.) , Telephone Survey Methodology. New York: John Wiley.

3. Herzog, A. R. & Kulka, R. A. Telephone and mail surveys with older populations: a methodological overview.

4. Couper, M. P. & Groves, R. M. (1992). The role of the interviewer in survey participation. Survey Methodology, 18, 263-277.

5. Groves, R. M. & Fultz, N. H. (1985). Gender effects among telephone interviewers in a survey of economic attitudes. Sociological Methods and Research, 14, 31-52.

6. Singer, E., Frankel, M. R., & Glassman, M. B. (1983). The effect of interviewer characteristics and expectation on response. Public Opinion Quarterly, 47, 68-83.

7. Morton-Williams, J. & Young, P. (1986). Obtaining the survey interview: an analysis of tape-recorded doorstep introductions. Journal of the Market Research Society, 27, 35-53.

8. Collins, M., Sykes, W., Wilson, P., & Blackshaw, N. (1988). Nonresponse: The UK Experience. In R. M. Groves, P. P. Biemer, L. E. Lyberg, J. T. Massey, W. L. Nicholls II, and J. Waksberg (Eds.) , Telephone Survey Methodology. New York: John Wiley.

9. Oksenberg, L. & Cannell, C. (1988). Effects of interviewer vocal characteristics on nonresponse. In R. M. Groves, P. P. Biemer, L. E. Lyberg, J. T. Massey, W. L. Nicholls II, and J. Waksberg (Eds.) , Telephone Survey Methodology. New York: John Wiley.

10. Heberlein, T. A., & Baumgartner, R. (1978). Factors affecting response rates to mailed questionnaires: A quantitative analysis of the published literature. American Sociological Review, 43, 447-462.

11. Groves, R., Cialdini, R., & Couper, M. (1992). Understanding the decision to participate in a survey. Public Opinion Quarterly, 56, 475-495.

12. Tourangeau, R., Shapiro, G., Kearney, A., & Ernst, L. (In press). "Who lives here? Survey undercoverage and household roster questions." Journal of Official Statistics.

13. Yammarino, F. J., Skinner, S. J., & Childers, T. L. (1991). Understanding mail survey response behavior. Public Opinion Quarterly, 55, 613-639.

14. Dillman, D. M. (1991). The design and administration of mail surveys. Annual Review of Sociology, 17, 225-249.

15. Applied Management & Planning Group. (1995). Dallas-Ft. Worth household travel survey: Pretest report. Los Angeles, CA: Applied Management & Planning Group.

16. Jenkins, C., & Dillman, D. (1997). Towards a theory of self-administered questionnaire design. In L. Lyberg, P. Biemer, M. Collins, E. DeLeeuw, C. Dippo, N. Schwarz, & D. Trewin (Eds.), Survey measurement and process quality. New York: Wiley.

17. Richardson, A. J., Ampt, E. S., & Meyburg, A. H. (1995). Survey Methods for Transport Planning. Parkville, Vic: Eucalyptus Press.

18. Groves, R., & Kahn, R. (1979). Surveys by telephone: A national comparison with personal interviews. New York: Academic Press.

19. Dillman, D. M. (1978). Mail and telephone surveys: The Total Design Method. New York: JohnWiley.

20. Sebold, J. (1988). Survey period length, unanswered numbers, and nonresponse in telephone surveys. In R. M. Groves, P. P. Biemer, L. E. Lyberg, J. T. Massey, W. L. Nicholls II, and J. Waksberg (Eds.) , Telephone Survey Methodology. New York: John Wiley.

21. Weeks, M. F. (1988). Call scheduling with CATI: Current capabilities and methods. In R. M. Groves, P. P. Biemer, L. E. Lyberg, J. T. Massey, W. L. Nicholls II, and J. Waksberg (Eds.), Telephone Survey Methodology. New York: John Wiley.

22. Weeks, M. F., Jones, B. L., Folsom, R. E., & Benrud, C. H. (1980). Optimal times to contact sample households. Public Opinion Quarterly, 44, 100-114.

23. Weeks, M. F., Kulka, R. A., & Pierson, S. A. (1987). Optimal call scheduling for a telephone survey. Public Opinion Quarterly, 51,540-549.

24. Kulka, R. A. & Weeks, M. F. (1988). Toward the development of optimal calling protocols for telephone surveys: a conditional probabilities approach. Journal of Official Statistics, 4, 319-332.

25. Greenberg, B. S. & Stokes, S. L. (1990). Developing an optimal call scheduling strategy for a telephone survey. Journal of Official Statistics, 6, 421-435.

26. Tuckel, P. S. & Feinberg, B. M. (1990). The answering machine poses many questions for telephone survey researchers. Public Opinion Quarterly, 55, 200-217.

27. Oldendick, R. W. (1993). The effect of answering machines on the representativeness of samples in telephone surveys. Journal of Official Statistics, 9, 663-672.

28. Hoagland, R. J., Warde, W. D., & Payton, M. E. (1988). Investigation of the optimum time to conduct telephone surveys. Proceedings of the Survey Research Methods Section, American Statistical Association, 755-760.

29. Stopher, P. R. & Metcalf, H (1996). Methods for Household Travel Surveys: A Synthesis of Highway Practice. Washington, DC: National Academy Press.

30. Kulka, R. A. (1994). The use of incentives to survey "hard-to-reach" respondents: a brief review of empirical research and current practice. Paper prepared for the Seminar on New Directions in Statistical Methodology, sponsored by the Council of Professional Associations on Federal Statistics (COPAFS), Bethesda, Maryland, May 25-26.

31. Kulka, R. A. (1992). A brief review of the use of monetary incentives in federal statistical surveys. Paper prepared for the Symposium on Providing Incentives to Respondents, convened by the Council of Professional Associations on Federal Statistics (COPAFS) for the Office of Management and Budget, John F. Kennedy School of Government, Harvard University, Cambridge, MA., October 1-3.

32. Chromy, J. R. & Horvitz, D. G. (1978). The use of monetary incentives in national assessment household surveys. Journal of the American Statistical Association, 73, 473-478.

33. Berlin, M., Mohadjer, L., Waksberg, J., Kolstad, A., Kirsch, I., Rock, D., & Yamamoto, K. (1992). An experiment in monetary incentives. Proceedings of the Survey Research Methods Section, American Statitical Association, 393-398.

34. Tooley, M. (1996). Incentives and rate of return for travel surveys. Paper presented at the Conference of the Transportation Research Board.

35. Moore, P. A. (1995). Using proxy interviews in a nationwide travel survey. Paper presented at the Annual Conference of the American Association of Public Opinion Research, Fort Lauderdale, Florida, May 18-21.

36. Johanna Zmud (1997). Personal communication.

37. Tourangeau, R., Rasinski, K., Jobe, J., Smith, T., & Pratt, W. (In press). Sources of error in a survey of sexual behavior. Journal of Official Statistics.

38. Willis, G., Royston, P. & Bercini, D. (1991). The use of verbal report methods in the development and testing of survey questionnaires. Applied Cognitive Psychology, 5, 251-267.