Sociology Panel Studies
by
Heather Laurie
  • LAST MODIFIED: 26 August 2013
  • DOI: 10.1093/obo/9780199756384-0108

Introduction

Panel studies are a particular design of longitudinal study in which the unit of analysis is followed at specified intervals over a long period, often many years. The key feature of panel studies is that they collect repeated measures from the same sample at different points in time. Most panel studies are designed for quantitative analysis and use structured survey data. Panel studies can also use qualitative methods for the data collection and analysis. They may also be constructed from register data, an approach that is common in some countries. This entry concentrates on household panels collected by surveys. Cross-sectional surveys are based on a sample of the population of interest drawn at one time point. In contrast, panel surveys follow the population of interest over an extended time period and are concerned with measuring change over time for the units of analysis within the population. The unit of analysis is typically an individual, but it could also be a firm or a dwelling or any other unit of analysis required by the research design. Panel surveys typically collect data at relatively frequent intervals depending on the design requirements of a given study. Some run over many years and others are short term, such as short panels conducted around elections. Panel surveys are distinct from cohort studies, which often sample an age cohort born in a particular month and year and follow that cohort at infrequent intervals, often with a focus on early childhood development. While the difference between cohort and panel designs can be overstated, panel studies typically sample from the entire age range and collect repeated measures across the age range and throughout the life course. Panel studies have been used extensively to monitor the dynamics of poverty, movements into and out of the labor market, and the process of demographic change. Longitudinal data generated from panel studies can be analyzed to understand the short-term dynamics of change, including movements into and out of employment or transitions into and out of poverty. Panel studies can also be used to examine long-term effects such as children’s education and labor market outcomes in the context of their family background, or later life health outcomes in the context of earlier health behaviors. Panel studies are therefore suited to the analysis of the life course and understanding the interrelationships between life events, behaviors, preferences, and later outcomes that affect people’s life chances and well-being and provide data which enhances our ability to make causal inferences through controlling unobserved heterogeneity.

General Overviews

The works in this section provide a combination of overviews to survey methods and longitudinal studies in general, as well as panel studies in particular. They provide an introduction to the range of issues to be considered when designing, implementing, and analyzing longitudinal data sets, which tend to be more complex than surveys done in a cross-sectional context. Groves, et al. 2009 and de Vaus 2002 are essential survey methodology texts for those unfamiliar with survey methodology. The principles of high-quality data collection that apply to cross-sectional surveys also apply to longitudinal surveys, but there are additional issues to consider due to the longitudinal design. The concept of “total survey error,” comprising error from sampling and nonsampling sources, has become central to assessing data quality within survey methodology, and it has particular implications for longitudinal studies (Groves 2005). Rose 2000 and Ruspini 2002 introduce the principles that apply to high-quality data collection for panel studies and are accessible introductions for those unfamiliar with how panel data can be used in analysis. They provide helpful examples of research using panel data to illustrate how these data can be exploited in analysis. The first major volume to synthesize the complexities involved in designing and managing a panel study was Kasprzyk, et al. 1989, covering particular aspects in the collection of the data, the implications for data quality of the longitudinal design, and analysis techniques for panel data. Menard 2008 and Lynn 2009 are up-to-date edited volumes, with contributions by experts in the field, on the methodology of longitudinal surveys, including new developments in the field. They highlight the advantages and disadvantages of panel studies for analysis, statistical adjustments such as weighting and imputation, and panel data analysis techniques.

  • de Vaus, D. A. 2002. Surveys in social research. 5th ed. London: Routledge.

    Save Citation »Export Citation »E-mail Citation »

    A textbook covering all aspects of the survey design, data collection, and analysis phases applicable to cross-sectional and longitudinal surveys. Emphasizes the importance of identifying clear research questions and operationalizing key concepts within a questionnaire to produce reliable and valid measures for analysis.

    Find this resource:

    • Groves, R. M. 2005. Survey errors and survey costs. 2d ed. Hoboken, NJ: Wiley.

      Save Citation »Export Citation »E-mail Citation »

      An essential text covering the concept of total survey error and its components and the costs and errors arising from sampling and nonsampling error.

      Find this resource:

      • Groves, R. M., F. J. Fowler, M. P. Couper, J. M. Lepkowski, E. Singer, and R. Tourangeau. 2009. Survey methodology. 2d ed. Hoboken, NJ: Wiley.

        Save Citation »Export Citation »E-mail Citation »

        This volume is an up-to-date textbook covering the basic key principles of survey design and implementation at all stages of the data collection process. The authors are leaders in the field of survey methodology and this is an ideal volume for those less experienced in survey methods.

        Find this resource:

        • Kasprzyk, D., G. Duncan, G. Kalton, and M. P. Singh, eds. 1989. Panel surveys. New York: Wiley.

          Save Citation »Export Citation »E-mail Citation »

          This edited volume is the first authoritative text dedicated to providing a comprehensive and systematic review on the design and analysis of panel surveys. While this volume was published at the end of the 1980s, it remains one of the best volumes on the subject.

          Find this resource:

          • Lynn, P., ed. 2009. Methodology of longitudinal surveys. Wiley Series in Survey Methodology. Chichester, UK: Wiley.

            DOI: 10.1002/9780470743874Save Citation »Export Citation »E-mail Citation »

            Written by a team of international experts, this volume covers all the main stages in the design, implementation, and analysis of longitudinal surveys and includes recent methodological developments in the field. These include the use of dependent interviewing and mixed-mode data collection.

            Find this resource:

            • Menard, S., ed. 2008. Handbook of longitudinal research: Design, measurement, and analysis. Amsterdam: Elsevier.

              Save Citation »Export Citation »E-mail Citation »

              Written by leaders in the field and designed to introduce readers to the topic. Chapters describe the design, collection, and analysis stages of longitudinal research, including panel surveys. Describes the range of analysis techniques available, including descriptive and causal analysis, event history analysis, structural equation models and multilevel models, and time-series analysis.

              Find this resource:

              • Rose, D., ed. 2000. Researching social and economic change: The uses of household panel studies. London: Routledge.

                Save Citation »Export Citation »E-mail Citation »

                An accessible introduction to panel studies, highlighting issues of data quality to be aware of when analyzing panel data. Includes chapters giving examples of how panel data have been used in substantive analysis of poverty transitions, low-income dynamics, household and family dynamics, and migration and residential mobility.

                Find this resource:

                • Ruspini, E. 2002. Introduction to longitudinal research. London: Routledge.

                  Save Citation »Export Citation »E-mail Citation »

                  Aimed at graduate students and those new to the area of longitudinal research, this volume provides a concise yet comprehensive introduction to the issues involved, including defining the concept of longitudinal research, sources of longitudinal data in Europe and the United States, and the advantages and disadvantages of certain types of research data and of different types of analysis.

                  Find this resource:

                  Software and Statistical Resources

                  There are a number of statistical software packages commonly used for panel analysis. These include well-known packages such as SPSS and Stata as well as more specialist packages such as Mplus, MLwiN, and REALCOM for multilevel modeling. Stata is well suited to panel data analysis, and there are many online resources to help users with understanding longitudinal data and analysis. One of the difficulties when specifying a longitudinal analysis is knowing what data are available, and how to access them, along with basic information on the size, duration, and content of the study. National data archives are good sources of information on longitudinal studies. Multiple Imputation is a statistical method for estimating values that may be missing in the data, and it has become increasingly popular as programs have become available to allow these types of estimation. ICE is one program developed within Stata for multiple imputation. The citations included here are just a small selection of the software and other resources that are available.

                  Training Resources

                  There are a number of sources of training at institutions around the world, as well as online training resources of various kinds. Training varies from introductory to advanced levels and will provide general training as well as specific training in particular analysis techniques. For someone who has not conducted longitudinal analysis previously, it is a good idea to become familiar with the range of analysis techniques, and taking a short course is often a good way of doing so. Some courses are listed below, mainly UK based, but this is not an exhaustive list, and analysts should find the course that is appropriate to their needs. These are often provided by longitudinal studies directly. The ICPSR program at the University of Michigan is extremely well regarded, taught by experts, and includes courses at different levels. In the United Kingdom, the Institute for Social & Economic Research at the University of Essex runs short courses designed to introduce the use of longitudinal data. Some prior knowledge of analysis methods is assumed. Essex University also run the annual Essex Summer School in Social Science Analysis, with a wide range of two-week courses dealing with specific analysis methods in detail. Many international tutors can be combined into an MA from a number of modules taken over a few years, if required. The National Centre for Research Methods is a central hub for training events conducted throughout the United Kingdom. See the website for the calendar of courses. The Centre for Multilevel Modeling at the University of Bristol, UK, provides training courses and online training materials. The Cathie Marsh Centre for Census and Survey Research is a major training center within the United Kingdom, providing analysis and statistical training, including for longitudinal data. Survival Analysis with Stata provides online materials with worked examples of survival analysis, and Panel Data Methods for Sociologists provides online materials designed for those who may be less familiar with quantitative analysis and are coming to panel analysis for the first time.

                  Journals

                  There are some specialist journals for longitudinal methods and analysis, but on the whole one will find longitudinal methodology papers in general methodology journals and longitudinal analysis papers in any of the major national and international journals. Longitudinal analysis is not necessarily tied to one disciplinary perspective, so relevant articles may be found in a range of discipline-specific journals. Some journals that focus either on survey methodology or longitudinal analysis are listed below, but this is by no means an exhaustive list. International Statistical Review is a highly ranked and well-cited journal suitable for articles with a statistical or methodological focus. The Journal of the Royal Statistical Society (Series A) is a major journal within the United Kingdom that regularly carries methodological papers, including for longitudinal analysis, and the Journal of the American Statistical Association is a similar highly ranked journal for the United States. The Journal of Official Statistics is a very good source of articles on survey methodology in general and longitudinal methods in particular. Longitudinal and Lifecourse Studies is an online journal focusing exclusively on articles using longitudinal data sources, including cohort and panel surveys. Public Opinion Quarterly is one of the major highly ranked journals for survey methodology and analysis methods, including longitudinal methods. Sociological Methods & Research and Survey Methodology are both well respected and include articles on longitudinal methods.

                  Panel Study Design

                  There are many variations in the design of panel surveys, and the design chosen will depend on the research questions for the study. Bailar 1989 and Buck, et al. 1996 outline the process of choosing the most appropriate design, depending on a number of factors, as well as setting out how the design will affect the type of analysis that can be carried out using the data. Rose 2000 focuses on panel studies in particular and is a good introductory text for those unfamiliar with panel studies. Binder 1998 draws out the essential differences between cross-sectional and longitudinal data, while Stafford 2010 discusses the design features in the context of a particular large-scale study. The first decision is the unit of analysis required to answer the research questions. Possible units of analysis for a panel study include the individual, household, dwelling (address), or establishment (e.g., a firm or an institution, but it could be any other relevant unit of analysis). The unit of analysis used for many panel studies is the individual, as the aim is to study change at the level of the individual. These are usually known as “household” panel studies even though the focus is on individuals. This is because it is not possible to define a “longitudinal household” in any consistent or rigorous way (Duncan and Hill 1985). The composition of households changes over time, but individuals remain a unit that can be followed over time even when moving between households. The other major consideration is the sampling design. The main distinction is between single indefinite life panel studies (i.e., a sample is drawn at one time point and followed thereafter) and rotating panel studies (i.e., the sample is periodically rotated out of the study and new sample rotated into the study on a predetermined cycle). The choice of sampling design will be affected by the frequency of interviewing required. Higher-frequency interviews, such as quarterly interviews, often use a rotating sample design, while indefinite life designs typically have lower-frequency interviews at intervals of one or two years. Lynn 2009 provides an overview of the key design features commonly used in longitudinal studies, including panel studies, while Menard 2002 discusses the analysis potential and advantages and disadvantages of differing types of designs. The design of the longitudinal study will affect the quality of the collected data due to sampling error and nonsampling error (see also Causes and Consequences of Unit Nonresponse and Attrition and Weighting and Imputation).

                  • Bailar, B. A. 1989. Information needs, surveys, and measurement error. In Panel surveys. Edited by D. Kasprzyk, G. Duncan, G. Kalton, and M. P. Singh, 1–24. New York: Wiley.

                    Save Citation »Export Citation »E-mail Citation »

                    This chapter summarizes some of the key trade-offs between design alternatives and the potential effects on measurement error.

                    Find this resource:

                    • Binder, D. A. 1998. Longitudinal surveys: Why are these surveys different from all other surveys? Survey Methodology 24.2: 101–108.

                      Save Citation »Export Citation »E-mail Citation »

                      Introduces the topic by reviewing the current status and challenges for longitudinal studies as compared to cross-sectional studies. Provides a useful discussion of the special issues and challenges encountered in the design, implementation, evaluation, and analysis of longitudinal surveys.

                      Find this resource:

                      • Buck, N., J. Ermisch, and S. P. Jenkins. 1996. Choosing a longitudinal survey design: The issues. Occasional Paper 96-1. Colchester, UK: ESRC Research Centre on Micro-social Change.

                        Save Citation »Export Citation »E-mail Citation »

                        This paper is written by key figures responsible for the British Household Panel Study (BHPS) and gives an accessible summary of the main considerations in terms of the advantages and disadvantages when choosing a longitudinal study design.

                        Find this resource:

                        • Duncan, G. J., and M. S. Hill. 1985. Conceptions of longitudinal households: Fertile or futile? Journal of Economic and Social Measurement 13.3–4: 361–375.

                          Save Citation »Export Citation »E-mail Citation »

                          Written by two founding members of the team responsible for the Panel Study of Income Dynamics at the University of Michigan (see also Panel Data Sets for Secondary Analysis), this article clearly articulates the problematic conceptual issue of “longitudinal households” and the rationale for using the individual as the unit of analysis.

                          Find this resource:

                          • Lynn, P. 2009. Methods for longitudinal surveys. In Methodology of longitudinal surveys. Edited by P. Lynn, 1–20. Wiley Series in Survey Methodology. Chichester, UK: Wiley.

                            DOI: 10.1002/9780470743874.ch1Save Citation »Export Citation »E-mail Citation »

                            Provides a good overview of the key design considerations and the potential effects on data quality.

                            Find this resource:

                            • Menard, S. 2002. Longitudinal research. 2d ed. Thousand Oaks, CA: SAGE.

                              Save Citation »Export Citation »E-mail Citation »

                              Describes four basic designs for longitudinal research, including panel designs, and discusses the analysis potential for each design type.

                              Find this resource:

                              • Rose, D. 2000. Household panel studies: An overview. In Researching social and economic change: The uses of household panel studies. Edited by D. Rose, 3–35. London: Routledge.

                                Save Citation »Export Citation »E-mail Citation »

                                An introductory overview to panel survey design and the analytic advantages of panel data.

                                Find this resource:

                                • Stafford, F. P. 2010. Panel surveys: Conducting surveys over time. In Handbook of survey research. 2d ed. Edited by P. V. Marsden and James D. Wright, 765–794. Bingley, UK: Emerald.

                                  Save Citation »Export Citation »E-mail Citation »

                                  Written by a US leader in the field of longitudinal studies, this chapter provides a summary of the key features, advantages, and issues for panel surveys in the context of the US Panel Study of Income Dynamics.

                                  Find this resource:

                                  Sample Design and Following Rules

                                  There are a range of sample design options available for panel studies (see also Panel Study Design). Kalton and Citro 2000 and Smith, et al. 2009 review the key issues to be considered with the sample design. As Duncan and Kalton 1987 set out, the sample design will aim to provide a representative sample of the population at the point of selection, and to reproduce the population of interest over the period of the study. The sampling strategy chosen will depend on the information needs of the study, the nature of the population being surveyed, and the available budget. A key feature of any panel study is that it follows the same cases over time. The sample design for a panel study therefore must define what are described as “following rules” to determine which cases should be followed, and according to what criteria and rules. Hill 1992, Kalton and Lepkowski 1985, and Lynn 2006 give examples from three major panel studies that show how the following rules vary according to the survey design and what is appropriate given the data needs of the study. As a secondary data user, it is important to understand these rules, as they will have a direct effect on the sample design as it evolves over a period of years and on the types of analysis that can be carried out. The following rules form part of the overall sample design, as the aim is to maintain a representative sample of the longitudinal population of interest over the life of the study. For a household panel study where the individual is the unit of analysis and all eligible household members are interviewed, the following rules will typically state that all original sample members, such as those resident in the wave 1 household (including children <16), will be followed when they move or change their address. In addition, there will be rules for incorporating new household members who are born into the sample or become co-resident with an original sample member. Lynn 2011 sets out some of the challenges in maintaining cross-sectional representativeness in a longitudinal study. For example, it is not always clear if sample members have died, and with an indefinite life panel with no sample refreshment, immigrants into the population after wave 1 will not be included in the sample. For that reason, some studies incorporate boost samples of new immigrants at some stage during the life of the study.

                                  • Duncan, G. J., and G. Kalton. 1987. Issues of design and analysis of surveys across time. International Statistical Review 55.1: 97–117.

                                    DOI: 10.2307/1403273Save Citation »Export Citation »E-mail Citation »

                                    An early article by two leaders in the field that clearly sets out some of the main considerations when designing or analyzing longitudinal data, including sampling issues, data collection, and data quality.

                                    Find this resource:

                                    • Hill, M. S. 1992. The panel study of income dynamics: A user’s guide. Newbury Park, CA: SAGE.

                                      Save Citation »Export Citation »E-mail Citation »

                                      A comprehensive description of the major design parameters for PSID, even though the design has extended subsequently. A useful guide for users coming to the data for the first time. Chapter 2 describes the sample design and following rules.

                                      Find this resource:

                                      • Kalton, G., and C. F. Citro. 2000. Panel surveys: Adding the fourth dimension. In Researching social and economic change: The uses of household panel studies. Edited by D. Rose, 36–53. London: Routledge.

                                        Save Citation »Export Citation »E-mail Citation »

                                        Provides a general review of designs for surveys across time and the types of analysis they will support, as well as discussing the issues involved in the design and conduct of a longitudinal study.

                                        Find this resource:

                                        • Kalton, G., and J. Lepkowski. 1985. Following rules in SIPP. Journal of Economic and Social Measurement 13.3–4: 319–329.

                                          Save Citation »Export Citation »E-mail Citation »

                                          Describes the following rules employed by the rotating panel design of the US Survey of Income and Program Participation (SIPP), the rationale for the design, and the implications for analysis.

                                          Find this resource:

                                          • Lynn, P. 2006. Quality profile: British Household Panel Survey. Colchester, UK: Univ. of Essex.

                                            Save Citation »Export Citation »E-mail Citation »

                                            Describes the sample design and following rules for the British Household Panel Survey (BHPS) and provides a helpful and informative assessment of the quality of the sample and data more generally. Read in conjunction with the online documentation this is a useful source of information for users new to the data set. Version 2.0: Waves 1–13: 1991–2003.

                                            Find this resource:

                                            • Lynn, P. 2011. Maintaining cross-sectional representativeness in a longitudinal general population survey. Understanding Society Working Paper Series 2001-04. Colchester, UK: Univ. of Essex.

                                              Save Citation »Export Citation »E-mail Citation »

                                              A helpful paper for those trying to understand the challenges and impact of correctly identifying members of the initial sample who leave the population through death or emigration, and to periodically add appropriate samples of people who join the population through birth or immigration.

                                              Find this resource:

                                              • Smith, P., P. Lynn, and D. Elliot. 2009. Sample design for longitudinal surveys. In Methodology of longitudinal surveys. Edited by P. Lynn, 21–34. Wiley Series in Survey Methodology. Chichester, UK: Wiley.

                                                DOI: 10.1002/9780470743874Save Citation »Export Citation »E-mail Citation »

                                                A good review of the key issues to be considered in sample design for longitudinal surveys, including defining the longitudinal population, sample size, clustering, stratification, following rules, and dealing with births and deaths.

                                                Find this resource:

                                                Questionnaire Design and Data Quality

                                                Panel surveys, due the nature of repeated collection of measures from the same population at different time points, raise specific issues for questionnaire design. These need careful consideration to produce high-quality data. Schwarz 1997 reviews the elements involved in the question/answer response process and the implications for measurement. The questionnaire design for a panel study needs to be appropriate for the collection of longitudinal data to allow the duration of spells to be estimated accurately, to collect the antecedents and outcomes of key events and situate those events accurately across time, to allow the processes of household formation and dissolution to be tracked as they occur, and to provide longitudinally comparable measures across time. Formal methods for evaluating questionnaires and questions are discussed in Presser, et al. 2004, while Dykema, et al. 1997 describes the use of interaction coding as a method for assessing data quality. Participant recall is often not reliable when collecting retrospective data, but as Dex 1995 points out, it can also affect data collected at shorter intervals on a panel survey. The misplacing of events in time can lead to “seam effects,” which are a particular feature of panel studies (Moore, et al. 2009). As Jäckle 2009 describes, the introduction of computer-assisted interviewing has enabled data to be preloaded from previous interviews to reduce inconsistencies across survey waves and reduce seam effects (see also Mode of Data Collection). Levels of item nonresponse, especially for critical measures such as income, and measurement error due to other potentially biasing response behaviors, such as satisficing or social desirability effects, all contribute to data quality. Questionnaire length, how burdensome the survey is for participants, and the perceived saliency of the survey content to participants all feed into determining response rates, and higher response rates are generally assumed to produce better-quality data. However, there is some debate about whether higher response rates necessarily improve data quality (Kaminska, et al. 2010). Cantor 2008 reviews what is known about the phenomenon of panel conditioning (i.e., where participants may become aware of the questions they are asked each year and change their response behavior as a result) and finds limited evidence of panel conditioning or the effects on data quality.

                                                • Cantor, D. 2008. A review and summary of studies on panel conditioning. In Handbook of longitudinal research, design, measurement and analysis. Edited by S. Menard, 123–138. Amsterdam: Elsevier.

                                                  Save Citation »Export Citation »E-mail Citation »

                                                  Panel conditioning is where participants, by virtue of taking part in a longitudinal study, may have their response behavior to the questions asked “conditioned” over time. This chapter provides a useful review of what is known about panel conditioning, even though evidence is scarce.

                                                  Find this resource:

                                                  • Dex, S. 1995. The reliability of recall data: A literature review. Bulletin de Methodologie Sociologique 49.1: 58–89.

                                                    DOI: 10.1177/075910639504900105Save Citation »Export Citation »E-mail Citation »

                                                    One of the advantages of panel data is minimizing recall error, as information on events is collected close to the time at which they occur. Many panels also include retrospective data on labor market, marriage, and fertility histories. This article provides a comprehensive review of what is known about recall error.

                                                    Find this resource:

                                                    • Dykema, J., J. J. Lepkowski, and S. Blixt. 1997. The effect of interviewer and participant behaviour on data quality: Analysis of interaction coding in a validation study. In Survey measurement and process quality. Edited by L. Lyberg, Paul Biemer, Martin Collins, et al., 287–310. New York: Wiley.

                                                      DOI: 10.1002/9781118490013Save Citation »Export Citation »E-mail Citation »

                                                      Standardized interviewing, where interviewers read the exact question wording consistently, has been seen as the key to providing reliable responses to survey questions. This study is interesting because it finds no consistent relationship between the exact reading of question wording and the accuracy of responses.

                                                      Find this resource:

                                                      • Jäckle, A. 2009. Dependent interviewing: A framework and application to current research. In Methodology of longitudinal surveys. Edited by P. Lynn, 93–110. Wiley Series in Survey Methodology. Chichester, UK: Wiley.

                                                        DOI: 10.1002/9780470743874Save Citation »Export Citation »E-mail Citation »

                                                        A helpful chapter that sets out why and how dependent interviewing is used in longitudinal surveys, as well as describing some of the advantages and potential disadvantages for data quality.

                                                        Find this resource:

                                                        • Kaminska, O., A. L. McCutcheon, and J. Billiet. 2010. Satisficing among reluctant participants in a cross-national context. In Special issue: Total survey error. Edited by P. Biemer and L. Lyberg. Public Opinion Quarterly 74.5: 956–984.

                                                          DOI: 10.1093/poq/nfq062Save Citation »Export Citation »E-mail Citation »

                                                          Higher response rates are commonly assumed to produce better quality data, and this article examines the relationship between reluctance and response quality. It explores the relationship between latent variables of reluctance and satisficing, before and after controlling for cognitive ability, and finds that response quality is explained by cognitive ability.

                                                          Find this resource:

                                                          • Moore, J., N. Bates, J. Pascale, and A. Okon. 2009. Tackling seam bias through questionnaire design. In Methodology of longitudinal surveys. Edited by P. Lynn, 73–90. Wiley Series in Survey Methodology. Chichester, UK: Wiley.

                                                            DOI: 10.1002/9780470743874Save Citation »Export Citation »E-mail Citation »

                                                            “Seam bias” is an effect of recall error in longitudinal surveys where events are misplaced in time around the point between interview periods. This chapter compares the seam bias on the Survey of Income and Program Participation prior to and after introducing dependent interviewing. The seam bias is reduced but does not completely disappear.

                                                            Find this resource:

                                                            • Presser, S., M. P. Couper, J. T. Lessler, et al. 2004. Methods for testing and evaluating survey questions. Public Opinion Quarterly 68.1: 109–130.

                                                              DOI: 10.1093/poq/nfh008Save Citation »Export Citation »E-mail Citation »

                                                              Although this article discusses questionnaire design and testing methods in a cross-sectional context, the issues apply also to panel and other longitudinal studies. Pretesting of questionnaires is common practice, but formal methods for evaluating the results are less common.

                                                              Find this resource:

                                                              • Schwarz, N. 1997. Questionnaire design: The rocky road from concepts to answers. In Survey measurement and process quality. Edited by L. Lyberg, Paul Biemer, Martin Collins, et al., 29–46. New York: Wiley.

                                                                DOI: 10.1002/9781118490013Save Citation »Export Citation »E-mail Citation »

                                                                For readers unfamiliar with designing questionnaires, this chapter reviews the key elements involved in the cognitive and communicative aspects of the response process. Questionnaires are designed to measure concepts reliably, and the process of measurement error due to participants understanding a given question in different ways starts with question design.

                                                                Find this resource:

                                                                Mode of Data Collection

                                                                The mode of data collection chosen (e.g., face-to-face, telephone, postal, web interviews, or a mixed mode approach using more than one data collection mode) will have implications for both response rates and attrition as well as measurement effects (see also Causes and Consequences of Unit Nonresponse and Attrition and Minimizing Nonresponse and Attrition). Couper 2011 provides an overview of what is currently known about the effects on measurement and response of different modes of data collection. Dex and Gumy 2011 also provides a comprehensive review of the experience of using mixed modes on a range of social surveys. Most of the large national panel surveys use face-to-face or telephone data collection. Face-to-face interviews are generally viewed as producing the highest-quality data on a number of metrics, including achieving higher response rates and minimizing attrition as a relationship of trust is built up between the participant and the interviewer over several years (see also Causes and Consequences of Unit Nonresponse and Attrition). Face-to-face interviewers are also able to provide explanations to participants about specific questions and to probe on items where participants may have difficulty responding. With the increasing cost of face-to-face fieldwork, using telephone interviews and online methods has become increasingly attractive. Liefbroer 2011 reports on the experience of switching to a mixed-mode approach following earlier waves of face-to-face interviewing, while Dillman 2009 warns of some of the potential measurement consequences of doing so in a longitudinal survey. A major drawback of online panel surveys such as consumer web panels is the self-selected nature of the sample and lack of a probability sample. Scherpenzeel and Das 2011 reports on a panel study designed as an online panel and using a probability sample where those who do not have access to the Internet have this provided by the study. The growth of mixed-mode data collection is in its early days for panel studies, but it is gaining momentum due to pressure on fieldwork costs and falling response rates. There are both advantages and disadvantages of adopting a mixed-method approach, even though many of the effects on attrition and measurement error or mode effects in response behavior in the context of a longitudinal panel study are not yet known. Experimental methods are used to assess mode effects net of selection effects (Jäckle, et al. 2010; Vannieuwenhuyze, et al. 2011). To date, little is known about the effects on attrition of using mixed methods, but Lynn 2011 provides some evidence in the context of a randomized experiment in the United Kingdom.

                                                                Causes and Consequences of Unit Nonresponse and Attrition

                                                                Nonresponse can be a major problem for any social survey. Groves and Couper 1998 is considered a classic within survey methodology, and Groves, et al. 2000 sets out the “leverage saliency” theory, which provides one theoretical framework for research into nonresponse. Dillman 2000 provides another theoretical approach based on social exchange and how this can motivate people to take part, while Goyder 1987 draws attention to the potentially biasing effects of nonresponse. Nonresponse is a nonsampling error that has implications for measurement error when measuring change over time (Kalton, et al. 1989), and as a result minimizing nonresponse and attrition are the focus of much effort for those managing panel studies. When a sample member does not participate this is known as “unit nonresponse” and when a sample member drops out of the panel altogether through a failure to trace them or refusal or other noncontact, this is called “attrition” (see also Minimizing Nonresponse and Attrition). Initial unit response rates at wave 1 and subsequent reinterview response rates are commonly used as a marker of survey quality. Fricker and Tourangeau 2010 challenges this assumption, finding that higher response rates do not necessarily lead to better-quality data, as the motivation of reluctant respondents to answer accurately may be reduced. Nonetheless, if the initial nonresponse at wave 1 is high, the sample may be subject to nonresponse biases that cannot be estimated. High levels of dropout at subsequent waves can result in reducing the sample size, and in particular can produce small cell sizes for subgroups that can be problematic for analysts. There is also a risk of introducing biases if the attrition is not random but is systematically higher for some groups than others. The nonresponse may also be informative nonresponse; in other words, it may be related to the outcome of interest. At wave 1 of a panel study little is known about the characteristics of those who do not respond, but nonresponse at wave 2 provides rich evidence of the correlates of attrition from the wave 1 data (Lepkowski and Couper 2002). Nathan 1999 reviews sample attrition on three UK studies and finds that, despite attrition, the samples remain representative of the population over a long time period. The advantage of the panel design for secondary analysts is that it is possible to control for the characteristics associated with attrition in statistical models. For cross-sectional estimates, having detailed information about those who drop out from prior waves enables robust weighting strategies to be used (see also Weighting and Imputation).

                                                                • Dillman, D. A. 2000. Mail and Internet surveys: The tailored design method. New York: Wiley.

                                                                  Save Citation »Export Citation »E-mail Citation »

                                                                  This volume discusses the psychological and social motivations for participants to take part in a survey, and it develops a framework for maximizing response rates through tailoring the approach to the participant. A key text from a leading survey methodologist, which has become a standard within survey methodology.

                                                                  Find this resource:

                                                                  • Fricker, S., and R. Tourangeau. 2010. Examining the relationship between nonresponse propensity and data quality in two national household surveys. Public Opinion Quarterly 74.5: 934–955.

                                                                    DOI: 10.1093/poq/nfq064Save Citation »Export Citation »E-mail Citation »

                                                                    An interesting analysis that demonstrates that a higher response rate does not necessarily mean better-quality data in terms of item nonresponse, rounding, and other metrics of response quality. Those who take part reluctantly may be less motivated to answer accurately, and thus provide poorer-quality data.

                                                                    Find this resource:

                                                                    • Goyder, J. 1987. The silent minority: Nonrespondents on sample surveys. Cambridge, UK: Polity.

                                                                      Save Citation »Export Citation »E-mail Citation »

                                                                      An early volume drawing attention to the potentially biasing effects of unit nonresponse on sample surveys if the characteristics of the nonresponders are systematically different from the characteristics of those who respond.

                                                                      Find this resource:

                                                                      • Groves, R. M., and M. P. Couper. 1998. Nonresponse in household interview surveys. New York: Wiley.

                                                                        DOI: 10.1002/9781118490082Save Citation »Export Citation »E-mail Citation »

                                                                        Written by two leading survey methodologists, this is one of the most authoritative volumes detailing what is known about the causes and correlates of nonresponse and the potential effect on data quality. An essential text for anyone interested in survey methodology.

                                                                        Find this resource:

                                                                        • Groves, R. M., E. Singer, and A. Corning. 2000. Leverage-saliency theory of survey participation: Description and an illustration. Public Opinion Quarterly 64.3: 299–308.

                                                                          DOI: 10.1086/317990Save Citation »Export Citation »E-mail Citation »

                                                                          This article summarizes and provides an illustration of the leverage-saliency theory of survey response. This theory suggests potential participants to a study respond to design features of a given study in different ways, depending on how salient those features are to them as individuals. The theory expects variation in response propensity by subgroups within the population.

                                                                          Find this resource:

                                                                          • Kalton, G., D. Kasprzyk, and D. B. McMillen. 1989. Nonsampling errors in panel surveys. In Panel surveys. Edited by D. Kasprzyk, G. Duncan, G. Kalton, and M. P. Singh, 249–270. New York: Wiley.

                                                                            Save Citation »Export Citation »E-mail Citation »

                                                                            A key chapter from an early volume summarizing the main sources of nonsampling error in panel studies, including nonresponse and potential for measurement error when measuring change over time.

                                                                            Find this resource:

                                                                            • Lepkowski, J. M., and M. P. Couper. 2002. Nonresponse in the second wave of longitudinal household surveys. In Survey nonresponse. Edited by R. M. Groves, D. A. Dillman, J. L. Eltinge, and R. J. A. Little, 259–274. New York: Wiley.

                                                                              Save Citation »Export Citation »E-mail Citation »

                                                                              This chapter analyzes the correlates of nonresponse at the second round of longitudinal surveys, the point at which attrition from the sample is usually the greatest and potentially most damaging in terms of introducing bias.

                                                                              Find this resource:

                                                                              • Nathan, G. 1999. A review of sample attrition and representativeness in three longitudinal surveys (The British Household Panel Survey, the 1970 British Birth Cohort Study, and the National Child Development Study). Government Statistical Service, Methodology Series 13. London: Office for National Statistics.

                                                                                Save Citation »Export Citation »E-mail Citation »

                                                                                Compares the effect of attrition on three different UK surveys, one of which is a panel study and the other two birth cohort studies. Finds that despite attrition, the samples remain broadly representative of the population over a long time period.

                                                                                Find this resource:

                                                                                Minimizing Nonresponse and Attrition

                                                                                There are three phases to gaining the cooperation of survey participants. First one needs to locate the participant, second one needs to make contact, and third one needs to gain their cooperation. In a panel study of individuals and households, there are three main sources of nonresponse and attrition: (i) geographical mobility and failure to trace the participants; (ii) a failure to make contact at known addresses; and (iii) refusals when contact is made. Typically, attrition rates in panel studies are highest between waves 1 and 2 and tend to flatten out at later waves of the study, once the trust of the participant is gained and they identify with the aims and objectives of the study. Respondent burden in terms of interview length and complexity may also affect attrition, and the continuity of interviewers has been shown to reduce attrition (Hill and Willis 2001). Wave 1 response rates vary across studies and by country and will range from response rates of between 30 and 40 percent at the lower end up to response rates of 70 and 80 percent. Follow-up rates also vary widely, but typically panels reinterview between 80 and 95 percent of previous wave participants at the subsequent wave. Watson and Wooden 2009 identifies the factors associated with attrition, while Burton, et al. 2006 assesses the effectiveness of fieldwork procedures on a UK study for maintaining sample sizes. Laurie 2008 summarizes the various techniques used to reduce the three sources of nonresponse, including between-wave “keeping in touch” exercises to keep address details up-to-date and reduce tracing failures; providing feedback on results from the study to promote loyalty to the study; using stable contacts to trace participants who may move; fieldwork procedures such as ensuring interviewers make a minimum number of calls at a household; using refusal conversion techniques for “soft” refusals; and reissuing untraced movers and soft refusals at subsequent waves of the study. Interviewers are also a key aspect of tracing participants who move, as they can gain information about the participant’s new address from the current occupants or neighbors. Burgess 1989 examines the implications of failing to trace respondents for data quality, and Couper and Ofstedal 2009 reports on a “keeping in touch” experiment between waves to reduce tracing failures at the following wave. A similar experiment in the United Kingdom is discussed in Fumagelli, et al. 2010. Participant incentives are also common on longitudinal surveys and may take the form of small gifts, cash, or cash-like incentives. These have been shown to be effective in helping to minimize attrition (see also Participant Incentives).

                                                                                • Burgess, R. D. 1989. Major issues and implications of tracing survey respondents. In Panel surveys. Edited by D. Kasprzyk, G. Duncan, G. Kalton, and M. P. Singh, 52–74. New York: Wiley.

                                                                                  Save Citation »Export Citation »E-mail Citation »

                                                                                  An essential reading to introduce the importance and challenges of tracing participants in panel studies and the potential effects on data quality of failing to trace cases.

                                                                                  Find this resource:

                                                                                  • Burton, J., H. Laurie, and P. Lynn. 2006. The long-term effectiveness of procedures for minimising attrition on longitudinal surveys. Journal of the Royal Statistical Society, Series A (Statistics in Society) 69.3: 459–478.

                                                                                    DOI: 10.1111/j.1467-985X.2006.00415.xSave Citation »Export Citation »E-mail Citation »

                                                                                    Analyzes and evaluates the effectiveness of procedures used on the British Household Panel Study for maintaining sample sizes over the life of a long-running panel study.

                                                                                    Find this resource:

                                                                                    • Couper, M. P., and M. B. Ofstedal. 2009. Keeping in contact with mobile sample members. In Methodology of longitudinal surveys. Edited by P. Lynn, 183–202. Wiley Series in Survey Methodology. Chichester, UK: Wiley.

                                                                                      DOI: 10.1002/9780470743874Save Citation »Export Citation »E-mail Citation »

                                                                                      One of a very few experimental designs assessing the results of an experiment using different approaches to keeping in touch with sample members between interviews and subsequent effect at the next wave on response rates, tracing rates, and costs.

                                                                                      Find this resource:

                                                                                      • Fumagelli, L., P. Lynn, and H. Laurie. 2010. Experiments with methods to reduce attrition in longitudinal surveys. ISER Working Paper 2010-04. Colchester, UK: Institute for Social & Economic Research.

                                                                                        Save Citation »Export Citation »E-mail Citation »

                                                                                        Conducted at a similar point in time to Couper and Ofstedal 2009, an experiment on PSID, this paper reports the results of experimentally targeted approaches to between-wave contacts on the British Household Panel Study, aiming to increase the saliency of the study for participants and improve response rates at the subsequent wave.

                                                                                        Find this resource:

                                                                                        • Hill, D., and R. J. Willis. 2001. Reducing panel attrition: A search for effective policy instruments. Journal of Human Resources 36.3: 416–438.

                                                                                          DOI: 10.2307/3069625Save Citation »Export Citation »E-mail Citation »

                                                                                          This paper provides a useful analysis of the effects of interview length and interviewer continuity on response rates in the Health and Retirement Study in the United States. The authors find no effect of reducing the length of the questionnaire to reduce attrition, but that assigning participants the same interviewer at each wave significantly reduces attrition.

                                                                                          Find this resource:

                                                                                          • Laurie, H. 2008. Minimizing panel attrition. In Handbook of longitudinal research, design, measurement and analysis. Edited by S. Menard, 167–185. London: Elsevier.

                                                                                            Save Citation »Export Citation »E-mail Citation »

                                                                                            A useful review of procedures considered to be best practice for minimizing attrition in panel studies.

                                                                                            Find this resource:

                                                                                            • Watson, N., and M. Wooden. 2009. Identifying factors affecting longitudinal survey response. In Methodology of longitudinal surveys. Edited by P. Lynn, 157–179. Wiley Series in Survey Methodology. Chichester, UK: Wiley.

                                                                                              DOI: 10.1002/9780470743874Save Citation »Export Citation »E-mail Citation »

                                                                                              A helpful analysis of a number of national panel studies identifying the common factors associated with attrition; while there are common patterns, a large number of variables are associated with nonresponse, providing no easy solutions for fieldwork managers but providing analysts with information about which variables to include in their models to account for attrition.

                                                                                              Find this resource:

                                                                                              Participant Incentives

                                                                                              The use of participant incentives is common on panel studies. The rationale for using incentives is recognizing the commitment and additional burden put on participants when taking part in a long-running study. Participants are asked to give a significant amount of time on many occasions across what may be a long time period, making taking part in a panel study more burdensome than a single cross-sectional survey. Showing some appreciation of their continued contribution to the study is generally accepted to be positive for gaining cooperation and minimizing attrition and is accepted good practice. The types of incentives used on panel studies vary from small gifts to lottery tickets or cash or cash-like incentives. There is a significant body of experimental research examining the effect of incentives on response rates in a cross-sectional context, and Singer 2002 provides a useful review of these. There is less evidence for longitudinal surveys, even though one might expect some of the same mechanisms to operate in a longitudinal context as in a cross-sectional context. Laurie and Lynn 2009 reviews current practice on longitudinal studies and the use of incentives on the major national panel surveys. The methodological research on incentives has concentrated on three main areas: type of incentive offered, for example, gifts versus cash; the value of the incentive, such as varying the amount of a cash incentive offered; and the method of delivery—either conditional on response to the interview or unconditionally in advance of the interview request. Warriner, et al. 1996 examines the effectiveness of different types of incentives on response rates, and Couper, et al. 2006 tests the effect across different modes of data collection. James and Bolstein 1990 reports on the effect of the size of a monetary incentive, and Rodgers 2001 examines the effect of increasing the size of a monetary incentive in a longitudinal study. Willimack, et al. 1995 shows that a prepaid monetary incentive offered unconditionally in advance is more effective in increasing response rates than conditional incentives. The question of whether incentives have a cumulative effect on nonresponse and bias in a longitudinal survey and how incentives interact with the mode of data collection is addressed in Jäckle and Lynn 2008.

                                                                                              • Couper, M. P., E. Ryu, and R. W. Marans. 2006. Survey incentives: Cash vs. in-kind; face-to-face vs. mail; response rate vs. nonresponse error. International Journal of Public Opinion Research 18.1: 89–106.

                                                                                                DOI: 10.1093/ijpor/edh089Save Citation »Export Citation »E-mail Citation »

                                                                                                This paper reports an experimental design that tested the effects of cash vs. noncash incentives on response rates, but also tested this across different modes.

                                                                                                Find this resource:

                                                                                                • Jäckle, A., and P. Lynn. 2008. Participant incentives in a multi-mode panel survey: Cumulative effects on nonresponse and bias. Survey Methodology 34:105–117.

                                                                                                  Save Citation »Export Citation »E-mail Citation »

                                                                                                  An interesting paper analyzing of the effects of incentive payments on attrition, nonresponse bias, and item nonresponse, and whether these effects change across waves of a multimode panel survey of young people in the United Kingdom. Find incentives are an effective means of maintaining sample sizes, especially for subgroup analyses but incentives had no effect on attrition bias.

                                                                                                  Find this resource:

                                                                                                  • James, J. M., and R. Bolstein. 1990. Large monetary incentives and their effect on mail survey response rates. Public Opinion Quarterly 56.4: 442–453.

                                                                                                    DOI: 10.1086/269336Save Citation »Export Citation »E-mail Citation »

                                                                                                    Reports on an experiment offering varying amounts of monetary incentive on a mail survey.

                                                                                                    Find this resource:

                                                                                                    • Laurie, H., and P. Lynn. 2009. The use of respondent incentives on longitudinal surveys. In Methodology of longitudinal surveys. Edited by P. Lynn, 205–234. Wiley Series in Survey Methodology. Chichester, UK: Wiley.

                                                                                                      DOI: 10.1002/9780470743874Save Citation »Export Citation »E-mail Citation »

                                                                                                      A comprehensive review of what is known about the effects of incentives in longitudinal surveys and the incentive strategies currently used by national panel studies. While the evidence base suggests incentives are important for attrition, little is known about the optimum strategies to use in a longitudinal context.

                                                                                                      Find this resource:

                                                                                                      • Rodgers, W. L. 2001. Effects of increasing the incentive size in a longitudinal study. Journal of Official Statistics 27.2: 279–299.

                                                                                                        Save Citation »Export Citation »E-mail Citation »

                                                                                                        One of the few experimental designs conducted in a longitudinal context to test the effect on response and data quality of increasing the incentive at one wave of a panel study.

                                                                                                        Find this resource:

                                                                                                        • Singer, E. 2002. The use of incentives to reduce nonresponse in household surveys. In Survey nonresponse. Edited by R. M. Groves, D. A. Dillman, J. L. Eltinge, and R. J. A. Little, 163–177. New York: Wiley.

                                                                                                          Save Citation »Export Citation »E-mail Citation »

                                                                                                          A useful review chapter of experimental research on incentives primarily in the context of cross-sectional surveys and including across different modes of data collection.

                                                                                                          Find this resource:

                                                                                                          • Warriner, K., J. Goyder, H. Gjertsen, P. Hohner, and K. McSpurren. 1996. Charities, no; lotteries, no; cash, yes: Main effects in a Canadian incentive experiment. Public Opinion Quarterly 60.4: 542–562.

                                                                                                            DOI: 10.1086/297772Save Citation »Export Citation »E-mail Citation »

                                                                                                            Experimental design testing different types of incentives offered that concludes response rates are increased most by a cash incentive compared to payments to a charity or a lottery.

                                                                                                            Find this resource:

                                                                                                            • Willimack, D. K., H. Schuman, B.-E. Pennell, and J. M. Lepkowski. 1995. Effects of a prepaid nonmonetary incentive on response rates and response quality in a face-to-face survey. Public Opinion Quarterly 59.1: 78–92.

                                                                                                              DOI: 10.1086/269459Save Citation »Export Citation »E-mail Citation »

                                                                                                              Experimental design showing that unconditional incentives are more effective than conditional incentives.

                                                                                                              Find this resource:

                                                                                                              Ethical Issues in Panel Studies

                                                                                                              Panel studies are subject to the same ethical considerations as cross-sectional studies, but they introduce additional ethical aspects for data collection and data release. Guidelines from funders such as the Economic and Social Research Council (ESRC) in the United Kingdom and the National Institutes of Health in the United States are examples of the ethical standards commonly required. In any study the key issue is gaining informed consent from participants. At the initial contact, participants need to understand what they are being asked to do, how the data will be archived, and how it will be used. Kimmel 1989 and Jowell 1986 detail how this can be done, the ethical dilemmas researchers face, and how these can be resolved. Mertens and Ginsberg 2009 is a useful resource for graduate students and research practitioners faced with ethical considerations. As Lessof 2009 discusses, in a longitudinal survey participants need to understand that the survey is longitudinal and that they will be asked for further interviews. It also needs to be clear to participants they can refuse at any stage of the study, and by agreeing to be interviewed at one wave they are not agreeing to take part in all future waves. Participants also need to be reassured that the information they provide will be anonymized and they will not be identified. Confidential details of names and addresses must be held and maintained over an extended period of time to conduct the study, and procedures to ensure this information cannot be linked by data users to the survey data must be in place. In addition, where more than one household member may be interviewed, confidentiality within the household should be maintained. Finally, there are a number of data release considerations to protect the confidentiality of participants. The data in a longitudinal study are potentially liable to a greater risk of disclosure, as a combination of individual and household characteristics and life events over a period of time may make it easier to identify individual cases. Lyberg and Duncan 1993 is a special issue of the Journal of Official Statistics that focuses on issues of privacy and data disclosure, including statistical disclosure control. Statistical disclosure techniques may be applied on some studies, the data may be aggregated on some variables, while other identifying data may be suppressed. Groves 2004 sets out the common methods used by studies for ensuring the data released to users are not disclosive.

                                                                                                              • Economic and Social Research Council. ESRC Framework for Research Ethics 2010. Swindon, UK: Economic and Social Research Council. 2010.

                                                                                                                Save Citation »Export Citation »E-mail Citation »

                                                                                                                For those applying for UK social science funding, this framework is required reading, but it also provides a more general set of guidelines that will be valid for most social science research.

                                                                                                                Find this resource:

                                                                                                                • Ethical Guidelines & Regulations. National Institutes of Health.

                                                                                                                  Save Citation »Export Citation »E-mail Citation »

                                                                                                                  For those applying for US funding, particularly health-related funding, the guidelines and regulations that apply are given here.

                                                                                                                  Find this resource:

                                                                                                                  • Groves, R. M. 2004. Survey errors and survey costs. Hoboken, NJ: Wiley-Interscience.

                                                                                                                    Save Citation »Export Citation »E-mail Citation »

                                                                                                                    Identifies five common methods used for reducing the risk of data disclosure, including restricting geographical identifiers, data swapping, recoding and top-coding, perturbation (additive noise), and postrandomization (imputation to replace values). While these methods are often used, there can be consequences for the analysis of data that the secondary user needs to consider. First published in 1989.

                                                                                                                    Find this resource:

                                                                                                                    • Jowell, R. 1986. The codification of statistical ethics. Journal of Official Statistics 2.3: 217–253.

                                                                                                                      Save Citation »Export Citation »E-mail Citation »

                                                                                                                      An early article discussing the ethical issues in survey research that is still valid today.

                                                                                                                      Find this resource:

                                                                                                                      • Kimmel, A. J. 1989. Ethics and values in applied social research. Newbury Park, CA: SAGE.

                                                                                                                        Save Citation »Export Citation »E-mail Citation »

                                                                                                                        This helpful volume details the ethical problems and dilemmas faced by applied social science researchers. Using case studies, the author discusses the need to review ethical problems and their implications in the context of ethical standards in both society and the scientific community.

                                                                                                                        Find this resource:

                                                                                                                        • Lessof, C. 2009. Ethical issues in longitudinal surveys. In Methodology of longitudinal surveys. Edited by P. Lynn, 35–54. Wiley Series in Survey Methodology. Chichester, UK: Wiley.

                                                                                                                          DOI: 10.1002/9780470743874Save Citation »Export Citation »E-mail Citation »

                                                                                                                          An up-to-date review of the current status of ethical issues for any longitudinal study, as well as the aspects to be considered by the research team to build trust with participants and meet the ethical standards required.

                                                                                                                          Find this resource:

                                                                                                                          • Lyberg, L., and G. T. Duncan, eds. 1993. Special issue Journal of Official Statistics 9.2.

                                                                                                                            Save Citation »Export Citation »E-mail Citation »

                                                                                                                            This special issue of JOS contains various articles devoted to issues of privacy, data disclosure, and confidentiality. A good starting point to get an overview of the range of issues to be considered.

                                                                                                                            Find this resource:

                                                                                                                            • Mertens, D. M., and P. E. Ginsberg. 2009. The handbook of social research ethics. Thousand Oaks, CA: SAGE.

                                                                                                                              Save Citation »Export Citation »E-mail Citation »

                                                                                                                              Useful resource for graduate students, researchers, and practitioners of social research. A more in-depth discussion of the history, theory, philosophy, and implementation of applied social research ethics from a multidisciplinary perspective.

                                                                                                                              Find this resource:

                                                                                                                              Panel Data Sets for Secondary Analysis

                                                                                                                              There are many panel data sets freely available for secondary analysis, providing a wonderful resource for longitudinal secondary analysis. These data sets are usually available either from the research team responsible for conducting the study or from national data archives. Sample sizes vary across studies and range from around five thousand to ten thousand households to very large studies of individuals in forty thousand households. The major household panel studies include the Panel Study of Income Dynamics in the United States; the German Socio-Economic Panel Study; the British Household Panel Survey, which has now become the much larger Understanding Society: The UK Household Longitudinal Study, for the United Kingdom; the Canadian Survey of Labour and Income Dynamics; and the Household Income and Labour Dynamics in Australia survey. These studies have national samples and are designed to cover a range of research topics, including household composition, housing, geographic mobility, education, health, income, labor market behavior, attitudes, and values, as well as tracing key life events such as births, deaths, marriage, cohabitation, divorce, and geographical mobility. They therefore aim to be general purpose and suitable for interdisciplinary analysis. Studies such as the US Survey of Income Program Participation are designed to focus on a particular topic, such as the short-term dynamics of welfare program participation. Cross-national comparative research is facilitated by the Cross-National Equivalent File, which provides a data set with comparable measures drawn from a wide range of national panels. Studies provide extensive documentation, usually online, to help the new user understand the study design; how the fieldwork was conducted; how the data files are structured; technical information on sampling, weighting, and imputation; the questionnaires; the variables on the data set; and any other relevant information, such as response rates achieved or assessments of data quality. Many studies also run training workshops for data users, and these are usually advertised on study websites (see also Training Resources). Before starting to use a panel data set, a first step is to make sure the design is suitable to answer your research question and that the relevant measures are available. Most studies also have a facility for responding to queries or FAQs on their study websites that answer many common queries. Some of the main national panels are listed below, all of which are considered to be of the highest data quality. This is not an exhaustive list of panel studies.

                                                                                                                              Panel Analysis

                                                                                                                              The aim of panel analysis is to understand the process of change and generate models that accurately predict the direction of change in order to begin to make causal inferences that are not possible with cross-sectional data. Dale and Davies 1994 is an accessible guide to longitudinal analysis and a useful starting point for those new to these methods. The key analytic advantage of panel data is the ability to measure gross change (i.e., change measured at the level of the individual rather than at the aggregate level for a population). Aggregate measures from repeated cross-sectional surveys often show little change, but panel data reveal much higher levels of change for individuals within the population, such as movements into and out of poverty. Panel data also allow the study of relationships between variables measured at different time points, such as labor market outcomes dependent on earlier education outcomes or family background. Panel data also allow the estimation of spell durations for particular types of events and the effects on later outcomes. Survival analysis (also known as event history analysis or hazard modeling) is described in Keiley, et al. 2008, while Box-Steffensmeier and Jones 2004 provides examples of interpretation using Stata. The ability to control for cohort and period effects and for unobserved heterogeneity using fixed-effects models is a further advantage of panel data, as is the ability to observe both short-term dynamics such as employment transitions as well as longer term outcomes such as health outcomes dependent on earlier health behaviors. Set against the advantages of panel data for analysis are some of the data quality issues discussed in other sections. Skinner 2000 outlines alternatives for dealing with measurement error in longitudinal data (see also Questionnaire Design and Data Quality). There are many analytical approaches possible using panel data, and the approach chosen will depend on the research questions and the suitability of the data for a particular type of analysis. Common methods include fixed effect, random effect, or logistic regression approaches; event history analysis; structural equation models; and multilevel models. Allison 2009 shows how to estimate fixed effects models that control for unobserved heterogeneity in differing contexts; Singer and Willet 2003 and Bijleveld, et al 1998 describe the main approaches used for panel data analysis, with examples; and Hsiao 2003 is for the more statistically advanced data user.

                                                                                                                              • Allison, P. D. 2009. Fixed effects regression models. Los Angeles: SAGE.

                                                                                                                                Save Citation »Export Citation »E-mail Citation »

                                                                                                                                Accessible volume that describes the estimation and interpretation of fixed-effects models and is appropriate for postgraduates and researchers using panel data.

                                                                                                                                Find this resource:

                                                                                                                                • Bijleveld, C. C. J. H., L. J. T. van der Kamp, A. Mooijaart, et al. 1998. Longitudinal data analysis: Designs, models and methods. London: SAGE.

                                                                                                                                  Save Citation »Export Citation »E-mail Citation »

                                                                                                                                  A useful volume explaining the main approaches to panel analysis, including structural equation modeling and multilevel modeling. Examples are provided for each approach, and issues of design, measurement, and significance are considered.

                                                                                                                                  Find this resource:

                                                                                                                                  • Box-Steffensmeier, J. M., and B. S. Jones. 2004. Event history modeling: A guide for social scientists. Cambridge, UK: Cambridge Univ. Press.

                                                                                                                                    DOI: 10.1017/CBO9780511790874Save Citation »Export Citation »E-mail Citation »

                                                                                                                                    An accessible and up-to-date guide to event history analysis, with examples estimated and interpreted using standard statistical packages such as Stata. Diagnostics are discussed and the authors point out common problems and make recommendations for implementing duration modeling methods.

                                                                                                                                    Find this resource:

                                                                                                                                    • Dale, A., and B. Davies. 1994. Analyzing social and political change: A casebook of methods. London and Thousand Oaks, CA: SAGE.

                                                                                                                                      Save Citation »Export Citation »E-mail Citation »

                                                                                                                                      Provides an accessible guide to approaches to longitudinal analysis, illustrated through substantive examples and with discussion of the types of research objectives to which different techniques are suited. A helpful starting point for those less familiar with panel analysis.

                                                                                                                                      Find this resource:

                                                                                                                                      • Hsiao, C. 2003. Analysis of panel data. 2d ed. Cambridge, UK: Cambridge Univ. Press.

                                                                                                                                        DOI: 10.1017/CBO9780511754203Save Citation »Export Citation »E-mail Citation »

                                                                                                                                        This volume is for the more advanced statistical user. Provides comprehensive coverage of panel models including fixed effect and random effects regression models and chapters on incomplete panel data, simulation methods, and data with multi-level structures.

                                                                                                                                        Find this resource:

                                                                                                                                        • Keiley, M. K., N. C. Martin, J. Canino, J. D. Singer, and J. B. Willett. 2008. Discrete-time survival analysis: Predicting whether, and if so, when, an event occurs. In Handbook of longitudinal research, design, measurement and analysis. Edited by S. Menard. 441–464. London: Elsevier.

                                                                                                                                          Save Citation »Export Citation »E-mail Citation »

                                                                                                                                          Intended for analysts new to the technique, this chapter describes the use of discrete-time survival analysis of panel data (also known as event history analysis and hazard modeling), to estimate the hazard rate of an event occurring.

                                                                                                                                          Find this resource:

                                                                                                                                          • Singer, J. D., and J. B. Willett. 2003. Applied longitudinal data analysis. Oxford: Oxford Univ. Press.

                                                                                                                                            DOI: 10.1093/acprof:oso/9780195152968.001.0001Save Citation »Export Citation »E-mail Citation »

                                                                                                                                            This book is offers an accessible in-depth presentation of multilevel models for individual change and hazard/survival models for event occurrence. Uses clear, concise prose and real data sets from published studies, and a step-by-step approach through example analyses.

                                                                                                                                            Find this resource:

                                                                                                                                            • Skinner, C. 2000. Dealing with measurement error in panel analysis. In Researching social and economic change: The uses of household panel studies. Edited by D. Rose, 113–125. London: Routledge.

                                                                                                                                              Save Citation »Export Citation »E-mail Citation »

                                                                                                                                              A very helpful chapter focusing on measurement error, which may distort analyses by masking true change with spurious change due to misclassification of categorical variables. Provides three broad alternatives to dealing with measurement error.

                                                                                                                                              Find this resource:

                                                                                                                                              Weighting and Imputation

                                                                                                                                              Longitudinal panel studies normally provide some analysis weights as well as some imputed values for item missing data. Weighting and imputation on panel studies is more complex than a cross-sectional survey, and Kalton and Brick 2000 sets out the main steps involved in generating weights for panel studies. Weighting strategies typically include cross-sectional weights at each round of the study, weighted to represent the population at the time the sample was drawn. For a long-running panel study this is an important consideration, as the population may have changed in composition in various ways since the sample was drawn, in particular through immigration. How to weight for new members joining panel households also needs consideration, and Lavallee 1995 describes the implementation of the weight share method as one solution. Longitudinal weights to account for sample attrition are generated by most studies, even though the approach taken can vary (Rendtel and Harms 2009). Some studies provide longitudinal weights for each pair of years (German Socio-Economic Panel, cited under Panel Data Sets for Secondary Analysis) and others provide weights across all years of the study for those who have participated at every wave (British Household Panel Study, cited under Panel Data Sets for Secondary Analysis). There may also be differences in the weights depending on the questionnaire instrument used, as studies often have multiple questionnaire instruments at differing levels of analysis (e.g., household versus individual level). Lynn and Kaminska 2010 provides a good example of the decisions faced when designing a weighting strategy for a panel study. When conducting a descriptive longitudinal analysis, it is always advisable to use the longitudinal weights to allow for attrition. There is some debate about whether longitudinal weights should be used when using any form of longitudinal modeling, as the variables used to generate the weights are associated with attrition, and these variables are often included in the analytic model. Panel studies may also include imputed values for some key variables. These are often income-related variables where item nonresponse may be higher and potentially biasing. Little and Su 1989 discusses the implications for imputation of items that are Missing At Random or Missing Completely At Random. Imputation techniques vary and may be based on “hot-deck” methods, reviewed in Andridge and Little 2010. Rubin 2004 demonstrates how multiple imputation can be used, and Goldstein 2010 describes how multilevel modeling has become an accepted technique for handling missing data.

                                                                                                                                              • Andridge, R. R., and R. J. A. Little. 2010. A review of hot deck imputation for survey non-response. International Statistical Review 78.1: 40–64.

                                                                                                                                                DOI: 10.1111/j.1751-5823.2010.00103.xSave Citation »Export Citation »E-mail Citation »

                                                                                                                                                An up-to-date review written by experts in the field of the hot deck imputation method for handling missing data, in which each missing value is replaced with an observed response from a “similar” unit.

                                                                                                                                                Find this resource:

                                                                                                                                                • Goldstein, H. 2010. Multilevel statistical models. 4th ed. Chichester, UK: Wiley.

                                                                                                                                                  DOI: 10.1002/9780470973394Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                  Multilevel modeling has become a well-accepted statistical technique for handling missing data in surveys. This volume brings these techniques together, starting from basic ideas and illustrating how more complex models are derived.

                                                                                                                                                  Find this resource:

                                                                                                                                                  • Kalton, G., and M. Brick. 2000. Weighting in household panel surveys. In Researching social and economic change: The uses of household panel studies. Edited by D. Rose, 96–112. London: Routledge.

                                                                                                                                                    Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                    Sets out the main steps and issues in generating cross-sectional and longitudinal weights for a panel survey. The advantage for weighting a panel study is the wealth of information available about participants who drop out during the study.

                                                                                                                                                    Find this resource:

                                                                                                                                                    • Lavallee, P. 1995. Cross-sectional weighting of longitudinal surveys of individuals and households using the weight share method. Survey Methodology 21.1: 25–32.

                                                                                                                                                      Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                      This article describes the weight share method often used to provide weights for including new entrants to the household. The longer a panel study runs, the more important it is to be able to include new household members.

                                                                                                                                                      Find this resource:

                                                                                                                                                      • Little, R. J. A., and Hong-Lin Su. 1989. Item non-response in panel surveys. In Panel surveys. Edited by D. Kasprzyk, G. Duncan, G. Kalton, and M. P. Singh, 400–425. New York: Wiley.

                                                                                                                                                        Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                        Introduces the concepts of Missing at Random (MAR) and Missing Completely at Random (MCAR), and is a useful introduction to some alternative imputation approaches under each assumption.

                                                                                                                                                        Find this resource:

                                                                                                                                                        • Lynn, P., and O. Kaminska. 2010. Weighting strategy for Understanding Society. Understanding Society Working Paper 2010-5. Colchester, UK: Institute for Social & Economic Research.

                                                                                                                                                          Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                          Outlines the development of analysis weights for Understanding Society: The UK Household Longitudinal Survey (cited under Panel Data Sets for Secondary Analysis). A good example of the complexities of designing a weighting strategy in the context of practical and statistical issues for a survey with a complex design.

                                                                                                                                                          Find this resource:

                                                                                                                                                          • Rendtel, U., and T. Harms. 2009. Weighting and calibration for household panels. In Methodology of longitudinal surveys. Edited by P. Lynn, 265–286. Wiley Series in Survey Methodology. Chichester, UK: Wiley.

                                                                                                                                                            DOI: 10.1002/9780470743874Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                            Gives a formal description of the construction of weights under the following rules of a given study, adjusting for nonresponse and how to incorporate new household members not in the original sample.

                                                                                                                                                            Find this resource:

                                                                                                                                                            • Rubin, D. 2004. Multiple imputation for nonresponse in surveys. Hoboken, NJ: Wiley-Interscience.

                                                                                                                                                              Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                              Written by a leading statistician in the field, this volume demonstrates how nonresponse in sample surveys can be handled by replacing each missing value with two or more multiple imputations.

                                                                                                                                                              Find this resource:

                                                                                                                                                              back to top

                                                                                                                                                              Article

                                                                                                                                                              Up

                                                                                                                                                              Down