1) BLS: CPS Response Rates
Declining response rates are a growing concern for the CPS as well as many other surveys. BLS and Census continuously monitor CPS response rates. The chart below provides the overall CPS response rate and response rates by month in sample (MIS) since 2013.
CPS overall response rate:
https://data.bls.gov/timeseries/LNU09300000&from_year=2013&output_type=column
CPS response rates by month in sample (MIS):
https://data.bls.gov/timeseries/LNU09397251&series_id=LNU09397252&series_id=LNU09397253&series_id=LNU09397254&series_id=LNU09397255&series_id=LNU09397256&series_id=LNU09397257&series_id=LNU09397258&from_year=2013&output_type=column
We measure nonresponse because it has a direct effect on data quality. If the rate of nonresponse is high, it increases the chance that the final survey estimates may reflect bias. CPS estimates may reflect bias if the characteristics and labor force statuses of nonresponding households differ from those of responding households.
Additionally, when a sample, rather than the entire population, is surveyed, there is a chance that the sample estimates may differ from the true population values they represent. The component of this difference that occurs because samples differ by chance is known as sampling error, and its variability is measured by the standard error of the estimate. Estimates involving many individuals generally have lower standard errors (relative to the size of the estimate) than estimates which are based on a small number of observations. A higher nonresponse rate contributes to an increase in the size of the standard error of estimates since the sample size observed is reduced from that originally sought.
The primary reasons for nonresponse in the CPS include privacy concerns, challenges making contact with respondents, and respondents being unavailable to provide data when contact is made. In cases where making contact with a respondent is the challenge, follow-up contacts are necessary to increase the chances of reaching the respondent. In recent years, it has become more difficult for Census interviewers to reach respondents, either in-person or by telephone.
CPS estimates have remained reliable despite declining response rates. However, a continued decline in the response rate would slowly erode the survey’s ability to detect meaningful change for our measures, particularly for estimates that are based on smaller sample sizes.
The U.S. Office of Management and Budget (OMB), in the Standards and Guidelines for Statistical Surveys, instructs statistical agencies to “plan for a nonresponse bias analysis if the expected unit response rate is below 80 percent.” These analyses help to determine where there is potential bias in survey estimates and assess how well the weighting adjustments are able to reduce any bias. The analysis provides transparency about what issues may exist in the data or survey estimates. See the following CPS nonresponse bias analyses: . . .
Maintaining survey response is just one way of ensuring the quality of CPS data. All data collection activities incorporate quality assurance assessments, including reinterviews with data providers, reviews by senior staff, computer-based data checks, and more. Information about the quality of published data is available to users through measures of variance, bias studies, and detailed descriptions of survey methods. Details on these and other steps to measure and ensure data quality can be found in the CPS methods documentation at
https://www.bls.gov/cps/documentation.htm
BLS and Census are working to address declining response rates by initiating a data collection modernization, intended to increase collection efficiencies and improve methods. This modernization will introduce a new self-response web collection method along with other data collection operational improvements. It also includes research into adaptive design, improved field training, and enhanced survey control systems.
BLS and Census are working together to develop a new web collection response option. CPS will continue to collect data by personal visit and telephone, but eligible households will also be offered the opportunity to report through a new web reporting system under development by Census.
CPS is making this update to provide an additional reporting option to respondents. Implementing web collection will reduce the effort required of survey respondents and improve collection efficiencies. Relying solely on personal visits and telephone has made it increasingly challenging to reach all respondents, and this has been evident in the declining response rates. The use of these traditional collection modes also has become progressively more expensive, which poses a challenge to the sustainability of the survey.
The development and implementation of this new collection tool will be challenging because CPS is committed to ensuring a successful transition that minimizes mode effects. Mode effects are inherent to the introduction of a new collection mode on a survey of this complexity. CPS plans to devote considerable resources to the evaluation and testing of an updated self-administered web questionnaire designed to minimize respondent burden and mode effects. Planning and development for this project will take place over a number of years, with an expected introduction to the survey in 2027.
CPS response rates webpage:
https://www.bls.gov/cps/methods/response_rates.htm
Household and establishment survey response rates:
https://www.bls.gov/osmr/response-rates/home.htm
2) Census: CPS 2023 Modernization Efforts
Conceived in the 1930s to meet the increased needs for unemployment statistics, the Current Population Survey (CPS) has been conducted in its present form since 1948, making it one of the oldest and most recognized surveys in the nation. Sponsored jointly by the U.S. Census Bureau and the U.S. Bureau of Labor Statistics (BLS), it is a primary source of labor force statistics for the U.S population, producing a key Principal Federal economic indicator. . . .
The CPS program leadership from the Census Bureau and BLS have agreed that modernization is needed to ensure the sustainability of the CPS. The Census Bureau and the Bureau of Labor Statistics have started the early stages of a multi-year effort to improve and modernize the operations of CPS. One of the largest efforts of this process is the introduction of an Internet self-response mode by 2027.
Over the years CPS has made changes to measure the ever-changing population and labor force characteristics of the US and implemented the latest in survey methodology practices. The last major change to the CPS was in the early 1990s when it went from paper data collection to computer assisted interviewing.
The response rates for all household surveys, including CPS, have been steadily declining for many years. Ten years ago, CPS regularly achieved response rates in the high 80 percentile. However, that is no longer true in the current environment. Issues like privacy concerns, challenges contacting respondents in cellphone only households, and respondents’ availability when contact is made have all contributed to the decline. By adding a new Internet self-response mode, it is anticipated that response rates can be increased or at the very least stabilized.
Declines in response rates are not uniquely a United States issue. International labor surveys have all seen a gradual decrease in participation rates. The quarterly United Kingdom Labour Force Survey (LS) has seen an even more drastic decrease in response rates over the same ten years when compared to CPS. Even the mandatory Labour Force Survey (LFS) administered by the Canadian govenment was not spared from these data collection challenges.
Across the board surveys have face higher non-response rates.
In addition, it has been found that a self -response mode can lead different types of households to respond to government-sponsored surveys. For people not interested in an interviewer-based data collection method, they may be more receptive to a self-response method. The American Community Survey (ACS) has done extensive research on this topic. . . . Internet modes have also been shown to perform comparably to mail modes as it pertains to numeric outlier analysis, correlations, and differences in response error. Lastly, research has shown that offering respondents multiple modes does not dampen response rates or have a significant impact on mode paralysis.
The first step in Modernization is to add an Internet Self Response (ISR) Mode. This section will review the testing and planning needed for this implementation. This is a multi-year project with the goal of extensively reviewing and testing ISR as a viable mode for CPS. The goal of testing is to ensure that no changes, including the option to complete the survey via ISR, affect the reliability of the estimates produced by CPS. This schedule seeks to bring the ISR mode to CPS as quickly as possible while doing the due diligence needed for such an important indicator. Our schedule and activities may change to accommodate unforeseen obstacles, including but not limited to funding restrictions, discovery of issues during testing and analysis, and project reprioritization.
The general timeline is as follows: [Proposed Five-year Project Phases]
In addition to the new internet mode, other modernization efforts currently on the way for CPS:
-- Census is deploying new operational control software to improve data collection processes. The new software will allow Census to improve cost-tracking, improve support for language barrier challenges, and better implement adaptive design. Field training will also be enhanced as part of the transition.
-- Research is being done on implementing adaptive design to implement cost savings while maintaining data quality. The design is looking at characteristics of a household from previous interview to predict if an interview is likely to be completed as well as based on the contact attempts for that data collection.
-- The Annual Social and Economic Supplement (ASEC) will also modernize. In addition to introducing internet self-response mode, review of the use of administrative records to help reduce respondent burden is being done.
-- Beginning research on other considerations related to the long-term sustainability of CPS. This includes analyzing current response rate trends and reviewing CPS as a whole to ensure the continuation of high-quality CPS data in the future.
Your feedback is important. If you have any questions, feedback or would like to be notified of updates, please email us at demo.cps@census.gov.
Census CPS modernization:
https://www.census.gov/programs-surveys/cps/about/modernization.html
3) Modernizing the Current Population Survey (CPS)-- Federal Economic Statistics Advisory Committee (FESAC) meeting (12.8.23)
-- The Effect of Declining Response Rates on CPS and LAUS Estimates, Justin McIllece, BLS
https://apps.bea.gov/fesac/meetings/2023-12-08/McIllece.pdf
-- Current Population Survey (CPS) Modernization, Kyra Linse, Census Bureau and Nicholas Johnson, BLS
https://apps.bea.gov/fesac/meetings/2023-12-08/Johnson-Linse.pdf
-- Modernizing the Current Population Survey, Discussant: Jason Faberman, FESAC Member
https://apps.bea.gov/fesac/meetings/2023-12-08/CPS_FESAC_discussion_Dec2023.pdf
Points of contact:
Kyra Linse Survey Director, CPS/American Time Use Survey Team, U.S. Census Bureau Kyra.M.Linse@census.gov
Nicholas Johnson Division Chief, Labor Force Statistics, BLS Johnson.Nicholas@bls.gov
