Feb 17 -- The National Center for Health Statistics (NCHS) invites comments by April 18, 2023 regarding the proposed new Rapid Surveys System (RSS). The RSS is a new survey system being designed to complement the current household survey systems at NCHS. The RSS will use survey data from probability-based online panels to produce time-sensitive estimates of new and emerging public health topics, attitudes, and behaviors.
Section 306 of the Public Health Service (PHS) Act (42 U.S.C.), as amended, authorizes that the Secretary of Health and Human Services (HHS), acting through NCHS, collect data about the health of the population of the United States. The NCHS Rapid Surveys System (RSS) will collect data on emerging public health topics, attitudes, and behaviors using cross-sectional samples from two commercially available, national probability-based online panels. The RSS will then combine these data to form estimates that approximate national representation in ways that many data collection approaches cannot. The RSS is intended to collect data in contexts in which decision makers' need for time-sensitive data of known quality about emerging and priority health concerns is a higher priority than their need for statistically unbiased estimates.
The RSS will complement NCHS's current household survey systems. As quicker turnaround surveys that require less accuracy and precision than CDC's more rigorous population representative surveys, the RSS will incorporate multiple mechanisms to carefully evaluate the resulting survey data for its appropriateness for use in public health surveillance and research (e.g., hypothesis generating) and facilitate continuous quality improvement by supplementing these panels with intensive efforts to understand how well the estimates reflect populations at most risk. The RSS data dissemination strategy will communicate the strengths and limitations of data collected through online probability panels as compared to more robust data collection methods.
The RSS has three major goals: (1) to provide CDC and other partners with time-sensitive data of known quality about emerging and priority health concerns; (2) to use these data collections to continue NCHS's evaluation of the quality of public health estimates generated from commercial online panels; and (3) to improve methods to communicate the appropriateness of public health estimates generated from commercial online panels.
Each round's questionnaire will consist of four main components: (1) basic demographic information on respondents to be used as covariates in analyses; (2) new, emerging, or supplemental content proposed by NCHS, other CDC Centers, Institute, and Offices, and other HHS agencies; (3) questions used for calibrating the survey weights; and (4) additional content selected by NCHS to evaluate against relevant benchmarks. NCHS will use questions from components (1) and (2) to provide relevant, timely data on new, emerging, and priority health topics to be used for decision making. NCHS will use questions from components (3) and (4) to weight and evaluate the quality of the estimates coming from questions in component (1) and (2). Components (1) and (2) will contain different topics in each round of the survey.
The Rapid Surveys System will use a platform designed to approximate national representation in ways that many data collection approaches cannot. Specifically, the platform is designed to collect self-reported health data using commercially available, statistically sampled national probability-based online panels. Since the resulting estimates will only approximate nationally representative estimates, the System includes some mechanisms to carefully evaluate the resulting survey data for its appropriateness for use in public health surveillance and research (e.g., hypothesis generating). As such, an important aspect of the System will be a data dissemination strategy that communicates the strengths and limitations of data collected through the System’s online panel approach, depending on the topic and sample, as compared to more robust data collection methods. The System also includes mechanisms to facilitate continuous quality improvement by supplementing these panels with intensive efforts to understand how well the estimates reflect populations at most risk.
Consistent with NCHS’s commitment to quality and professional standards of practice, the Rapid Surveys System will carefully evaluate the quality of the data and clearly communicate what is known about its validity, accuracy, and the appropriateness of the data for a given purpose. This quality assessment will include analytic evaluations of alternative methods for obtaining estimates after adjusting, calibrating, and modeling data from commercial survey panels with other data sources. It will benefit from statistical work with commercial survey panels completed over the past few years by the NCHS Research and Development Survey (RANDS) and from the ability to coordinate content with NCHS’s established surveys. These gold-standard surveys will be used to help calibrate the panel survey data. These gold-standard collections and other sources will also provide benchmarks to help judge the objectivity and utility of the Rapid Surveys estimates.
The Rapid Surveys are designed to have four rounds of data collection each year with two contractors. As part of the base (minimum sample size), Ipsos will be completing 2,000 surveys per quarter with KnowledgePanel respondents and NORC will be completing 2,000 surveys with AmeriSpeak respondents. Additionally, for the first year, NORC has also been contracted to do an additional 2,000 surveys each quarter. This leads to as many as 6,000 completed surveys per quarter or 24,000 surveys per year. The average time per response is expected to be 20 minutes.
The Rapid Surveys System can be expanded by increasing the number of completed surveys per round and/or the number of rounds per year as needed up to a maximum of 28,000 surveys per year per contractor or 56,000 total surveys per year.
As noted earlier, Ipsos and NORC have submitted plans for improving the representativeness of the panels. This may include adding oversamples using sampling strata based on demographics such as age, race/Hispanic ethnicity, education, household income, and gender, or recruiting additional respondents from opt-in nonprobability samples or from mailings to address-based sample. It may also include adding additional panelists to the sample and experimenting with increased incentives or enhanced recruitment efforts. Each contractor may complete up to 2,000 additional surveys per quarter (8,000 for the year) for that task. This increases the maximum burden by up to 16,000 surveys per year. These are also reflected on two separate lines on the table.
As noted earlier, the Rapid Surveys System may also target individual surveys to collect data only from specific subgroups within existing survey panels and may supplement data collection for such groups with additional respondents from other probability or nonprobability samples. An additional 12,000 surveys per year is included as a separate line in the table to reflect the possibility of these developmental activities.
Each respondent is expected to provide a single response; however, there may be situations where a panel member is sampled into the Rapid Surveys more than once; however, respondents are not expected to deliberately be sampled into the Rapid Surveys multiple times.
Cognitive interviews will also be conducted with 20 subjects per quarter for a total of 80 interviews per year. The average time per response is 60 minutes for a burden of 80 hours per year.
Draft technical documentation:
https://www.dropbox.com/scl/fi/zzef9ltde8fwhcdk89ic3/60-day-combined-request-form-Rapid-Surveys-02_7_23.docx?dl=0&rlkey=bjtlsfz1ggrqrxyg2uvfjgckq
FRN:
https://www.federalregister.gov/d/2023-03319