Vital and Health Statistics - CDC

 
NATIONAL CENTER FOR HEALTH STATISTICS

Vital and Health Statistics
Series 2, Number 175                                                                                   August 2017

National Center for Health Statistics
Data Presentation Standards for
Proportions
Data Evaluation and Methods Research

         U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES
         Centers for Disease Control and Prevention
         National Center for Health Statistics

                                             This report was revised on November 1, 2018, where on page 4, RSE(Var) was
                                             misattributed as RSE(SE). This revision had no effect on the related standard.
Copyright information

All material appearing in this report is in the public domain and may be
reproduced or copied without permission; citation as to source, however, is
appreciated.

Suggested citation

Parker JD, Talih M, Malec DJ, et al. National Center for Health Statistics Data
Presentation Standards for Proportions. National Center for Health Statistics. Vital
Health Stat 2(175). 2017.

Library of Congress Cataloging-in-Publication Data

Names: National Center for Health Statistics (U.S.), issuing body.
Title: National Center for Health Statistics Data Presentation Standards for
  Proportions.
Other titles: Vital and health statistics. Series 2, Data evaluation and
  methods research ; no. 175. | DHHS publication ; no. (PHS) 2017-1375.
  0276-4733
Description: Hyattsville, Maryland : U.S. Department of Health and Human
  Services, Centers for Disease Control and Prevention, National Center for
  Health Statistics, August 2017. | Series: Vital and health statistics.
  Series 2, Data evaluation and methods research ; number 175 | Series: DHHS
  pub ; number 2017-1375 | Includes bibliographical references and index.
Identifiers: LCCN 2017033531 | ISBN 0840606826 (pbk. : alk. paper)
Subjects: | MESH: National Center for Health Statistics (U.S.) | Data
  Accuracy | Health Surveys--standards | Data Display--standards | Research
  Design | Statistics as Topic | United States
Classification: LCC RA409 | NLM W2 A N148vb no.175 2017 | DDC 614.4/2--dc23
  LC record available at https://lccn.loc.gov/2017033531

For sale by the U.S. Government Printing Office
Superintendent of Documents
Mail Stop: SSOP
Washington, DC 20402–9328
Printed on acid-free paper.
Vital and Health Statistics
Series 2, Number 175

National Center for Health
Statistics Data Presentation
Standards for Proportions

Data Evaluation and Methods Research

U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES
Centers for Disease Control and Prevention
National Center for Health Statistics

Hyattsville, Maryland
August 2017
DHHS Publication No. 2017–1375
National Center for Health Statistics
Charles J. Rothwell, M.S., M.B.A., Director
Jennifer H. Madans, Ph.D., Associate Director for Science

Division of Research and Methodology
Charles J. Rothwell, M.S., M.B.A., Acting Director
Donald J. Malec, Ph.D., Associate Director for Science

Office of Analysis and Epidemiology
Irma E. Arispe, Ph.D., Director
Makram Talih, Ph.D., Associate Director for Science

Division of Health and Nutrition Examination Surveys
Kathryn S. Porter, M.D., M.S., Director
Ryne Paulose-Ram, Ph.D., Associate Director for Science

Division of Health Interview Statistics
Marcie L. Cynamon, Director
Stephen J. Blumberg, Ph.D., Associate Director for Science

Division of Health Care Statistics
Denys T. Lau, Ph.D., Acting Director
Alexander Strashny, Ph.D., Associate Director for Science

Division of Vital Statistics
Delton Atkinson, M.P.H., M.P.H., P.M.P., Director
Hanyu Ni, Ph.D., M.P.H., Associate Director for Science
Contents

Acknowledgments                                                                                                    iv
Abstract                                                                                                           1
Introduction                                                                                                       1
NCHS Data Presentation Standards for Proportions                                                                   2
  Sample Size                                                                                                      2
  Confidence Intervals                                                                                             3
  Degrees of Freedom                                                                                               4
  Complementary Proportions                                                                                        4
Discussion                                                                                                         5
References                                                                                                         6
Appendix I Figure Implementation of NCHS Data Presentation Standards for Proportions                               7
Appendix II Examples of the Application of NCHS Data Presentation Standards for Proportions                        8

Text Table
       NCHS Data Presentation Standards for Proportions                                                            2

Appendix Tables
I    Percentage of children aged 8–17 years with elevated and normal or borderline blood pressure, by race and
     Hispanic origin: United States, 2013–2014                                                                     8
II   Percentage of adults aged 60 and over who are overweight, by sex and race and Hispanic origin:
     United States, 2013–2014                                                                                      9
III. Percentage of adults aged 18 and over with hearing difficulties, by race and Hispanic origin and
     family income: United States, 2013                                                                            10
IV   Percentage of ambulatory care visits, by setting type according to diagnosis: United States, 2009–2010        11
V    Percentage of fathers aged 15–44 with children under age 5 years who live with their children,
     by how often they played with their children in the last 4 weeks and father’s age: United States, 2006–2010   12
VI   Proportion of triplet and higher-order multiple births, by age and race and Hispanic origin of mother for
     mothers aged 15–19 and 40–54: United States, 2015                                                             14

                                                                                                                   iii
Acknowledgments
     The authors would like to
acknowledge the contribution of Cynthia
A. Reuben, NCHS Office of Analysis and
Epidemiology and the contributions of
the following people who participated in
sessions on this topic in 2015 at the Joint
Statistical Meetings and the National
Center for Health Statistics (NCHS)
National Conference on Health Statistics:
Michael Davern, NORC at the University
of Chicago; Jennifer Madans, NCHS
Associate Director for Science; Virginia
Lesser, Oregon State University; Ron
Jarmin, U.S. Census Bureau; and Arthur
Hughes, Substance Abuse and Mental
Health Services Administration. This
report was edited and produced by the
NCHS Office of Information Services,
Information Design and Publishing Staff:
Jen Hurlburt edited the report, typesetting
was done by Erik L. Richardson
(contractor), and graphics were produced
by Dorothy M. Day.

iv
Abstract
Background
   The National Center for Health
                                          National Center for Health
Statistics (NCHS) disseminates
information on a broad range of           Statistics Data Presentation
                                          Standards for Proportions
health topics through diverse
publications. These publications
must rely on clear and transparent
presentation standards that can
be broadly and efficiently applied.       by the Data Suppression Workgroup: Jennifer D. Parker, Ph.D.,
Standards are particularly important      Division of Research and Methodology; Makram Talih, Ph.D., Office
for large, cross-cutting reports where
estimates cannot be individually
                                          of Analysis and Epidemiology; Donald J. Malec, Ph.D., Division of
evaluated and indicators of precision     Research and Methodology; Vladislav Beresovsky, Ph.D., Division of
cannot be included alongside the          Research and Methodology; Margaret Carroll, M.S.P.H., Division of
estimates.                                Health and Nutrition Examination Surveys; Joe Fred Gonzalez, Jr.,
                                          M.S., Division of Research and Methodology; Brady E. Hamilton,
Objective
                                          Ph.D., Division of Vital Statistics; Deborah D. Ingram, Ph.D., Office
   This report describes the NCHS
Data Presentation Standards for           of Analysis and Epidemiology; Kenneth Kochanek, M.A., Division of
Proportions.                              Vital Statistics; Frances McCarty, M.Ed., Ph.D., Division of Research
                                          and Methodology; Chris Moriarity, Ph.D., Division of Health Interview
Results                                   Statistics; Iris Shimizu, Ph.D., Division of Research and Methodology;
   The multistep NCHS Data                Alexander Strashny, Ph.D., Division of Health Care Statistics; and
Presentation Standards for
                                          Brian W. Ward, Ph.D., Division of Health Care Statistics
Proportions are based on a minimum
denominator sample size and on
the absolute and relative widths of
a confidence interval calculated          Introduction                                   in part, attributed to each data system’s
                                                                                         unique features and constraints.
using the Clopper-Pearson method.                                                        Standards also change over time, due
Proportions (usually multiplied by             The National Center for Health            to changes in the purpose and scope of
100 and expressed as percentages)
                                          Statistics (NCHS) collects, analyzes,          the data’s use, the feasibility of users
are the most commonly reported
                                          and disseminates information on a broad        carefully reviewing published estimates,
estimates in NCHS reports.
                                          range of health topics through diverse         the ability to provide explanatory text
                                          publications, databases, and tables. Some      discussing the precision of the published
Conclusions
                                          data products present information based        estimates, and advances in statistical
   The NCHS Data Presentation             on a single data system, while others          methodology.
Standards for Proportions will be
                                          summarize information from many data                This report describes the NCHS Data
applied to all NCHS publications.
                                          systems. These reports and data products       Presentation Standards for Proportions.
Using these Standards, some
estimates will be identified as           may include estimates on a wide range          Proportions (usually multiplied by 100
unreliable and suppressed and             of topics or focus on a particular health      and expressed as percentages) are the
some estimates will be flagged for        outcome. Furthermore, data products may        most commonly reported estimates
statistical review. For reports where     include estimates based on data sources        in NCHS reports. For many NCHS
estimates are evaluated individually, a   outside of NCHS. However, many of              reports, an automated rule is needed
particular proportion not meeting the     these reports do not display supporting        to determine whether or not estimates
NCHS Data Presentation Standards          information about an estimate to provide       are sufficiently stable for publication.
could be identified as unreliable         the reader with information about the          The multistep NCHS Data Presentation
but not be suppressed if it can be        estimate’s reliability, such as its standard   Standards for Proportions are based on
interpreted appropriately in the
                                          error (SE) or confidence interval (CI),        a minimum denominator sample size
context of subject-specific factors
                                          due to space and format constraints. As        and on criteria based on the absolute and
and report objectives.
                                          a result, reports must rely on clear and       relative widths of a CI calculated using
Keywords: confidence interval •           transparent presentation criteria that can     the Clopper-Pearson method (2,3). The
sample size • degrees of freedom •        be broadly and efficiently implemented.        next section provides details about the
health surveys • vital statistics              Statistical standards for data            Standards and their implementation. For
                                          presentation vary across agencies,             all NCHS data products, the Standards
                                          data systems, and data products (1).           will be applied and departures from the
                                          Differences among standards can be,            Standards will be justified. A flow chart

                                                                                                                            Page 1
Page 2  Series 2, No. 175

describing the Standards and examples of      Table. NCHS Data Presentation Standards for Proportions
their implementation are provided in the                            Statistic                                                       Standard
appendices.
                                              Sample size                                           Estimated proportions should be based on a minimum
                                                                                                     denominator sample size and effective denominator sample

NCHS Data
                                                                                                     size (when applicable) of 30. Estimates with either a
                                                                                                     denominator sample size or an effective denominator sample
                                                                                                     size (when applicable) less than 30 should be suppressed.

Presentation Standards                                                                                                                                    1
                                                                                                    If the number of events is 0 (or its complement ), then the
                                                                                                      denominator sample size should be used to obtain confidence

for Proportions                                                                                       intervals. If all other criteria are met for presentation, an
                                                                                                                                                          1
                                                                                                      estimate based on 0 events (or its complement ) should
                                                                                                      be flagged for statistical review by the clearance official.
                                                                                                      The review could result in either the presentation or the
                                                                                                      suppression of the proportion.
      The Table summarizes the NCHS
                                              Confidence interval                                   If the sample size criterion is met, calculate a 95% two-sided
Data Presentation Standards for                                                                       confidence interval using the Clopper-Pearson method, or the
Proportions, and Appendix I shows the                                                                 Korn-Graubard method for complex surveys, and obtain its
                                                                                                      width.
process used to implement the Standards.          Small absolute confidence interval width          If the absolute confidence interval width is greater than 0.00
When applying the NCHS Data                                                                           and less than or equal to 0.05, then the proportion can be
Presentation Standards for Proportions,                                                               presented if the number of events is greater than 0 and the
                                                                                                      degrees of freedom criterion (below) is met. If the number of
estimates identified as unreliable will                                                                                               1
                                                                                                      events is 0 (or its complement ) or the degrees of freedom
be suppressed. Other estimates will                                                                   criterion is not met, then the estimate should be flagged for
                                                                                                      statistical review by the clearance official. The review could
be flagged for statistical review by the                                                              result in either the presentation or the suppression of the
clearance official. Flagged estimates may                                                             proportion.
                                                  Large absolute confidence interval width          If the absolute confidence interval width is greater than or equal
be suppressed or presented, depending                                                                 to 0.30, then the proportion should be suppressed.
on the outcome of the review. To ease             Relative confidence interval width                If the absolute confidence interval width is between 0.05 and
production of large reports, the decision                                                             0.30 and the relative confidence interval width is more than
                                                                                                      130%, then the proportion should be suppressed.
process for flagged estimates could
                                                  Relative confidence interval width                If the absolute confidence interval width is between 0.05 and
be automated at the discretion of the                                                                 0.30 and the relative confidence interval width is less than
clearance official, with the suppression of                                                           or equal to 130%, then the proportion can be presented if
                                                                                                      the degrees of freedom criterion below is met. If the degrees
all flagged estimates. When an estimate                                                               of freedom criterion is not met, then the estimate should
is flagged or suppressed, a footnote                                                                  be flagged for statistical review by the clearance official.
                                                                                                      The review could result in either the presentation or the
indicating the reason the estimate has                                                                suppression of the proportion.
been flagg                                    Degrees of freedom                                    When applicable for complex surveys, if the sample size and
                                                                                                     confidence interval criteria are met for presentation and the
                                                                                                     degrees of freedom are fewer than 8, then the proportion
     The NCHS Data Presentation                                                                      should be flagged for statistical review by the clearance
Standards for Proportions will be applied                                                            official. This review may result in either the presentation or the
                                                                                                     suppression of the proportion.
to all NCHS reports and data systems.         Complementary proportions                             If all criteria are met for presenting the proportion but not for its
Importantly, the Standards presented                                                                                 1
                                                                                                      complement , then the proportion should be shown. A footnote
                                                                                                      indicating that the complement of the proportion may be
in this report apply to estimates that                                                                unreliable should be provided.
are known to meet all confidentiality         1
                                               The complement of a proportion p is (1 – p). The complement of the number of events in the numerator for p is the number of
requirements, or in the case of vital         events in the numerator for (1 – p).
statistics, meet all agreements between
NCHS and the states. Like other
statistical agencies, NCHS may restrict       America’s Children and Older Americans,                           Proportions are described more fully in
access to certain data or estimates to        and journal articles authored by NCHS                             the following sections.
protect the confidentiality of survey         staff members. For such reports, the
participants (4,5).                           analyst(s) should provide justification                           Sample Size
     Departures from the Standards            for including the estimate in the report
should be justified. In reports in which      to the clearance official(s) and a final                               The sample size (denominator) is
estimates are evaluated individually,         determination will be made jointly by                             an important indicator of an estimate’s
a particular estimate not meeting             the analyst(s) and clearance official(s)                          precision. The variance of a proportion
the Standards could be identified as          on a case-by-case basis. If the final                             is directly related to the sample size
unreliable but not be suppressed if it        determination is to present an estimate,                          and, with large samples, applying the
can be interpreted appropriately in           a footnote or appropriate language in the                         normal approximation to the binomial
the context of subject-specific factors       report indicating that the estimate does                          distribution for proportions can be useful
and report objectives. Currently such         not meet the NCHS Data Presentation                               for many analyses (6).
reports include, but are not limited to,      Standards should be provided.                                          For vital statistics, the sample size
Data Briefs, National Health Statistics            The specific components of the                               for a proportion is the number of births
Reports, multi-agency reports such as         NCHS Data Presentation Standards for                              (or deaths) in the denominator. For an
Series 2, No. 175  Page 3

estimate from a complex survey, the             Standard                                      and relative CI widths for estimated
effective sample size, ne, is defined as                                                      proportions are based on the 95% two-
the sample size, n, divided by the design       ● Estimated proportions should be based       sided Clopper-Pearson CI. The Clopper-
effect (7). One approach used to calculate        on a minimum denominator sample             Pearson CI is known to perform well for
ne for estimated proportions from a               size and effective denominator sample       estimated proportions with few events
complex sample survey is:                         size (when applicable) of 30. Estimates     in the numerator. For complex sample
                                                  with either a denominator sample size       surveys, the calculation of the Clopper-
                                                  or an effective denominator sample          Pearson CI using the approach of Korn and
                                                  size (when applicable) less than 30         Graubard incorporates information from
                                                  should be suppressed.                       the survey design, including the effective
where, in this case, the design effect is:      ● If the number of numerator events           sample size and, when appropriate, the
                                                  is 0 (or its complement), then the          degrees of freedom. Finally, the coverage
                                                  denominator sample size should be           of a 95% Clopper-Pearson CI is generally
                                                  used to obtain confidence intervals.        conservative, meaning that the Clopper-
                                                  If all other criteria are met for           Pearson CI includes the true proportion
                                                  presentation, an estimate based on 0        more than 95% of the time. Other intervals
     Documentation for specific surveys           events (or its complement) should be        have been shown to have poorer coverage,
should be consulted when calculating              flagged for statistical review by the       meaning that a 95% CI includes the true
design effects, as approaches can differ          clearance official. The review could        proportion less than 95% of the time
among surveys and for specific analytic           result in either the presentation or the    (10–12).
purposes.                                         suppression of the proportion.                    In particular, the commonly used
     If the number of numerator events                                                        Wald CI [p ± 1.96 × SE(p) for a two-sided
is 0 or equal to the denominator (the           Confidence Intervals                          95% CI] is known to perform poorly
complement of 0 events), the estimated                                                        for proportions (10–12). In addition to
proportion will be 0 or 1, respectively.              The NCHS Data Presentation              sometimes producing negative lower
As a result, the estimated variance of the      Standards for Proportions are based on the    bounds for small proportions or upper
proportion will be 0, and the effective         evaluation of absolute and relative 95%       bounds greater than 1 for large proportions,
sample size will be undefined. In these         CI widths. CIs provide a way to assess        the Wald CI does not always produce
cases, the sample size should be used to        an estimate’s precision, and technical        adequate coverage. In other words,
determine whether the minimum sample            definitions are available in many standard    published simulation studies demonstrate
size criterion is met, and it should also       statistical texts, including Bickel and       that the true proportion is contained within
be used for CIs and other computations          Doksum (8) and Casela and Berger (9).         a 95% Wald CI in less than 95% of the
that include the effective sample size.         More generally, under repeated sampling, if   simulated CIs, with greater undercoverage
Because observing no events or events           a proportion and its 95% CI are estimated     for smaller and larger proportions.
for everyone in a category can provide          from each sample, the true value of the             From a calculated CI, the absolute
important information (e.g., in the context     proportion is expected to be contained in     CI width is obtained by subtracting the
of rare health outcomes or conditions),         95% of the calculated intervals. A handful    value of the lower confidence limit from
estimates based on 0 events (or the             of methods to calculate CIs for proportions   the value of the upper confidence limit.
complement) that meet absolute CI and           are available and the expectation of 95%      The relative CI width is calculated as the
degrees of freedom criteria should be           coverage may not be attained for some         absolute CI width divided by the proportion
flagged and considered for presentation         intervals or under some conditions.           and multiplied by 100%. Past practice
after statistical review by a clearance         Methods used to calculate a CI lead to        for many surveys has been to evaluate
official to confirm the validity of the point   undercoverage if the true proportion is       relative standard errors (RSEs), which
and interval estimates.                         contained in fewer than the expected          are calculated by dividing the SE by the
     For complex sample surveys, due            number of intervals (e.g., less than 95%).    proportion and multiplying by 100%.
to sampling design and variability,             Conversely, methods are considered            However, when dividing the SE by very
there may be cases where the effective          conservative if the true proportion is        small proportions, the RSE can be too
sample size is greater than the sample          contained in more than the expected           conservative, and when dividing the SE
size. When the effective sample size is         number of intervals.                          by very large proportions, the RSE can be
greater than the sample size, the sample              The Clopper-Pearson CI (2) [adapted     too liberal. Because the relative CI width is
size should be used to determine whether        for complex surveys by Korn and Graubard      calculated in a similar manner, this property
the minimum sample size criterion is            (3) when applicable] should be used to        of the RSE also applies to the relative
met, and it should also be used for CIs         determine whether or not the CI of a          CI width. The NCHS Data Presentation
and other computations that include the         proportion meets the presentation criteria.   Standards for Proportions rely on both the
effective sample size.                          For the purposes of setting thresholds        relative and absolute CI widths to reduce
                                                for the Standards, the determinations of      the impact of this property.
                                                small and large in the context of absolute
Page 4  Series 2, No. 175

      For complex sample surveys, default           less than or equal to 130%,              Standard
calculations from survey software                   then if the degrees of freedom
may not be appropriate or feasible for              criterion described in the next          ● When applicable for complex
all situations, including age-adjusted              section is met, the proportion             surveys, if the sample size and CI
estimates, estimates based on multiple              can be presented. If the degrees           criteria are met for presentation and
imputation, estimates for subgroups                 of freedom criterion is not met,           the degrees of freedom are fewer
represented in only a subset of primary             then the estimate should be                than 8, then the proportion should be
sampling units (PSUs) (e.g., some                   flagged and statistically reviewed         flagged for statistical review by the
racial and ethnic groups and region-                by the clearance official. This            clearance official. This review may
specific estimates), and when calculating           review may result in either the            result in either the presentation or the
annual or survey cycle estimates using              presentation or the suppression of         suppression of the proportion.
a multiyear or multicycle data file. In             the proportion.
these instances, the relevant information                                                    Complementary
should be extracted and the CIs directly    Degrees of Freedom                               Proportions
calculated.
                                                 For complex sample surveys, the                   The SE and width of the CI for the
Standard                                    precision of the estimated variance is           complement of a proportion (1 – p) are
                                            approximately related to the square root         the same as those for the proportion,
● If the sample size criterion is met,      of the degrees of freedom. Using resulting       p. As described in previous sections,
  calculate a 95% two-sided confidence      SEs with low precision to assess estimated       relative measures for the smaller
  interval using the Clopper-Pearson        proportions may lead to poor measures            proportion are much larger than for its
  method (2), or the Korn-Graubard          of effective sample size and CI widths.          larger complement. Consequently, there
  method (3) for complex sample             Under certain conditions, the variance           is a range of proportions where the CI
  surveys, and obtain the width of the      estimate is approximately proportional to        criteria will yield conflicting assessments
  confidence interval.                      a chi-squared distribution, and the RSE          of reliability. For these proportions, the
    ○ If the absolute confidence            of the variance obtained from a complex          relative CI width may indicate that a
      interval width is greater than        sample survey can be approximated as             small proportion is unreliable but that its
      0.00 and less than or equal to                                           . From this   complement is not.
      0.05, then the proportion can be      expression, estimated proportions based                For a given health indicator or
      presented if the number of events     on fewer than 8 degrees of freedom have          publication, the larger proportion may
      is greater than 0 and the degrees     an              of 50% or higher.                be the most salient measure, while for
      of freedom criterion (described            As one rule of thumb, the degrees of        others, the smaller proportion may be
      in the next section) is met. If       freedom can be calculated as the number          the most important. Typically, both
      the number of events is 0 (or         of PSUs minus the number of strata. This         proportions are not shown (e.g., only
      the complement) or the degrees        calculation is used in most NCHS surveys         the proportion with health insurance
      of freedom criterion is not met,      and implemented in survey software,              would be shown, not both the proportion
      then the estimate should be           although specific calculations can vary          with and the proportion without health
      flagged and statistically reviewed    across packages. However, default                insurance). Given that the complement
      by the clearance official. This       calculations of degrees of freedom from          of the presented proportion can be
      review may result in either the       survey software may not be appropriate           determined by subtraction, consideration
      presentation or the suppression of    for subgroups represented in only a subset       of the precision of the complement
      the proportion.                       of PSUs (e.g., some racial and ethnic            is important. For some publications,
    ○ If the absolute confidence            groups and region-specific estimates) and        the practice has been to suppress both
      interval width is greater than or     when calculating annual or survey cycle          proportions if one of the proportions is
      equal to 0.30, then the proportion    estimates using a multiyear or multicycle        identified as unreliable. However, this
      should be suppressed.                 data file. In these instances, the relevant      practice may lead to the suppression of
    ○ If the absolute confidence            information should be extracted and the          important information.
      interval width is between             degrees of freedom directly calculated to
      0.05 and 0.30 and the relative        assess estimate precision. The calculation       Standard
      confidence interval width is more     of degrees of freedom as a measure of
      than 130%, then the proportion        precision for the SE may not be applicable       ● If all criteria are met for presenting
      should be suppressed.                 for all surveys (see survey-specific               the proportion but not for its
    ○ If the absolute confidence            documentation) and does not apply to               complement, then the proportion
      interval width is between             vital statistics. For additional information       should be shown. A footnote
      0.05 and 0.30 and the relative        on degrees of freedom, see Korn and                indicating that the complement of the
      confidence interval width is          Graubard (13) and Valliant and Rust (14).          proportion may be unreliable should
                                                                                               be provided.
Series 2, No. 175  Page 5

Discussion                                    previously, measures derived solely
                                              from the SE (e.g., Wald CI and RSE)
                                                                                            means are also regularly reported. Death
                                                                                            rates, for example, are calculated as the
                                              can perform poorly for proportions.           number of events reported in a calendar
     The NCHS Data Presentation               Consequently, whenever space permits,         year divided by the corresponding census
Standards for Proportions will be applied     appropriate CIs should be provided,           population estimate at the midpoint of
to all NCHS publications. Using these         rather than just SEs.                         the calendar year. The NCHS Standards
Standards, some estimates will be                  Age-adjusted estimates are often         were not developed to apply to these
identified as unreliable and suppressed       produced for national statistics. Age         estimators. Although the principles
and some estimates will be flagged for        adjustment allows for a comparison            considered by the workgroup for
statistical review. There are two scenarios   of outcomes between two groups with           proportions can be considered for other
where the estimates will be flagged for       differing age distributions, as many health   estimators, including the evaluation of
statistical review by a clearance official:   outcomes are highly correlated with age       effective sample size and CIs to guide
a) when the number of events is 0 (or its     (15,16). These estimates can be handled       decisions, no specific thresholds for
complement, equal to the denominator);        in a similar manner as unadjusted             these estimators are provided by these
and b) for estimates from sample surveys      estimates. There may be instances in          Standards. Rates calculated from vital
with fewer than 8 degrees of freedom.         which the age-adjusted estimates will         statistics will continue to use the criterion
Statistical review by a clearance official    not meet the presentation criteria but the    of requiring 20 or more events for
of flagged estimates will consider the        crude estimate would, or vice versa. In       reporting. In addition to precision, there
recommendation of the analyst(s) and          these cases, the estimate that meets the      are other statistical issues, not addressed
factors such as the estimate’s sample size,   presentation criteria will be shown, and      here, that affect the quality of the
CI, and degrees of freedom; the report’s      the one that does not will be suppressed.     estimates, including measurement error
objectives and format (including the          If CIs for age-adjusted estimates are not     and survey nonresponse (17).
ability to present CIs or other measures      readily obtained from survey software,             The NCHS Data Presentation
of precision); and the other estimates in     the relevant information can be extracted     Standards for Proportions can be
the report to determine whether to present    and the CIs can be directly calculated.       considered in planning analytic studies
or suppress the estimate. In some large            For the NCHS Data Presentation           or table shells for reports. Collapsing
reports, this process may be automated        Standards for Proportions, there is           subgroups and aggregating years leads
to ease the production process, with all      no minimum number of events (i.e.,            to estimates with less uncertainty and
flagged estimates suppressed without          numerator size). For the calculation of       might be more appropriate for a specific
review.                                       the Clopper-Pearson CI, a minimum             report than disaggregated estimates, even
     When the NCHS Data Presentation          numerator is not needed. A numerator          when minimum presentation guidelines
Standards for Proportions are used for        with few events may provide useful            are met. Other items to be considered
shorter, more focused reports, specific       information for some purposes. Although       may include the intended use of the
estimates that do not meet the standards      observing no events (or its complement,       estimate, the amount and structure of
may be reported after being evaluated         observing events for all records in           supporting information about uncertainty
individually by the analyst(s) and            the category) can provide important           that can be conveyed, and the availability
clearance official(s). Some estimates         information (e.g., in the context of          of corresponding estimates for other
identified as unreliable based on the         subgroup-specific analyses of rare [or        subgroups or other time periods.
Standards may be important and can be         prevalent] health outcomes), an estimate
interpreted appropriately in the context of
measures of precision and other subject-
                                              based on 0 events (or its complement)
                                              should be flagged for statistical review      References
specific information. In these cases, the     to confirm the validity of the point and
estimate could be presented. Because          interval estimates. In addition, inferences   1.   Klein RJ, Proctor SE, Boudreault
report objectives and subject-specific        based on the normal approximation,                 MA, Turczyn KM. Healthy People
factors vary widely, justification for        including statistical comparisons of               2010 criteria for data suppression.
presenting an unreliable estimate should      proportions between subgroups, require             Statistical Notes, no 24. Hyattsville,
be provided by the analyst(s) and final       a minimum number of events. Finally,               MD: National Center for Health
determination should be made by the           some estimates based on restricted data            Statistics. 2002.
analyst(s) and clearance official(s) on a     or vital statistics may require a minimum     2.   Clopper CJ, Pearson ES. The use
case-by-case basis. In all publications,      number of events to decrease disclosure            of confidence or fiducial limits
unreliable estimates, whether presented       risks, and reporting of these estimates            illustrated in the case of the
or suppressed, should be identified with a    may be based on different criteria (e.g.,          binomial. Biometrika 26(4):404–13.
footnote.                                     confidentiality).                                  1934.
     Many NCHS data products include               Although most estimates produced
SEs so that data users can assess the         in NCHS reports are proportions, other
precision of the point estimates. As stated   estimates, such as rates, percentiles, and
Page 6  Series 2, No. 175

3.    Korn EL, Graubard BI. Confidence         15. Anderson RN, Rosenberg HM. Age           25. Jones J, Mosher WD. Fathers’
      intervals for proportions with small         standardization of death rates:              involvement with their children:
      expected number of positive counts           Implementation of the year 2000              United States, 2006–2010. National
      estimated from survey data. Surv             standard. National vital statistics          health statistics reports; no 71.
      Methodol 24(2):193–201. 1998.                reports; vol 47 no 3. Hyattsville, MD:       Hyattsville, MD: National Center for
4.    National Center for Health Statistics.       National Center for Health Statistics.       Health Statistics. 2013.
      NCHS staff manual on                         1998.                                    26. National Center for Health Statistics.
      confidentiality. Available from:         16. National Center for Health Statistics.       The public use natality file—2015
      https://www.cdc.gov/nchs/data/misc/          Health, United States, 2015: With            update. 2016. Available from: ftp://
      staffmanual2004.pdf.                         special feature on racial and ethnic         ftp.cdc.gov/pub/Health_Statistics/
5.    National Center for Health Statistics        health disparities. Hyattsville, MD.         NCHS/Dataset_Documentation/
      Research Data Center. Disclosure             2016. Available from: https://               DVS/natality/UserGuide2015.pdf.
      manual: Preventing disclosure: Rules         www.cdc.gov/nchs/data/hus/               27. Martin JA, Hamilton BE, Osterman
      for researchers. Available from:             hus15.pdf.                                   MJK, et al. Births: Final data
      https://www.cdc.gov/rdc/data/b4/         17. Groves RM, Fowler Jr FJ, Couper              for 2015. National vital statistics
      disclosuremanual.pdf.                        MP, Lepkowski JM, Singer E,                  reports; vol 66 no 1. Hyattsville, MD:
6.    Johnson NL, Kotz S, Kemp AW.                 Tourangeau R. Survey methodology.            National Center for Health Statistics.
      Univariate discrete distributions. 2nd       Hoboken, NJ: John Wiley & Sons.              2017. Available from: https://
      edition. New York: John Wiley &              2004.                                        www.cdc.gov/nchs/data/nvsr/nvsr66/
      Sons, Inc. 114. 1992.                    18. National Center for Health Statistics.       nvsr66_01.pdf.
7.    Kish L. Survey sampling. New York:           National Health and Nutrition
      John Wiley & Sons, Inc. 1965.                Examination Survey. Available from:
8.    Bickel PJ, Doksum KA.                        https://www.cdc.gov/nchs/
      Mathematical statistics: Basic ideas         nhanes.htm.
      and selected topics, Volume 1,           19. National Center for Health Statistics.
      Second Edition. Chapman & Hall/              National Health Interview Survey.
      CRC. 2015.                                   Available from: https://www.cdc.
9.    Casella G, Berger RL. Statistical            gov/nchs/nhis.htm.
      inference. 2nd edition. Cengage          20. Parsons VL, Moriarity C, Jonas K, et
      Learning. 2001.                              al. Design and estimation for the
10.   Dean N, Pagano M. Evaluating                 National Health Interview Survey,
      confidence interval methods for              2006–2015. National Center for
      binomial proportions in clustered            Health Statistics. Vital Health Stat
      surveys. J Surv Stat Methodol                2(165). 2014.
      3(4):484–503. 2015.                      21. National Center for Health Statistics.
11.   Brown LD, Cai TT, Dasgupta A.                Health, United States, 2014: With
      Interval estimation for a binomial           special feature on adults aged 55–64.
      proportion. Stat Sci 16(2):101–17.           Hyattsville, MD. 2015. Available
      2001.                                        from: https://www.cdc.gov/nchs/data/
12.   Newcombe RG. Two-sided                       hus/hus14.pdf.
      confidence intervals for the single      22. National Center for Health Statistics.
      proportion: Comparison of seven              National Ambulatory Medical Care
      methods. Stat Med 17(8):857–72.              Survey. Available from:
      1998.                                        https://www.cdc.gov/nchs/ahcd/
13.   Korn EL, Graubard BI. Chapter                about_ahcd.htm#NAMCSNAMCES.
      5: Additional issues in variance         23. National Center for Health Statistics.
      estimation. In: Korn EL, Graubard            National Ambulatory Medical Care
      BI. Analysis of health surveys. New          Survey: 2012 state and national
      York, NY: John Wiley & Sons, Inc.            summary tables. Available from:
      192–234. 1999.                               https://www.cdc.gov/nchs/data/ahcd/
14.   Valliant R, Rust KF. Degrees of              namcs_summary/2012_namcs_web_
      freedom approximations and rules-of-         tables.pdf.
      thumb. J Off Stat 26(4):585–602.         24. National Center for Health Statistics.
      2010.                                        National Survey of Family Growth.
                                                   Available from: https://www.cdc.
                                                   gov/nchs/nsfg.htm.
Series 2, No. 175  Page 7

Appendix I. Figure. Implementation of NCHS Data Presentation
Standards for Proportions

                            Calculate effective
                               sample size

                           Is nominal or effective         YES
                              sample size < 30?

                               NO

                              Calculate 95%
                          confidence interval (CI)

                              Is absolute value            YES
                             of CI width ≥ 0.30?                   Suppress

                               NO

             YES             Is absolute value of
                               CI width ≤ 0.05?

                               NO

                             Calculate relative
                                 CI width

                             Is relative CI width          YES
                          > 130% of the proportion?

                               NO

                                                      NO                          NO
                                Is number of                     Are degrees of            Present
                                 events = 0?                      freedom < 8?

                               YES

                                 Statistical               YES
                                  review

    SOURCE: NCHS, 2017.
Page 8  Series 2, No. 175

Appendix II. Examples of the Application of NCHS Data Presentation
Standards for Proportions
     This appendix describes National                                   only and should not be used as the                                 Table I shows the percentage of
Center for Health Statistics (NCHS)                                     source for subject-specific information.                      children aged 8–17 years with elevated
data sets and provides examples of the                                                                                                blood pressure, a relatively rare outcome
application of the NCHS Presentation                                    National Health and                                           for children, by race and Hispanic origin
Standards using excerpts of previously                                                                                                using the 2013–2014 NHANES data.
published tables or new tabulations.                                    Nutrition Examination                                         Percentage estimates for elevated blood
Shown in these examples are: the                                        Survey                                                        pressure range from 0.5% among non-
sample size, the effective sample                                                                                                     Hispanic white children to 2.5% among
size (when applicable), the estimated                                        The National Health and Nutrition                        Hispanic children. For illustration, the
proportion, the standard error (SE) of                                  Examination Survey (NHANES)                                   complements of these percentages, the
the proportion, the upper and lower                                     is a program of studies designed to                           percentages with normal or borderline
bounds of the two-sided 95% Clopper-                                    assess the health and nutritional status                      elevated blood pressure, are also shown.
Pearson (or Korn-Graubard when                                          of adults and children in the United
                                                                        States. Currently, the survey examines                        ● All sample sizes and effective
applicable) confidence interval (CI), the
                                                                        a nationally representative sample of                           samples sizes are 30 or higher.
absolute CI width, the relative CI width,
                                                                        about 5,000 participants each year. These                     ● Absolute CI widths for elevated high
and, if appropriate, the degrees of
                                                                        participants are located in counties across                     blood pressure for non-Hispanic
freedom and the number of events in the
                                                                        the country, 15 of which are included in                        white and Hispanic children are less
numerator. Because survey estimates
                                                                        the sample each year. Data are released                         than 5 percentage points, so these
with high relative standard errors
                                                                        in 2-year cycles. The NHANES interview                          estimates can be presented if the
[RSE = 100% × (SE/Estimate)] have
                                                                        includes demographic, socioeconomic,                            number of events is not 0 (or equal to
been marked as potentially unreliable
                                                                        dietary, and health-related questions.                          the denominator) and there are 8 or
or suppressed in previous NCHS
                                                                        The examination component consists                              more degrees of freedom.
publications, the implications of the
                                                                        of medical, dental, and physiological                         ● Absolute CI widths for elevated
Standards are compared with decisions
                                                                        measurements, as well as laboratory tests                       high blood pressure for non-
based on the RSE       > 30% criterion
                                                                        administered by highly trained medical                          Hispanic black and Asian children
for survey estimates. Estimates in the
                                                                        personnel (18).                                                 are between 5 and 30 percentage
Appendix II tables are for illustration

Table I. Percentage of children aged 8–17 years with elevated and normal or borderline blood pressure, by race and Hispanic origin:
United States, 2013–2014

                                                                                                Standard    Relative
    Race and Hispanic origin                Sample        Effective         Percent              error of   standard    Lower            Upper          Absolute        Relative       Degrees of
                                                                    1                                               2         3                3                 4               5
      and blood pressure                     size       sample size         estimate             percent      error     bound            bound          CI width        CI width        freedom

Non-Hispanic white,
                                                           8
 elevated . . . . . . . . . . . . . . . .    460            583.0                    0.5          0.29        58.0         0.1             1.6             1.6            320.0             15
Non-Hispanic white,
                                                           8
 normal or borderline . . . . . . .          460            583.0                99.5             0.29         0.3       98.4             99.9             1.6               1.6            15
Non-Hispanic black,
                                                                                 6
 elevated . . . . . . . . . . . . . . . .    447            145.3                 2.0             1.17        58.5         0.3             6.4             6.1            305.0             13
Non-Hispanic black,
                                                                             7
 normal or borderline . . . . . . .          447               145.3          98.0                1.17         1.2       93.6             99.7             6.1               6.2            13
Non-Hispanic Asian,
                                                                                 6
 elevated . . . . . . . . . . . . . . . .    164            124.8                 0.9             0.83        92.2         0.0             5.1             5.1            566.7             13
Non-Hispanic Asian,
                                                                             7
 normal or borderline . . . . . . .          164            124.8             99.1                0.83         0.8       94.9           100.0              5.1              5.2             13
                                                           8
Hispanic, elevated . . . . . . . . .         589            946.8              2.5                0.51        20.4        1.4             4.1              2.7            108.0             15
Hispanic, normal or
                                                           8
 borderline . . . . . . . . . . . . . . .    589            946.8                97.5             0.51         0.5       95.9             98.6             2.7               2.8            15

1
 Sample size divided by the design effect. It is calculated in this table using this formula:                   .
2
 Standard error divided by the estimated percent multiplied by 100%.
3
 Lower and upper confidence bounds were computed using the Korn-Graubard method.
4
 Difference between the upper and lower confidence bounds.
5
 Absolute width divided by the estimated percent multiplied by 100%.
6
 Relative CI width is greater than 130%. Estimate would be suppressed.
7
 Estimate would have a footnote indicating its complement is suppressed.
8
 Effective sample size is greater than sample size. Sample size used in CI calculations.
NOTES: CI is confidence interval. Percent estimate and its lower and upper confidence bounds are equal to the proportion estimate and its lower and upper confidence bounds, respectively, multiplied
by 100. Standard error of percent and absolute CI width are expressed in percentage points. Numbers in the table are subject to rounding.
SOURCE: NCHS, National Health and Nutrition Examination Survey.
Series 2, No. 175  Page 9

       points, so the relative CI width                                        Table II shows the prevalence of                         ● The degrees of freedom for non-
       should be examined.                                                overweight among adults aged 60 and                             Hispanic Asian men and women are
●      Relative CI widths for elevated high                               over by sex and race and Hispanic                               7, indicating that these estimates
       blood pressure for non-Hispanic                                    origin using NHANES 2013–2014 data.                             would be flagged for statistical
       black and non-Hispanic Asian                                       Prevalence estimates for these subgroups                        review by the clearance official.
       children are greater than 130%, so                                 range from around 25% for non-Hispanic                        ● The RSE       for all subgroups is less
       these estimates would be suppressed.                               black and Asian women to more than                              than 30%. Under the RSE         > 30%
●      The number of events is greater than                               50% for Hispanic men.                                           suppression criterion, all estimates
       0 (and not equal to the denominator)                                                                                               would have been presented, whereas
                                                                          ● All sample sizes and effective
       and there are more than 8 degrees                                                                                                  using the NCHS Data Presentation
                                                                            sample sizes are 30 or higher.
       of freedom for all estimates, so                                                                                                   Standards for Proportions, the
                                                                          ● Absolute CI widths for men in each
       estimates that have met all other                                                                                                  percentage for non-Hispanic Asian
                                                                            race and Hispanic-origin group and
       sample size and CI criteria would be                                                                                               women would be suppressed.
                                                                            for Hispanic, non-Hispanic white,
       presented.
                                                                            and non-Hispanic black women are
●      If the outcome of interest is the
                                                                            between 5 and 30 percentage points,                         National Health Interview
       complement (normal or borderline
       high blood pressure), the absolute
                                                                            so the relative CI width would be                           Survey
                                                                            examined.
       CI widths are the same. However,                                                                                                      First fielded in 1957, the main
                                                                          ● The relative CI widths for men in
       the relative CI widths for non-                                                                                                  objective of the National Health
                                                                            each race and Hispanic-origin group
       Hispanic black and non-Hispanic                                                                                                  Interview Survey (NHIS) is to monitor
                                                                            and for Hispanic, non-Hispanic
       Asian children are less than 130%,                                                                                               the health of the U.S. population through
                                                                            white, and non-Hispanic black
       so these estimates would be                                                                                                      the collection and analysis of data on a
                                                                            women are all less than 130%, so
       presented with a note indicating                                                                                                 broad range of health topics (19). NHIS
                                                                            the percentages could be presented
       that their complements do not meet                                                                                               is conducted continuously throughout the
                                                                            if the degrees of freedom are 8 or
       presentation standards.                                                                                                          calendar year, collecting data from the
                                                                            greater.
●      Estimates of elevated blood pressure                                                                                             civilian noninstitutionalized population
                                                                          ● The absolute CI width for non-
       for non-Hispanic white children would                                                                                            of the United States. Each year, a
                                                                            Hispanic Asian women is 30.6
       be presented using the NCHS Data                                                                                                 representative sample of households
                                                                            percentage points, indicating that
       Presentation Standards for Proportions                                                                                           across the country is selected for NHIS
                                                                            the percentage would be suppressed.
       but would have been suppressed or                                                                                                using a multistage cluster sample design.
                                                                          ● The number of events for all
       identified as unreliable based on the                                                                                            Parsons et al. (2014) contains more
                                                                            estimates is greater than 0 (and not
       RSE       > 30% criterion.                                                                                                       information on the design and estimation
                                                                            equal to the denominator).
                                                                                                                                        methods for 2006–2015 (20).

Table II. Percentage of adults aged 60 and over who are overweight, by sex and race and Hispanic origin: United States, 2013–2014

                                                                          Effective              Standard       Relative
                                                                 Sample   sample      Percent     error of      standard        Lower          Upper        Absolute      Relative      Degrees of
                                                                            size1                                       2             3              3               4             5
       Sex and race and Hispanic origin                           size                estimate    percent         error         bound          bound        CI width      CI width       freedom

                         Men
Hispanic . . . . . . . . . . . . . . . . . . . . . . . . . . .    169       124.3      52.2         4.49           8.6           42.0              61.5       19.5           37.4            11
Non-Hispanic white. . . . . . . . . . . . . . . . . . .           384       246.2      38.6         3.10           8.0           32.0              47.1       15.2           39.3            15
                                                                           6
Non-Hispanic black. . . . . . . . . . . . . . . . . . .           195       240.7      34.9         3.06           8.8           27.6              45.2       17.6           50.5            12
                                                                           6          7
Non-Hispanic Asian . . . . . . . . . . . . . . . . . .             74       108.8       47.8        4.79          10.0           33.8              62.6       28.7           60.1             7

                      Women
Hispanic . . . . . . . . . . . . . . . . . . . . . . . . . . .    184        152.1      36.1        3.89          10.8           27.7              47.7       20.1           55.6            12
Non-Hispanic white. . . . . . . . . . . . . . . . . . .           466        296.8      34.3        2.76           8.0           28.4              42.3       13.8           40.4            15
                                                                           6
Non-Hispanic black. . . . . . . . . . . . . . . . . . .           175       236.5      25.1         2.83          11.3           18.2              36.5       18.3           72.8            12
                                                                           6          8
Non-Hispanic Asian . . . . . . . . . . . . . . . . . .             82       107.9       25.0        4.16          16.6           14.6              45.2       30.6          122.4             7

1
 Sample size divided by the design effect. It is calculated in this table using this formula:                   .
2
 Standard error divided by the estimated percent multiplied by 100%.
3
 Lower and upper confidence bounds were computed using the Korn-Graubard method.
4
 Difference between the upper and lower confidence bounds.
5
 Absolute width divided by the estimated percent multiplied by 100%.
6
 Effective sample size is greater than sample size. Sample size used for CI calculations.
7
 Degrees of freedom fewer than 8. Estimate would be flagged for statistical review by the clearance official and may be presented or suppressed.
8
 Absolute CI width is greater than or equal to 30. Estimate would be suppressed.
NOTES: CI is confidence interval. Overweight is defined as a body mass index (BMI) of at least 25 but less than 30 kilograms per square meter. BMI is calculated as weight (kilograms) divided by
height squared (meters). Percent estimate and its lower and upper confidence bounds are equal to the proportion estimate and its lower and upper confidence bounds, respectively, multiplied by 100.
Standard error of percent and absolute CI width are expressed in percentage points. Numbers in the table are subject to rounding.
SOURCE: NCHS, National Health and Nutrition Examination Survey.
Page 10 Series 2, No. 175

     Table III illustrates the application of                             denominator), so percentages that                             anesthesiologists, and radiologists) and to
the presentation standards using an                                       meet all other sample size and CI                             community health centers. The sampling
excerpt of data from the 2013 NHIS,                                       criteria for presentation would be                            frames for physicians are constructed
showing hearing difficulty among adults                                   presented.                                                    from the master files of the American
by race and Hispanic origin and family                                  ● The RSE        for several subgroups,                         Medical Association and the American
income relative to the federal poverty                                    including Hispanic adults with                                Osteopathic Association; the sampling
level (FPL). Percentage estimates range                                   incomes below the FPL, 200%–399%                              frames for community health centers
from 0.47% among non-Hispanic black                                       of the FPL, and 400% of the FPL or                            are obtained from the federal Health
adults with incomes 400% of the FPL                                       higher and non-Hispanic black adults                          Resources and Services Administration
and higher to 3.2% among non-Hispanic                                     with incomes 200%–399% of the                                 and the Indian Health Service. NAMCS
white adults with incomes below the                                       FPL and 400% of the FPL or higher                             data were collected on approximately
FPL. NHIS guidelines do not recommend                                     is greater than 30%. Percentages for                          32,000 and 31,000 visits to office-
calculating degrees of freedom using                                      these subgroups would have been                               based physician practices in samples
variance units available on the data file                                 suppressed or identified as unreliable                        of geographic primary sampling units
(https://www.cdc.gov/nchs/data/                                           using the RSE        > 30% criterion                          (PSUs) in 2009 and 2010, respectively.
nhis/2006var.pdf). For national estimates,                                but would be presented using the                                   The National Hospital Ambulatory
the degrees of freedom are assumed to be                                  NCHS Data Presentation Standards                              Medical Care Survey (NHAMCS)
large enough for a normal approximation.                                  for Proportions.                                              is a nationally representative survey
Table III is based on Table 49 in Health,                                                                                               of nonfederal general and short-stay
United States, 2014 (21).                                               National Ambulatory Medical                                     hospitals that is conducted annually.
                                                                                                                                        NHAMCS uses a multistage probability
● All sample sizes and effective                                        Care Survey and National                                        design with samples of geographic
  sample sizes are 30 or higher.
● All absolute CI widths are less
                                                                        Hospital Ambulatory Medical                                     PSUs, hospitals within PSUs, and
  than 5 percentage points, so all                                      Care Survey                                                     patient visits within emergency and
                                                                                                                                        outpatient departments. NHAMCS data
  percentages should be presented
                                                                             The National Ambulatory Medical                            were collected on approximately 35,000
  if the number of events is greater
                                                                        Care Survey (NAMCS) is an annual,                               emergency department visits in both
  than 0 (and not equal to the
                                                                        nationally representative survey                                2009 and 2010 and on approximately
  denominator).
                                                                        of visits to nonfederal office-based                            34,000 and 35,000 outpatient department
● The numbers of events are all
                                                                        physicians (excluding pathologists,                             visits in 2009 and 2010, respectively.
  greater than 0 (and not equal to the

Table III. Percentage of adults aged 18 and over with hearing difficulties, by race and Hispanic origin and family income: United States, 2013

                                                                     Effective                      Standard          Relative
      Race and Hispanic origin and                     Sample        sample           Percent        error of         standard          Lower            Upper           Absolute         Relative
                                                                           1                                                 2               3                 3                  4                5
    family income (percentage of FPL)                   size           size           estimate       percent            error           bound            bound           CI width         CI width

                    Hispanic
Under 100% . . . . . . . . . . . . . . . . . . . . .   1,675           559.3            1.43           0.50             35.2             0.62              2.79            2.18             152.7
100%–199% . . . . . . . . . . . . . . . . . . . . .    1,744         1,061.7            1.55           0.38             24.4             0.90              2.49            1.59             102.6
200%–399% . . . . . . . . . . . . . . . . . . . . .    1,621           586.8            1.10           0.43             39.1             0.42              2.32            1.90             172.0
400% and higher . . . . . . . . . . . . . . . . .        903           516.1            0.80           0.39             49.1             0.22              2.00            1.78             223.6

             Non-Hispanic white
Under 100% . . . . . . . . . . . . . . . . . . . . .   2,468         1,514.3            3.21           0.45             14.1             2.38              4.22            1.85              57.5
100%–199% . . . . . . . . . . . . . . . . . . . . .    3,650         3,167.7            2.86           0.30             10.3             2.31              3.51            1.19              41.7
200%–399% . . . . . . . . . . . . . . . . . . . . .    6,419         4,713.9            2.36           0.22              9.4             1.95              2.84            0.89              37.7
400% and higher . . . . . . . . . . . . . . . . .      8,020         4,909.4            1.78           0.19             10.5             1.43              2.19            0.76              42.7

             Non-Hispanic black
Under 100% . . . . . . . . . . . . . . . . . . . . .   1,470            912.8           1.36           0.38             28.2             0.72              2.35            1.63             119.6
                                                                    6
100%–199% . . . . . . . . . . . . . . . . . . . . .    1,367          1,421.8           1.11           0.28             25.0             0.63              1.83            1.20             107.4
200%–399% . . . . . . . . . . . . . . . . . . . . .    1,375          1,143.7           0.71           0.25             35.0             0.31              1.38            1.08             152.2
                                                                    6
400% and higher . . . . . . . . . . . . . . . . .        940          1,301.5           0.47           0.19             40.3             0.14              1.15            1.01             214.9

1
  Sample size divided by the design effect. It is calculated in this table using this formula:                .
2
  Standard error divided by the estimated percent multiplied by 100%.
3
  Lower and upper confidence bounds were computed using the Korn-Graubard method.
4
  Difference between the upper and lower confidence bounds.
5
  Absolute width divided by the estimated percent multiplied by 100%.
6
  Effective sample size is greater than the sample size. Sample size is used in CI calculations.
NOTES: FPL is federal poverty level. CI is confidence interval. Percent estimate and its lower and upper confidence bounds are equal to the proportion estimate and its lower and upper confidence
bounds, respectively, multiplied by 100. Standard error of percent and absolute CI width are expressed in percentage points. This table is based on Health, United States, 2014 Table 49. Numbers in the
table are subject to rounding.
SOURCE: NCHS, National Health Interview Survey.
Series 2, No. 175  Page 11

More details about the designs of                                              neoplasm of breast visits to primary                     so this estimate would have been
NAMCS and NHAMCS are available                                                 care offices are less than 30, so these                  suppressed or identified as unreliable
elsewhere (22,23).                                                             estimates would be suppressed.                           using the RSE       > 30% criterion
     Table IV shows the percent                                         ●      Absolute CI widths for the                               but would be presented using
distribution of ambulatory care visits                                         percentage of malignant neoplasms                        the NCHS Data Standards for
by setting type according to diagnosis                                         of colon and rectum visits and the                       Proportions.
group for two conditions: a) malignant                                         percentage of malignant neoplasm                       ● The RSEs       for the percentage of
neoplasms of the colon and rectum and                                          of breast visits to emergency                            malignant neoplasms of colon and
b) malignant neoplasm of the breast. As                                        departments are less than 5                              rectum visits and the percentage
these are percent distributions, the sum                                       percentage points, so these estimates                    of malignant neoplasm of breast
of the values across all settings is 100%,                                     could be presented if the number of                      visits in medical specialty physician
and the values range from less than 1% to                                      events is not 0 (and not equal to the                    offices are each less than 30%, so
more than 50%. As a result, if one value                                       denominators) and the other criteria                     these estimates would have been
is suppressed among the setting types, it                                      for presentation are met.                                presented using the RSE       > 30%
may be inferred from those shown. These                                 ●      Absolute CI widths for the                               suppression criterion but would be
data are an extract of a larger table found                                    percentage of malignant neoplasms                        suppressed using the NCHS Data
at: https://www.cdc.gov/nchs/data/ahcd/                                        of colon and rectum visits and the                       Standards for Proportions.
combined_tables/2009-2010_combined_                                            percentage of malignant neoplasm
web_table01.pdf. For this table, data for                                      of breast visits in medical specialty                  National Survey of Family
visits to office-based practices of                                            physician offices are greater than 30
physicians in primary care, surgical                                           percentage points, so these estimates                  Growth
specialties, and medical specialties                                           would be suppressed.                                        The primary purpose of the National
were obtained from the 2009–2010                                        ●      Degrees of freedom are all greater                     Survey of Family Growth (NSFG)
NAMCS, and data for visits to outpatient                                       than 8 and the numbers of events are                   is to produce national estimates of:
departments and emergency departments                                          all greater than 0 (and not equal to                   factors affecting pregnancy, including
were obtained from the 2009–2010                                               the denominator), so estimates that                    sexual activity, contraceptive use, and
NHAMCS.                                                                        meet all other sample size and CI                      infertility; medical care associated with
                                                                               criteria for presentation would be                     contraception, infertility, and childbirth;
● Design effects (not shown) for many
                                                                               presented.                                             factors affecting marriage, divorce,
  estimates are large. Effective sample
                                                                        ●      The RSE        for the percentage of                   cohabitation, and family building;
  sizes for the percentage of malignant
                                                                               malignant neoplasms of colon and                       adoption and caring for nonbiological
  neoplasms of colon and rectum
                                                                               rectum visits made to emergency                        children; fathers’ involvement with their
  visits and percentage of malignant
                                                                               departments is greater than 30%,                       children; use of sexual and reproductive

Table IV. Percentage of ambulatory care visits, by setting type according to diagnosis: United States, 2009–2010

                                                                          Effective                  Standard    Relative
                                                              Sample      sample          Percent     error of   standard       Lower         Upper         Absolute       Relative     Degrees of
                                                                            size1                                       2            3              3                4              5
            Diagnosis and setting type                         size                       estimate    percent      error        bound         bound         CI width       CI width      freedom

    Malignant neoplasms of colon and rectum
                                                                                            6
Primary care office . . . . . . . . . . . . . . . . . . . .    411             22.4         12.4       6.6         56.2           2.4           33.0          30.6          247.7            52
Surgical specialty office . . . . . . . . . . . . . . . .      411             68.1         17.3       4.6         26.5           9.2           28.4          19.2          110.6            52
                                                                                            7
Medical specialty office. . . . . . . . . . . . . . . . .      411             37.5         53.4       8.2         15.3          36.5           69.8          33.3           62.4            52
Hospital outpatient department . . . . . . . . . .             411             64.4         16.3       4.6         28.2           8.3           27.6          19.3          118.3            52
                                                                            8
Hospital emergency department. . . . . . . . . .               411            416.7          0.6       0.4         64.3           0.1            1.9           1.8          311.2            52

          Malignant neoplasm of breast
                                                                                            6
Primary care office . . . . . . . . . . . . . . . . . . . .   1,006          22.3             10.4     6.5         62.0           1.6           30.7          29.1          278.8            77
Surgical specialty office . . . . . . . . . . . . . . . .     1,006          97.8             19.4     4.0         20.6          12.1           28.6          16.5           85.3            77
                                                                                            7
Medical specialty office. . . . . . . . . . . . . . . . .     1,006          42.4             54.2     7.6         14.1          38.2           69.6          31.4           57.9            77
Hospital outpatient department . . . . . . . . . .            1,006          71.4             15.8     4.3         27.4           8.2           26.3          18.1          114.8            77
Hospital emergency department. . . . . . . . . .              1,006          57.7              0.2     0.1         27.4           0.0            0.8           0.7          316.8            77

1
  Sample size divided by the design effect. It is calculated in this table using this formula:               .
2
  Standard error divided by the estimated percent multiplied by 100%.
3
  Lower and upper confidence bounds were computed using the Korn-Graubard method.
4
  Difference between the upper and lower confidence bounds.
5
  Absolute width divided by the estimated percent multiplied by 100%.
6
  Effective sample size is less than 30. Estimate would be suppressed.
7
  Absolute CI width is greater than or equal to 30. Estimate would be suppressed.
8
  Effective sample size is greater than sample size. Sample size used in CI calculations.
NOTES: CI is confidence interval. Percent estimate and its lower and upper confidence bounds are equal to the proportion estimate and its lower and upper confidence bounds, respectively, multiplied
by 100. Standard error of percent and absolute CI width are expressed in percentage points. Numbers in the table are subject to rounding.
SOURCES: NCHS, National Ambulatory Medical Care Survey and National Hospital Ambulatory Medical Care Survey.
You can also read
NEXT SLIDES ... Cancel