Problems with rates based on small numbers  Page 1 
Previous  1 of 6  Next 

small (250x250 max)
medium (500x500 max)
Large
Extra Large
large ( > 500x500)
Full Resolution
All (PDF)

This page
All

STATISTICAL PRIMER State Center for Health Statistics • 1908 Mail Service Center • Raleigh, NC 27699 1908 • 919/ 733 4728 www. schs. state. nc. us/ SCHS/ NORTH CAROLINA DEPARTMENT OF HEALTH AND HUMAN SERVICES No. 12 Originally Published April 1997; Revised August 2008 Problems with Rates Based on Small Numbers by Paul A. Buescher Introduction Most health professionals are aware that estimates based on a random sample of a population are subject to error due to sampling variability. Fewer people are aware that rates and percentages based on a full population count are also estimates subject to error. Random error may be substantial when the measure, such as a rate or percentage, has a small number of events in the numerator ( e. g., less than 20). A rate observed in a single year can be considered as a sample or estimate of the true or underlying rate. This idea of an “ underlying” rate is an abstract concept, since the rate observed in one year did actually occur. However, since annual observed rates may fluctuate dramatically, it is the underlying rate that health policies should seek to address. The larger the numerator of the observed rate, the better the observed rate will estimate the underlying rate. Many publications of the State Center for Health Statistics contain rates or percentages with a small numerator. This is a problem with a measure such as the infant mortality rate. In a single year many counties may have only one or two infant deaths and such rates in a small population may fluctuate dramatically from year to year. One means of addressing this problem is to look at five year rates where the numerator will be larger. Even with five year rates, however, many counties will have few events and therefore unstable rates. Many cause specific death rates for individual counties will have small numerators. This statistical problem is compounded when age adjusted rates are produced because, in the process of calculating an age adjusted rate by the direct method, the deaths and population are broken up into smaller groups. Rates are calculated for a number of specific age groups and numerators for each rate are often small. Some customers of the State Center for Health Statistics may treat our published rates and percentages as completely accurate. Unfortunately, there is the danger of making unwarranted comparisons between geographic areas or comparisons over time when the rates or percentages have small numerators. We do not consider it feasible to completely ignore all rates based on small numbers. In one sense, the rates do describe what actually happened in a year, but you must use caution and interpret any comparisons critically. The following section provides some methods for quantifying random errors in rates as a basis for making decisions about when changes or differences in rates are meaningful.
Object Description
Description
Title  Problems with rates based on small numbers  Page 1 
Pres Local File PathM  \Preservation_content\StatePubs\pubs_borndigital\images_master\ 
Full Text  STATISTICAL PRIMER State Center for Health Statistics • 1908 Mail Service Center • Raleigh, NC 27699 1908 • 919/ 733 4728 www. schs. state. nc. us/ SCHS/ NORTH CAROLINA DEPARTMENT OF HEALTH AND HUMAN SERVICES No. 12 Originally Published April 1997; Revised August 2008 Problems with Rates Based on Small Numbers by Paul A. Buescher Introduction Most health professionals are aware that estimates based on a random sample of a population are subject to error due to sampling variability. Fewer people are aware that rates and percentages based on a full population count are also estimates subject to error. Random error may be substantial when the measure, such as a rate or percentage, has a small number of events in the numerator ( e. g., less than 20). A rate observed in a single year can be considered as a sample or estimate of the true or underlying rate. This idea of an “ underlying” rate is an abstract concept, since the rate observed in one year did actually occur. However, since annual observed rates may fluctuate dramatically, it is the underlying rate that health policies should seek to address. The larger the numerator of the observed rate, the better the observed rate will estimate the underlying rate. Many publications of the State Center for Health Statistics contain rates or percentages with a small numerator. This is a problem with a measure such as the infant mortality rate. In a single year many counties may have only one or two infant deaths and such rates in a small population may fluctuate dramatically from year to year. One means of addressing this problem is to look at five year rates where the numerator will be larger. Even with five year rates, however, many counties will have few events and therefore unstable rates. Many cause specific death rates for individual counties will have small numerators. This statistical problem is compounded when age adjusted rates are produced because, in the process of calculating an age adjusted rate by the direct method, the deaths and population are broken up into smaller groups. Rates are calculated for a number of specific age groups and numerators for each rate are often small. Some customers of the State Center for Health Statistics may treat our published rates and percentages as completely accurate. Unfortunately, there is the danger of making unwarranted comparisons between geographic areas or comparisons over time when the rates or percentages have small numerators. We do not consider it feasible to completely ignore all rates based on small numbers. In one sense, the rates do describe what actually happened in a year, but you must use caution and interpret any comparisons critically. The following section provides some methods for quantifying random errors in rates as a basis for making decisions about when changes or differences in rates are meaningful. 