Nurse employer survey study design and methods.

              2006 Nurse Employer Survey

Study Design and Methods

April, 2007

The North Carolina Center for Nursing

Research Department staff: Linda M. Lacey

Associate Director: Research

Todd P. McNoldy

Research Associate

2006 Nurse Employer Survey: Study Design and Methods April, 2007

North Carolina Center for Nursing Page 1

Study Design

A two page questionnaire was mailed to the population of nurse employers in North Carolina

beginning in January, 2007. Non- respondents were contacted up to four times after the initial

mailing, approximately every two weeks. An additional copy of the questionnaire was included

with the first and third follow- up letters. Surveys were accepted through mid- March, 2007. The

survey asked respondents questions about nursing issues affecting them in fiscal year 2006,

which for most organizations ran from October 1st, 2005 to October 1st, 2006.

Population of Nurse Employers

Four different industry groups that rely on nurses as an integral part of their labor force were

surveyed as part of the 2006 Employer Survey: hospitals, long term care facilities, Medicare-certified

home health care and hospice agencies, and county- level public health departments.

In addition, for the first time we surveyed all state prison facilities that employed nurses.

Identified through Division of Facility Services directories for licensed health care facilities, or

other sources as noted below, the entire population of organizations in each industry group was

included in the study.

Hospitals were identified through lists maintained by the Division of Facility Services ( DFS),

which licenses all hospital facilities in the state, and from a list of North Carolina Hospital

Association member organizations. The lists include acute care general hospitals as well as

specialty, long- term acute care, and psychiatric hospitals. In addition, we include six hospitals

that are part of the Department of Defense system: four Veterans Administration hospitals, the

Naval Hospital at Camp Lejeune, and the Womack Army Medical Center at Fort Bragg. We

also added Cherokee Indian Hospital, which is run through the Indian Health Service and not

regulated by the DFS. These hospitals, although federally owned and operated, also draw upon

the nursing labor force in the state.

As part of the 2006 Nurse Employer Survey, we also sent a different psychiatric hospital survey

to all four of the publicly- funded psychiatric hospitals and all three of the private psychiatric

hospitals. We did not survey psychiatric facilities without in- patient services, nor did we survey

in- patient psychiatric facilities that were units of general hospitals.

A number of hospital and/ or health care systems exist within the state that include two or more

hospitals under their jurisdiction or management umbrella. We contacted the Chief Nursing

Officer ( CNO) in these systems to determine whether questionnaires could be completed

separately for each hospital facility or whether nurse planning and budgeting is done centrally

for the system. When possible, we requested that questionnaires be completed at the facility

level rather than the system level. In several cases, however, we aggregated multiple facilities

within a system on a single questionnaire in order to reduce the survey burden to the

respondent. In all, we consolidated twenty- three hospitals into surveys sent to seven large

hospitals systems – Carolinas HealthCare System, Cape Fear Valley Health System, Moses

Cone Health System, the Nash General System, the New Hanover Regional Medical Center

system, Presbyterian Hospitals, and the WakeMed hospital system.

Our key informant at each hospital was the Chief Nursing Officer ( CNO). Some questions on the

survey required coordination with Human Resource Departments, and some portions of the

questionnaire may have been delegated at the discretion of the CNO.

2006 Nurse Employer Survey: Study Design and Methods April, 2007

North Carolina Center for Nursing Page 2

All hospitals in the state, with the exceptions noted above, were included in the study and sent a

survey questionnaire. The total number of hospitals ( including psychiatric hospitals) and

hospital systems included in the study was 135.

Long Term Care facilities were identified through a list developed and maintained by the

Division of Facility Services. Every skilled nursing facility licensed to provide services in the

state was included in the study. Although there are many chains or systems operating long

term care facilities in North Carolina, most facilities are managed independently, as least in

terms of their nursing staff. For that reason we asked each facility in the state to respond to the

survey. Questionnaires were sent to the Directors of Nursing at each facility. The final

population list included a total of 393 long term care skilled nursing facilities. One facility was

removed from the list after we were unable to contact it.

Home Health and Hospice agencies included in the 2006 Survey of Nurse Employers were

identified through lists maintained by the Division of Facility Services. DFS manages two lists,

one for home care and one for hospices, and from these we created a subset of only those

agencies or facilities with Medicare/ Medicaid provider numbers. This removed from the

population list those home care agencies that do not specifically provide home health services

and which do not employ nurses. We then cross- referenced the Medicare/ Medicaid- certified

home health and hospice agencies with membership lists provided by the North Carolina

Association for Home and Hospice Care and the Center for End- of- Life Care to verify and

update agency names, addresses, and contacts.

Surveying home health and hospice agencies presents some unique problems because of this

industry group’s structure. Many agencies are operated by chains and systems, and unlike long

term care facilities, in many cases these agencies are unable to report nurse staffing data at the

facility level. We contacted the parent or corporate office of these chains to determine whether

branch and/ or unit offices could complete separate surveys. Whenever possible, we surveyed

chains at the facility- level. In 33 cases – or 17.3% of the organizations surveyed -- we combined

parent, branch and unit locations on a single questionnaire at the request of the facility

administrators. Most systems requesting aggregation in a single questionnaire operate in one

region of the state, but a few of them have agencies stretching across the state. When this

occurs the result is that these aggregated facilities must be dropped from any analysis using

geographic identifiers such as AHEC region, or location in metropolitan areas. In two cases,

respondents agreed to report staffing data separately for each region served by the system so

that they could at least be included in regional analysis of the data. Most decisions to aggregate

multiple facilities were agreed upon before the survey was put into the field.

Home health and hospice surveys were directed to the person responsible for nursing staff

planning – usually the facility administrator or the Director of Nursing if such was available.

Unlike other industry groups in this study, home health and hospice agencies were offered an

incentive for participation in the study. The North Carolina Association for Home and Hospice

Care donated one free conference registration for their annual meeting, which we awarded to

one randomly selected respondent. Mentions of the potential prize were included in the cover

letter and all reminder mailings.

The total number of home health and/ or hospice care agencies/ systems included in the study

was 194. We reduced our final population list by three cases when one agency responded with

aggregated data for itself and three other agencies in its system, for an adjusted N of 191.

2006 Nurse Employer Survey: Study Design and Methods April, 2007

North Carolina Center for Nursing Page 3

Public Health Departments across the state were also included in the study. We used the

‘ white pages’ of the 2006 Directory of Nurses Employed in Local Health Departments in North

Carolina, published by NC Department of Health and Human Services, to identify each facility

location ( some health departments operate in multiple locations) and the Director of Nursing at

each location.

There are a six regional public health districts that organize and administer multiple county

health departments. Where possible, we had facilities in each county of the conglomerate

complete separate questionnaires. In five cases, a single questionnaire covered multiple

facilities in a district. There were also cases where multiple physical facilities exist within a

single county under the purview of a single health department. In all of these cases we asked

the health departments to combine their multiple facilities and report the aggregated information.

The total number of health departments and regional districts included in the study was 88.

State Prisons were included for the first time on the 2006 survey. The North Carolina

Department of Corrections Director of Nursing provided a complete list of state prison facilities

that employ nursing personnel. Since facilities vary in size and in the level of care they provide

– from no care in small level 1 prison facilities to acute medical, psychiatric, and skilled nursing

care in very large, level 4 prison facilities – we sent the DON a separate survey for each prison

rather than aggregating them all on one survey plus a survey covering the DOC nursing

administrative office. The DON completed surveys for the central office and all but one of the

78 state prisons.

Response Rates

Response rates varied by industry group from a high of 98.7% for prisons to a low of 40.1% for

long term care facilities. The table below reports the final response rates. The second column

reports the number of cases in each industry group that were not deliverable ( bad addresses) or

were aggregated into a single survey after the initial survey mailing. This number effectively

reduces the size of the original group and is taken into account when calculating response

rates.

In all industry groups, except long term care, there were no undeliverable questionnaires. One

unplanned aggregation was allowed for a home health and hospice agency that answered for all

agencies in its system.

Industry Group # of Surveys

Sent Out

# of non-deliverables/

aggregations

# of Surveys

Returned

Adjusted

Response

Rate 1

Hospitals ( All) 135 0 86 63.7%

Non- psychiatric 128 0 81 63.3%

Psychiatric 7 0 5 71.4%

Home Health/ Hospice 194 3 124 64.9%

Long Term Care 393 1 157 40.1%

Public Health 88 0 76 86.4%

Prisons 79 0 78 98.7%

2006 Nurse Employer Survey: Study Design and Methods April, 2007

North Carolina Center for Nursing Page 4

1 The response rate is adjusted by subtracting the number of non- deliverables from the total number of surveys sent

out before calculating the response rate.

Bias Analysis

We conducted a bias analysis of the hospitals, home health and hospice agencies, and long

term care facilities that did and did not respond to our survey in order to determine if any

response bias exists. We were able to compare respondents and non- respondents in terms of

their geographic regional location in the state ( East, Central, West), and the metropolitan status

of their county location. We were also able to compare hospitals and long term care facilities in

terms of size ( i. e. licensed beds) in 2006. All prisons responded to our survey, and only twelve

public health departments did not respond to our survey, so a bias analysis was not necessary

for these groups.

Region

For each industry group – hospitals, long term care facilities, and home health and hospice

agencies -- respondent and non- respondent cases were assigned to their respective regional

groups ( see the section below for how geographic regions in the state and metropolitan status

were defined). We then compared the proportion of respondents and non- respondents in each

region, using chi- square to test for statistically significant differences. A similar procedure was

followed to test for differences by metropolitan status of the facility’s county. The results of this

analysis showed no statistically significant differences between the organizations that

responded to the survey and those that did not in terms of their geographic regional location or

location in a metropolitan area for any of the industry groups on which we conducted bias

analyses.

Size

Number of licensed beds in each hospital was obtained from the facility licensure data

maintained by the NC Division of Facility Services. After running the analysis of difference with

all cases, we also omitted the top and bottom 2% of cases in order to minimize the impact of

extremely large and small outliers in the respondent and non- respondent groups. We then

compared the average number of beds between the two groups, computing a 95% confidence

interval around each mean. This t- test of difference between averages indicated no significant

differences in size between our respondents and non- respondents for hospitals. We also

attempted to categorize hospitals by size, first by cutting hospitals into three types based on the

number of beds -- small, medium, and large – and then into just two sizes, small and large,

using the median number of beds ( 136) as the cutpoint. We employed a chi- square to test for

distribution differences between respondents and non- respondents for both the three- category

and the two- category hospital sizes. The three- category chi- square revealed a significant

difference in size between respondent and non- respondent groups with medium- sized hospitals

comprising 53.86% of the non- respondents. However, the two- category chi- square did not

indicate any significant differences between respondents and non- respondents. Since the two-category

hospital size results matched those found in the more robust t- test of differences

between respondents and non- respondents, we can feel confident in the generalizability of the

hospital data.

Licensed bed counts for long term care facilities in 2006 were obtained from the facility licensure

data maintained by the NC Division of Facility Services. As with hospitals, we created three

2006 Nurse Employer Survey: Study Design and Methods April, 2007

North Carolina Center for Nursing Page 5

categories for long term care facility size - small facilities had 90 or fewer beds, medium facilities

had 91- 120 beds, and large facilities had 121 or more beds. In addition, we split long term care

facility size into two categories, small and large, using the median number of beds ( 120) as the

cutpoint. The average number of licensed beds in responding long term care facilities was not

significantly different from the average number of beds in facilities that did not respond, and

comparisons of the distribution across small, medium and large facilities as well as across small

and large facilities did not reveal any response bias based on facility size.

Geographic Definitions

Each county in the state has been classified according to general region ( East, Central or

West), Area Health Education Center ( AHEC) service areas, and metropolitan status. The table

below lists each county’s regional, AHEC, and metro designation.

Three regions of the state have been defined for analysis purposes based on Area Health

Education Center ( AHEC) service areas. The Western region includes all of the counties

served by the Mountain and Northwest AHECs. The Central part of the state includes all of the

counties served by the Charlotte, Greensboro and Wake AHECs. The Eastern region includes

all of the counties served by the Area L, Coastal, Eastern and Southern Regional AHECs.

These three regions of the state, while not equal in size or population coverage, roughly

correspond to the distribution of health care market areas operating within the state.

This study uses the 2000 federal Office of Management and Budget definition of counties as

metropolitan, micropolitan, or neither. These classifications were developed using data from the

2000 Census. Metropolitan Statistical Areas have a core urban area with a population of at least

50,000 and include all counties with significant commuting and economic ties with the core.

Micropolitan Statistical Areas have a core urban population cluster of more than 10,000 but less

than 50,000 and include counties with ties to the core. Counties classified as neither

metropolitan nor micropolitan do not have sufficient ties to an urban area to be included under

its core- based statistical area umbrella. The metropolitan and micropolitan statistical areas

standards do not equate to a classification of rural or urban since all counties in a metro or

micro areas – as well as all other counties – contain both urban and rural territories and

population. i

2006 Nurse Employer Survey: Study Design and Methods April, 2007

North Carolina Center for Nursing Page 6

The table below contains the regional designation, AHEC area, and metropolitan status for

every county in North Carolina.

County Name

Regional

Designation AHEC Region

Metropolitan

Status

( 2000 OMB

Definition)

Alamance Central Greensboro Metropolitan

Alexander West Northwest Metropolitan

Alleghany West Northwest Neither

Anson Central Charlotte Metropolitan

Ashe West Northwest Neither

Avery West Northwest Neither

Beaufort East Eastern Micropolitan

Bertie East Eastern Neither

Bladen East Southern Regional Neither

Brunswick East Coastal Metropolitan

Buncombe West Mountain Metropolitan

Burke West Northwest Metropolitan

Cabarrus Central Charlotte Metropolitan

Caldwell West Northwest Metropolitan

Camden East Eastern Micropolitan

Carteret East Eastern Micropolitan

Caswell Central Greensboro Neither

Catawba West Northwest Metropolitan

Chatham Central Greensboro Metropolitan

Cherokee West Mountain Neither

Chowan East Eastern Neither

Clay West Mountain Neither

Cleveland Central Charlotte Micropolitan

Columbus East Coastal Neither

Craven East Eastern Micropolitan

Cumberland East Southern Regional Metropolitan

Currituck East Eastern Metropolitan

Dare East Eastern Micropolitan

Davidson West Northwest Micropolitan

Davie West Northwest Metropolitan

Duplin East Coastal Neither

Durham Central Wake Metropolitan

2006 Nurse Employer Survey: Study Design and Methods April, 2007

North Carolina Center for Nursing Page 7

Edgecombe East Area L Metropolitan

County Name Region AHEC

Metropolitan

Status

( 2000 OMB

Definition)

Forsyth West Northwest Metropolitan

Franklin Central Wake Metropolitan

Gaston Central Charlotte Metropolitan

Gates East Eastern Neither

Graham West Mountain Neither

Granville Central Wake Neither

Greene East Eastern Metropolitan

Guilford Central Greensboro Metropolitan

Halifax East Area L Micropolitan

Harnett East Southern Regional Micropolitan

Haywood West Mountain Metropolitan

Henderson West Mountain Metropolitan

Hertford East Eastern Neither

Hoke East Southern Regional Metropolitan

Hyde East Eastern Neither

Iredell West Northwest Micropolitan

Jackson West Mountain Neither

Johnston Central Wake Metropolitan

Jones East Eastern Micropolitan

Lee Central Wake Micropolitan

Lenoir East Eastern Micropolitan

Lincoln Central Charlotte Micropolitan

Macon West Mountain Neither

Madison West Mountain Neither

Martin East Eastern Metropolitan

McDowell West Mountain Neither

Mecklenburg Central Charlotte Metropolitan

Mitchell West Mountain Neither

Montgomery Central Greensboro Neither

Moore East Southern Regional Micropolitan

Nash East Area L Metropolitan

New Hanover East Coastal Metropolitan

Northampton East Area L Micropolitan

Onslow East Eastern Metropolitan

Orange Central Greensboro Metropolitan

Pamlico East Eastern Micropolitan

2006 Nurse Employer Survey: Study Design and Methods April, 2007

North Carolina Center for Nursing Page 8

Pasquotank East Eastern Micropolitan

Region AHEC

Metropolitan

Status

( 2000 OMB

Definition)

Pender East Coastal Metropolitan

Perquimans East Eastern Micropolitan

Person Central Wake Metropolitan

Pitt East Eastern Metropolitan

Polk West Mountain Neither

Randolph Central Greensboro Metropolitan

Richmond East Southern Regional Micropolitan

Robeson East Southern Regional Micropolitan

Rockingham Central Greensboro Metropolitan

Rowan West Northwest Micropolitan

Rutherford Central Charlotte Micropolitan

Sampson East Southern Regional Neither

Scotland East Southern Regional Micropolitan

Stanly Central Charlotte Micropolitan

Stokes West Northwest Metropolitan

Surry West Northwest Micropolitan

Swain West Mountain Neither

Transylvania West Mountain Micropolitan

Tyrrell East Eastern Neither

Union Central Charlotte Metropolitan

Vance Central Wake Micropolitan

Wake Central Wake Metropolitan

Warren Central Wake Neither

Washington East Eastern Neither

Watauga West Northwest Micropolitan

Wayne East Eastern Metropolitan

Wilkes West Northwest Micropolitan

Wilson East Area L Micropolitan

Yadkin West Northwest Metropolitan

Yancey West Mountain Neither

i See the Federal Register, December 27, 2000, for the standards used to define Metropolitan and Micropolitan Statistical Areas.