Guidelines governing local plans for gifted education

              GUIDELINES
Governing Local Plans
for Gifted Education
Public Schools of North Carolina
State Board of Education | Department of Public Instruction
Exceptional Children Division
Spring
2004
STATE BOARD OF EDUCATION
HOWARD N. LEE
Chairman
Raleigh
JANE P. NORWOOD
Vice Chair
Charlotte
KATHY A. TAFT
Greenville
MICHELLE HOWARD-VITAL
Wilmington
EDGAR D. MURPHY
Durham
EVELYN B. MONROE
West End
MARIA T. PALMER
Chapel Hill
ROBERT “TOM” SPEED
Boone
WAYNE MCDEVITT
Asheville
JOHN TATE III
Charlotte
PATRICIA NICKENS WILLOUGHBY
Raleigh
BEVERLY PERDUE
Lieutenant Governor
New Bern
RICHARD MOORE
State Treasurer
Kittrell
NC DEPARTMENT OF PUBLIC INSTRUCTION
Michael E. Ward, State Superintendent
301 N. Wilmington Street :: Raleigh, North Carolina 27601-2825 :: www.ncpublicschools.org
In compliance with federal law, including the provisions of Title IX of the Education Amendments of 1972, N C Public Schools
administers all state-operated educational programs, employment activities and admissions without discrimination because
of race, religion, national or ethnic origin, color, age, military service, disability, or gender, except where exemption is
appropriate and allowed by law.
Inquiries or complaints should be directed to:
Dr. Elsie C. Leak, Associate Superintendent :: Office of Curriculum and School Reform Services
6307 Mail Service Center :: Raleigh, NC 27699-6307 :: Telephone 919-807-3761 :: Fax 919-807-3767
Foreword
Historically, the mission of the North Carolina Department of Public Instruction has been
to provide leadership to serve and meet the educational needs of students so that the most
effective educational environments may be created with positive learning outcomes for
all students. The Exceptional Children Division has a strong commitment to provide an
appropriate education for all children so they may reach their fullest potential. Therefore,
it is imperative to recognize and develop giftedness among traditionally underrepresented
populations in the academically or intellectually gifted programs. Barriers related to
attitudes, equity and access must be recognized and overcome. As the “Third
Generation” plans are developed, major attention should be given to children from
culturally diverse, economically disadvantaged and/or disability populations.
Academically or intellectually (AIG) programs should be designed to assure that
traditionally underrepresented students who demonstrate characteristics of academic or
intellectual giftedness are recognized. These students should be provided opportunity to
experience the philosophy, rigor and challenge that gifted programs across our state offer.
AIG programs can provide the opportunity to help underachieving students from diverse
cultural and economic groups with exceptional potential to overcome the significant
achievement gap between advantaged and disadvantaged groups in our state. As program
guidelines based on best practices are developed, it must be recognized that the
implementation of any screening, identification and placement policies, programming
procedures and evaluations are viewed as a dynamic and ongoing process. The
Exceptional Children Division is committed to evolve with these paradigm changes as
educational assessment and instructional research practices continue to develop.
We hope you find Guidelines: Governing Local Plans for Gifted Education a useful tool
as you review, revise or rewrite your “Third Generation” AIG plans.
Mary N. Watson, Director
Exceptional Children Division
2
Acknowledgements
Recognition and appreciation is extended to the following individuals for their
contributions, time and effort in reviewing this document.
Ashe County – Jeannie Caviness
Avery County – Candace Englert
Buncombe County – Barbara Maloney
Buncombe County – Jeanette Christain
Charlotte-Mecklenburg – Sandra Payne
Elizabeth City-Pasquotank – Janet Swain
Greene County – Judy Davenport
Hoke County – Belinda Coleman
Hoke County - Maxine Colston
Hoke County – Debbie Stanley
McDowell County – Emily Wilson
Moore County – Beverlee Beale
Moore County – Donna Gephart
Pender County – Angela Jeffreys
Perquimans County – Pam Price
Warren County – Nanci Lucas
Wake County – Deborah Matthews
Wake County - Ellen Pritchett
Wake County – Dan Turner
Whiteville City Schools – Glenda Phillips
NCDPI - Valorie Hargett
3
Special Acknowledgements
On behalf of the Exceptional Children Division, Academically or Intellectually Gifted
(AIG) Program and the North Carolina AIG Guidelines Review Committee, a very
special recognition and appreciation is extended to the following individuals for their
contributions, time and effort in reviewing this document. Many hours of thought and
dedication to gifted children were donated. For this we are deeply appreciative.
Dr. E. Susanne Richert
Global Institute for Maximizing Potential, Inc.
Ocean Grove, New Jersey
Dr. Linda K. Silverman
Study of Advanced Development and the Gifted Development Center
Denver, Colorado
Dr. Holly Hertberg, Assistant Professor of Gifted Education
University of Virginia
4
Table of Contents
Historical Perspective ………………………………………………………………….. 5
Guidelines: Governing Local Plans for Gifted Education – Spring 2004 …………….. 6
Bibliography …………………………………………………………………………… 21
Appendix A
Citations From Dr. Linda Silverman’s Presentation – October 3, 2003
Radisson Research Triangle Park ……………………………………………… 25
Appendix B
Richert, E.S. (2003). Excellence With Justice in Identification and
Programming. In N. Colangelo & G.A. Davis (Eds.) Handbook of
Gifted Education (Third Edition) ……………………………………………… 31
Appendix C
Supplementary Materials for Screening and Identification of Gifted
Students ……………………………………………………………………….. 32
5
Historical Perspective
The North Carolina Department of Public Instruction commissioned the Task Force on
Academically Gifted Education in November 1993 at the direction of the General
Assembly. The Task Force published its recommendations in March 1994 and nine
model sites were chosen to pilot proposed changes. In January 1996 the Department of
Public Instruction reported the progress of the model sites to the Education Oversight
Committee of the General Assembly.
In July the General Assembly passed Article 9B effectively re-creating gifted education
in North Carolina to reflect the recommendations in the Task Force Report and the
planning process developed by the nine model sites. The 1996 legislation moved gifted
education from the law governing children with special needs. The Exceptional Children
Division oversees programs for academically or intellectually gifted.
All 117 school systems have developed local plans for the education of academically or
intellectually gifted students. The plans contain comprehensive descriptions of services
available to students in the local school systems. These plans are submitted to the State
Board of Education for review, comments and recommendations. The plans have had two
program reviews from the State Board since the implementation of Article 9B, and the
“Third Generation” review will be conducted in June 2004.
The guidelines for the development of local plans incorporate components of the statute
and serve as suggestions for best practice. They provide some statewide consistency in
the education of academically or intellectually gifted students. Information in bold print
has been lifted from the statute.
6
Guidelines
Governing Local Plans for Gifted Education
Spring 2004
North Carolina Definition of Giftedness
Academically or intellectually gifted students perform or show the potential to
perform at substantially high levels of accomplishment when compared with others
of their age, experience or environment. Academically or intellectually gifted
students exhibit high performance capability in intellectual areas, specific academic
fields or in both intellectual areas and specific academic fields. Academically or
intellectually gifted students require differentiated education services beyond those
ordinarily provided by the regular educational program. Outstanding abilities are
present in students from all cultural groups, across all economic strata and in all
areas of human endeavor.
Local Plans
Each local board of education shall develop a local three-year plan designed to
identify and establish a procedure for providing appropriate education services to
each academically or intellectually gifted student. The board shall include parents,
the school community, representatives of the community and others in the
development of this plan. The plan may be developed by or in conjunction with
other committees.
The Gifted Leadership Team
At the onset of planning, the gifted director/coordinator should form a 6-8 member
Steering Committee composed of central office senior administrators, principals,
gifted/regular teachers, parent representatives from various cultural and social-economic
groups in the community. This team selects participants for the at-large Gifted
Leadership Team that is representative of the local education agency (LEA) and
community. Invitations may be extended to school psychologists/counselors, curriculum
specialists, Exceptional Children Directors, school board members, parents, community
representatives and student representatives. The leadership committee will review the
existing gifted plan and will assess the strengths and weaknesses of the present program.
The committee should assess the over or under representation of cultural and
socioeconomic groups in the program and make recommendations/plans to address
inequities. These assessment documents may include, but not be limited to, past state
rubrics, state peer reviews, action plans, gifted student achievement data, child count
records and annual self-assessments. Annual surveys, focus groups, interviews and
questionnaires may be included to reveal strengths and weaknesses of existing programs.
Data from these sources provide valuable direction in setting goals, writing objectives
and making recommendations for the gifted program within the context of the total
7
school program. LEAs may write goals to improve or maintain existing programs; they
may write goals to reflect program component additions. Possible areas that goals may
address are as follows:
Procedures for Equitable Representation of Local Cultural and Socioeconomic Groups
Developing Local Norms for Norm-Referenced Test and Nomination Forms
Assessing Appropriateness of Multiple Program Options for Students with Diverse Needs
(Including Underachieving Students With Exceptional Potential Particularly Among
Under-Represented Populations)
Child Search/Referral Process/Screening/Identification
Administrator/Faculty/Parent Training
Academic Nurturing Programs (especially for LEAs and/or schools with low-incidence
gifted)
Highly Gifted Programs
Staffing Needs
Assessments
Program Evaluation
The above list should not be considered inclusive. Each Gifted Leadership Team
determines the number of goals, objectives and/or recommendations to forward to the
Steering Committee.
The Steering Committee and AIG director/coordinator will meet periodically during the
plan’s three-year duration to support and assess the goals and objectives established by
the Gifted Leadership Team.
Plan Revision
Based on the goals, objectives and recommendations established by the gifted leadership
team, the AIG coordinator/director and designees will review the AIG plan and how best
determine to infuse the goals, objectives and recommendation of the leadership
committee. The revised plan should reflect the data collected on equitable representation
of diverse cultural and socio-economic groups, achievement of identified gifted students
(current DEPs in place) and clearly establish procedures for assessing the achievement
and growth of academically or intellectually gifted students. In addition, the plan should
articulate the LEA’s efforts to prepare, identify and serve underrepresented populations.
Plan Approval
The local school board must approve the plan for serving the needs of academically or
intellectually gifted students.
8
Plan Review
Upon its approval of the plan, the local board shall submit the plan to the State
Board of Education for its review and comments. The local board shall consider the
comments it receives from the State Board before it implements the plan.
A plan shall remain in effect for no more than three years; however, the local board
may amend the plan as often as it considers necessary or appropriate. Any changes
to a plan shall be submitted to the State Board of Education for its review and
comments. The local board shall consider the State Board’s comments before it
implements the changes.
Upon the approval of the local school board, the local school system must send the plan
for gifted education to the Exceptional Children Division of the Department of Public
Instruction for preview to ascertain that all components of the statutes are addressed. If
all components are not present, the plan will be returned to the local school system for
completion. If all required components are present, the plan will be accepted for review.
Once the plan has been reviewed, comments recorded, and such information returned to
the local school board, the plan may be implemented.
The Planning Components
Screening, Identification and Placement
The plan shall include screening, identification and placement procedures that
allow for the identification of specific educational needs and for the assignment of
academically or intellectually gifted students to appropriate services .
Child Search/Referrals
Each local education agency should create the same level of awareness about the
academically or intellectually gifted program as it does about other significant school
programs. Information on the definition of giftedness, student referrals, basic eligibility
criteria, various program options and contact persons shall be made available to students,
parents, faculty, administrators and community members.
A “public awareness” campaign may effectively employ the following types of media to
inform multiple audiences on the gifted program and its services: Radio, television,
newsletters, newspapers, posters, brochures, videos, and letters.
The child find/referral process should be an extensive effort to locate all children who
may be potentially gifted by informing all stakeholders. Local education agencies may
seek assistance from other local agencies in order to provide a continuum of child find
services. These may include but are not limited to the following:
Title I Preschool Programs
9
Title I Program Directors
Day Care Centers
Head Start Programs
Public Health Departments
Community Service Group
Groups Representing or Serving Under-Represented Groups
Underserved Populations
A special effort should be made when designing “public awareness” campaigns for
finding hard-to-reach children whose parents may not be aware of the need for or the
availability of services for students with exceptional potential. Programs should also be
designed to inform individuals who may not understand English or those who may live in
rural or isolated geographical areas in the state.
Screening
Each system shall develop a screening component that includes a well-articulated
procedure for equal opportunity, equity/access to all students including minority students,
students with disabilities, students who are economically disadvantaged and other under-served
groups. The screening process should occur annually and include provisions for
an ongoing screening and selection process. Students who may show potential for
giftedness, those who show potential for performing at exceptionally high levels of
accomplishment in relation to their own demographic group in areas defined by the local
plan, and those who may have been missed throughout the year under the standard
screening process should be included in the screening procedure. The system should
include a “check and balance” procedure (inter-rater reliability) that insures that a child
being screened using multiple criteria in School A would also be included in the pool for
possible identification as gifted with the same data in School B in that LEA. Each
student should be viewed as an individual without limiting giftedness to any one score or
measure. The screening process establishes a broad-based pool of students for placement
considerations in service delivery options for academically or intellectually gifted
students and includes the following:
• The use of standardized instruments, process and performance indicators and
diverse sources of data (e.g., student, teacher, parent, peers or community
members) to get the most complete picture of the student whose needs for
programming are being assessed.
• The use of multiple criteria for decision-making that includes informal
assessments. No one criterion should exclude a child from consideration for
placement.
• The use of multiple sources of data to complement, not confirm each other.
• The use of assessment of the construct under consideration by using reliable and
valid instruments.
• The policy of identification or screening of students without using single cutoff
scores or summed matrix scores.
10
• The use of nontraditional methods for effective identification of underserved
populations who may manifest giftedness in different ways.
• The employment of No Child Left Behind (NCLB) disaggregated data, using
groups in order to establish local norms to find and meet the needs of underserved
populations.
(Callahan, Hunsaker, Adams, Moore, Bland, 1995; Richert 2003, Colangelo and Davis,
2003)
Needs Assessment for Program Placement
Differentiated Education
Identification/instructional or performance-based placement of a student in a program
option that provides differentiated instruction should be based on assessment of a
student’s academic and affective needs. The process should be structured to match
diverse student needs and multiple program options. The emphasis for placement should
be to match students with particular services. Program options should appropriately
differentiate both content level and instructional strategies so that students may achieve
their maximum intellectual and social potential. In order to prevent burdening or
constraining students with inappropriate expectations, labeling students with designations
of kinds or degrees of giftedness should be minimized or avoided.
As a minimum standard, a K-12 differentiated education program should be provided in
the areas of reading, writing and mathematics to align with NC’s accountability program
in measuring students’ achievement and growth. This differentiated educational
approach that involves extended thinking, applied thinking and conceptual thinking
builds upon and extends beyond the North Carolina Standard Course of Study. The
matching (placement) of students with the appropriate learning environment(s), content
modification(s) differentiated instructional strategies, including modified assessment, and
special program options will be determined by the student’s demonstrated abilities and/or
performance as well as by academic and social needs. It is recommended that LEAs
provide effective differentiation in the areas of social studies and science for gifted
students. Program options should be aligned with as well as expand the curriculum goals
listed in the North Carolina Standard Course of Study. Since resources are limited in
programs for gifted education, general education has an integral responsibility to
differentiate appropriately the curriculum for gifted students.
There is a symbiotic relationship between general education and program options for
students with gifted potential. The purpose of gifted program options is to provide for the
academic and social needs of identified students that cannot be met in general education.
General education, however, has two critical responsibilities in the education of the
gifted. First, the regular classroom is a de facto identification procedure. IF the
curriculum and instruction develops gifted potential, then students can be identified.
Second, students do not become “ungifted” in heterogeneous settings. Just as
modifications are necessary for included special education students included in general
11
education, identified gifted students need to be provided appropriate differentiated
instruction in heterogeneous settings (Richert, 2003).
Criteria for instructional placement should match the particular service delivery option.
The student’s demonstrated strength-based needs should be considered when the
committee makes program placement decisions.
Needs Determination Team
Each school (K-12) should form a committee to serve as a Needs Determination Team.
This committee should include individuals who have a background in gifted education
and/or experience in working with gifted students. These participants may be counselors,
school psychologists, ESL teachers, administrators and gifted/special/general education
teachers.
The duties and responsibilities of the Needs Determination Team are as follows:
• To implement the LEA’s gifted child search/referral, screening, identification and
instructional placement processes.
• To make decisions based on the demonstrated strength-based needs of the student.
Different options will be appropriate for different students depending on the level
of differentiation needed.
• To establish procedures to provide equitable access of students from diverse
cultural and economic backgrounds.
• To receive permission from the parent/guardian for any aptitude or achievement
testing that will not be generally administered in the regular school program.
• To record the information used in making the decision.
• To determine the program option(s) the student will access.
• To receive permission from the parents/guardian to place the student in the
option(s).
• To exhibit sensitivity to cultural, economic and/or linguistic differences that need
to considered in examining information about students’ needs.
The AIG coordinator/director should develop a system-wide procedure to assess the
membership and procedures of the school-based Needs Determination Team to ensure
that school-based decisions made at School A in the LEA would be the same decisions
those made at School B using the same criteria/assessment information. In addition,
procedures should be developed on how to address “borderline” students in order to
ensure equity and access by all students to the gifted program.
Decision Documentation
Each identified gifted student should have an annual Differentiated Education Plan
(DEP) or Individual Differentiated Education Plan (IDEP) that outlines the program
service option(s) appropriate for that student at each grade. The Differentiated Education
Plan lists the learning environments, content modifications and special programs
available to the student during that year. A student who clearly demonstrates outstanding
12
intellectual gifts but does not meet the criteria for the Differentiated Education Plan,
should be identified, and an Individual Differentiated Educational Plan should be
developed to meet his/her needs.
There should be a schedule for an annual performance review for each student in order to
follow student growth/ achievement and to determine the appropriateness of his/her
continuing to receive differentiated services in each service option (Richert, 2003). The
decision for a student to continue in a program service option should be based on the
student’s performance, academic or social need. Each option should have clearly defined
instructional goals that must be assessed annually to determine student growth. Student
progress will be recorded and shared with parents. If a student no longer requires
services in a particular option or options, the school-based Needs Determination Team
should convene to discuss the concerns and to consider more effective placement or
modifications in the student’s DEP. If the team recommends changes, the
parent/guardian should be invited to a conference to discuss the student’s performance.
When a gifted student is returned to a general education program, an Individual
Differentiated Education Plan (IDEP) is developed and support provided to the student in
order to reenter the gifted service option(s) at a later date, if and when appropriate.
Students do not become “ungifted” in heterogeneous settings. As for special education
students who are included in general education classes, the DEP/IDEP should specify
instructional modifications needed by identified gifted students in general education
classes that may include being excused from mastered skills or offered choices of more
challenging assignments (Richert, 2003).
In using grades as one of the multiple criteria for gifted placement decisions (entering or
exiting), it is imperative that record keeping teachers (AIG and general), administrators
and counselors have professional development in what is considered appropriate in
reporting student performance. When using grades to determine eligibility or
modification of placements, there are several cautions that must be considered. First, it is
appropriate to use grades to include, but not to exclude students from eligibility, if there
are other indicators of exceptional potential. Furthermore, once students’ have been
identified, grades may be used as a criterion to provide additional academic or social
support or to move students to a more effective option. Grades alone, however, unless
parents or students request it, are an insufficient criterion for exiting students from all
program services. Grades must not be used to penalize students in advanced program
options. Unless a weighted grading system is used, students’ academic grades should be
what they would have earned in heterogeneous placements with chronological peers
(Richert, 2003).
A student-led conference format is recommended for all annual and mid-term
performance reviews. Also, the plan may include provisions regarding furloughs,
reassessment, exiting of students from program services and transfer students. The intent
is to assure that all K-12 school-based teams are in alignment with one another and best
research practices are followed concerning the placement of potentially gifted and gifted
students within program options of that LEA.
13
Program Service Options
The plan shall include a clear statement of the program to be offered that includes
different types of services provided in a variety of settings to meet the diversity of
identified academically or intellectually gifted students.
The K-12 Program Service Options shall describe the array of differentiated services
available and the criteria for entry into each major option. Services should be determined
based on student need, not on availability. Equitable and appropriate access to services
for all students who have been identified as academically or intellectually gifted should
be provided system-wide. A base line of services that the system can offer system-wide
with equal access should be established. Since the needs of high achieving/advanced
students and highly gifted students vary widely, the program service options shall reflect
this diversity. The program service options applicable to the vast majority of
academically or intellectually gifted students should be listed in the Differentiated
Education Plan (DEP). Program learning options should indicate the learning
environment where differentiation will occur (advanced class, cluster group, resource
room, grade or content acceleration, dual enrollment, etc.), the method of content
modification (advanced language arts, advanced math, AP Calculus, AP English,
enrichment, etc.) and special programs that enhance learning (Odyssey of the Mind,
Future Problem Solving, Mathcounts, Invent America, mini courses, etc,). Program
service options should be described in a narrative form with local system-wide
instructional placement criteria established for each major option.
That very small group of unique students identified as academically or intellectually
gifted who do not fit into any programs listed in the Differentiated Education Plan should
have Individual Differentiated Education Plans to define and describe their program
service options.
High School Program Service Options
Research addressing gifted education at the high school level is relatively scarce.
Programs to address the needs of the gifted have not been as fully developed at the
secondary as at the elementary level. For any form of homogenous grouping at the high
school level, as at middle and elementary level, to qualify as a legitimate program option
for identified students, specific instructional modifications must be included. These
modifications should include, at a minimum, the highest levels of cognitive taxonomies
for presentation of content, instructional modifications that provide choices to evoke
intrinsic motivation, grouping to provide interaction with gifted peers, and evaluation
procedures that do not penalize identified students (Richert, 2003).
If such modifications to meet students’ intellectual as well as social needs are provided
and documented in DEPs, structures such as Advanced Placement (AP), honors courses,
early entrance to college, parallel enrollment at college and high school may qualify as
formal gifted program options (Richert, 2003). Also, a misconception exists that
Advanced Placement (AP) and honor courses address most, if not all, the needs of gifted
students. With the movement toward “open access” for these classes, it is essential that
14
gifted secondary programming move toward developing additional opportunities for
meeting the needs of gifted students beyond AP and honor classes in the academics and
social/emotional areas.
The high school Needs Determination Team, composed of counselors, administrators and
gifted/special/general education teachers, should meet annually (and ongoing when
appropriate) to continue to screen/identify and recommend/place gifted students in
appropriately rigorous and challenging classes. If a student fails to perform according to
potential or neglects to select an instructional placement that matches potential, that
student should be placed on an IDEP. This student should be provided support and
monitored for student growth and achievement over time. The team should provide
additional support in the areas of special programs, special schools/camps and
scholarships and other services beyond those provided within the local high school.
A student-led conference format is recommended at the high school level when
reviewing four-year plan and developing the annual Differentiated Education Plan in
order to increase student achievement and growth. This format may be used for
student/parent conferences as well. It is recommended that a DEP be attached to
identified AIG high school students’ four-year academic plans. These DEPs may be
reviewed and dated when the Needs Determination Team members are reviewing yearly
progress of students. Appropriate changes, if any, are indicated on the DEPs; other
appropriate components should be addressed.
Evaluation
The plan shall include measurable objectives for the various services aligned with
the core curriculum and a method to evaluate the plan and the services offered. The
evaluation shall focus on improved student performances.
Program Evaluation
The local plan shall address how the program for gifted education is to be assessed. The
plan will ensure that student assessment and services for academically or intellectually
gifted students align and comply with the accountability standards as defined by the
North Carolina ABCs. Students and parents, as appropriate, should be involved in the
assessment of the effectiveness of some program elements, in particular, in class
instructional delivery.
• Are the services on the Differentiated Education Plan (DEP) or Individual
Differentiated Education Plan (IDEP) being offered?
• Are services available during the school day as well as throughout the entire
school year?
• Do all of the district’s schools receive gifted services/programs that are
comparable with respect to quality and duration?
15
• Do program options enable academically or intellectually gifted students to work
together as a group, work with other students, and work independently during the
school day as well as throughout entire school year?
• Have the professional development activities outlined in the plan been completed
and assessed for impact on changed teacher/administrator behaviors and student
achievement and growth?
• Does the program reflect the diversity of the local population?
• Do identification procedures result in equitable representation of diverse cultural
and economic groups in the community?
• How many teachers have received licensure in gifted education?
• How many teachers need professional development in the characteristics of the
gifted, equitable identification, and meeting intellectual, emotional and social
needs of gifted in homogeneous and heterogeneous groupings?
• Does the curriculum differentiation within each program option offer
modifications that develop highest levels of intellectual and emotional taxonomies
and accommodate individual interests, achievement levels and learning styles?
These management objectives are thought provoking; the answer will be yes or no or a
“numerical” answer will be appropriate. In essence, the program evaluation should
address the question, “Are we providing what our plan says we will provide?”
Student Evaluation
LEAs should provide an array of learning environments/opportunities that allow for
ongoing assessments by teachers for students in kindergarten through grade twelve.
Options may include the following:
• Instructional and organizational patterns that enable gifted students to work
together as a group, to work with other students and to work independently on
challenging and rigorous tasks.
• A continuum of learning experiences that leads to the development of advanced-level
products and performances designed with criteria to ensure quality.
• Assessments that require students to create more realistic responses to tasks and
require the application of skills within knowledge utilization of the cognitive
domain.
• In-school and, when possible, out-of-school options relevant to the student’s area
of strength that are available during the entire school year.
• Opportunities to accelerate in areas of strength.
• Procedures for authentic self-assessment by students of both process of learning
and achievement of objectives within each program option.
The End-of-Year and End-of-Course tests may provide evidence of student growth in
some instructional settings. These measures are not sufficient as the only source of
evaluation. Examples of appropriate measures for documenting growth in students are as
follows:
16
Portfolio assessments,
Expert reviews of products,
Off-grade-level assessments,
Authentic assessments,
Performance assessments,
Attitudinal surveys, and
Student criterion-referenced self-assessment of both process of learning and achievement
of objectives.
.
Developing multiple types of assessments beyond standardized testing is challenging. It
is important to acknowledge what research tells us about providing opportunities and
generating support for modifying assessment procedures in organizations.
Professional Development
The local plan shall include professional development clearly matched to the goals
and objectives of the plan, the needs of the staff providing services to academically
or intellectually gifted students, the services offered and the curricular
modifications.
All personnel in the planning, development and delivery of services should have
knowledge to enable them to offer appropriate options and curricula for academically or
intellectually gifted. The LEA’s plan may include yearly assessment of professional
development’s effectiveness in regard to number of teachers receiving training and/or
state licensure, student achievement and growth patterns, and trends over time with those
teachers who receive and those who do not receive training.
LEAs should structure professional development opportunities for all teachers who teach
gifted students. Topics that should be addressed include the following:
• Different characteristics, intellectual and social needs of students with gifted
potential.
• Appropriate use of multiple sources of data in identification of the gifted.
• Differentiation of content/process/instruction, and evaluation in program options
needed to evoke gifted potential.
• Recognition of traits and needs of at-risk students and academic and
social/emotional needs.
• Intervention strategies for gifted students.
• Awareness of cultural and socio-economic differences that can influence different
manifestations of giftedness.
17
Name and Role Description
The plan shall include the name and role description of the person responsible for
implementation of the plan.
Role(s) Responsibilities
The plan shall include the title(s) and role description(s) of the person(s) responsible for
each level of implementation of the plan. Beginning with the system level coordinator,
the plan should outline the responsibilities of each instructional position assigned to each
program service option as well as the support personnel at the school and central office
level. The level of preparation in gifted education for each role description should be
listed in this section.
Community Involvement
The plan shall involve the school community, parents, and representatives of the
local community in the ongoing implementation of the local plan, monitoring of the
local plan, and integration of educational services for academically or intellectually
gifted students into the total school program. This should include a public
information component.
The LEA regularly encourages community and family participation in services designed
for academically or intellectually gifted students and/or those who show potential to enter
gifted programs. To the extent possible, translations should be available in the primary
language of the students’ families in regard to the local plan and policies.
Procedure To Resolve Disagreement
The plan shall include a procedure to resolve disagreements between parents or
guardians and the local school administrative unit when a child is not identified as
an academically or intellectually gifted student or concerning the appropriateness of
services offered to the academically or intellectually gifted.
Disagreements
The local plan shall establish grievance procedures consistent with local policy through
which parents may resolve concerns regarding identification, evaluation, eligibility or
services for gifted students. The first attempt to reach accord could occur with the
teacher and principal at the local school level. If no resolution is possible, the system-level
coordinator of the academically or intellectually gifted program could possibly
review the disagreement. The next level for discussion might be the superintendent or
his/her designee. If agreement cannot be reached administratively, the local board should
18
review the disagreement. At any point in the hierarchy of discussions, the school system
should consider the possibility of mediation with an impartial facilitator.
As an alternative to the hierarchy procedure, a school system could elect to form a
Grievance Committee to hear disagreements. If the committee cannot settle the dispute,
the local school board should review the issue. Mediation could occur at any point
during the process.
115C-150.8 Review of Disagreements
In the event that the procedure developed under G.S. 115C-150.7 (b)(7) fails to
resolve a disagreement, the parent or guardian may file a petition for a contested
case hearing under Article 3 of Chapter 150B of the General Statutes. The scope of
review shall be limited to (i) whether the local school administrative unit improperly
failed to identify the child as an academically or intellectually gifted student, or (ii)
whether the local plan developed under G.S. 115C-150.7 has been implemented
appropriately with regard to the child. Following the hearing, the administrative
law judge shall make a decision that contains findings and conclusions of law.
Notwithstanding the provisions of Chapter 150B of the General Statutes, the
decision of the administrative law judge becomes final, is binding on the parties, and
is not subject to further review under Article 4 of Chapter 150B of the General
Statutes.
If the parents/guardians and the local school system cannot reach agreement, the parent
has the right to petition an Administrative Law Judge whose decision will be final.
Attorney’s fees are not available to parents in the event they prevail in a due process.
Additional Information
The local plan shall include any other information the local board considers
necessary or appropriate to implement the legislation or to improve the educational
performance of academically or intellectually gifted students .
The plan should be a reflection of the community’s goals for its academically or
intellectually gifted students. Although the statutes define the minimum standards for the
program, they do not in any way limit the program options and services a system may
provide its students. If resources are available, school systems may include services in
the cultural arts, academic counseling, leadership, talent development, etc.
Items the plan may include are goals, gifted philosophy of the school system, additional
components (nurturing, highly gifted and/or social and emotional programs), budget,
timelines and acknowledgements of individuals who developed the plan, etc.
19
Pertinent Information
Child Counts
To accurately reflect the AIG population and to have some consistency across the state in
counting AIG students, LEAs should only count identified gifted students who are
receiving services and have a current Differentiated Education Plan or Individual
Differentiated Education Plan in place. Students who have transferred from one system
and are not receiving services when the child count is taken should not be counted. It is
recommended that all identified AIG high school students’ four-year academic plans be
attached to a DEP. These DEPs may be reviewed and dated when counselors are
reviewing yearly progress of students and appropriate changes, if any, are indicated on
the DEPs. Other appropriate components should be addressed. These are the only
students who should be counted on the state child count, and all these students should
meet the criteria for gifted identification in that LEA.
The Child Count of Children with Disabilities Who Are Also Academically or
Intellectually Gifted is due on April 1 of each year. This is the same time the child count
of other exceptionalities is due to the Exceptional Children Division.
Nurturing child counts reflect the innovative programs across the state. There are two
optional child count forms: AIG Nurturing Child Count (general enrichment) and a
Nurturing Differentiated Education Plan (a well-articulated program with screening,
placement and curriculum of students who show potential for gifted programs). These
children should not be counted on the official state child count for academically or
intellectually gifted students. The nurturing child count forms are submitted by June 1 of
each calendar year to the State Consultant, Academically or Intellectually Gifted
Program, Exceptional Children Division, North Carolina Department of Public
Instruction.
Monitoring
The Department of Public Instruction according to the legislation will monitor the local
program plans for academically or intellectually gifted students. The monitoring will
determine whether the local plan contains the components required by law and whether
the system is providing the services outlined in the plan.
Student Transfers
Any student who transfers with evidence of participation in a program for academically
or intellectually gifted students should be reviewed for instructional placement in the
existing gifted program based on the student’s educational needs and local options
available. LEAs should establish a timeline (30/60 or 90 days) for this process to occur.
If a student’s performance is not satisfactory after placement in the gifted program
options, the Needs Determination Team will meet to review student records and
performance and then make an appropriate recommendation for the student’s
20
instructional placement. At this time an Individual Differentiated Education Plan may be
developed based on performance for the student’s most appropriate instructional
placement. This placement may or may not provide for gifted program option(s). The
Needs Determination Team should closely monitor the student’s progress and design
academic, emotional and/or social support for a student, if needed. Furlough periods will
vary depending on the LEA’s philosophy of retaining a child in the gifted program.
However, every effort should be made to return the student to the gifted program option
if appropriate.
State Funds
Effective July 1, 1997, funds allocated for academically or intellectually gifted students
may be used only for academically or intellectually gifted students; to implement the plan
developed under G.S. 115C-150.7; or in accordance with an accepted school
improvement plan, for any purpose so long as that school demonstrates it is providing
appropriate services to academically or intellectually gifted students assigned to that
school in accordance with the local plan. Funding for gifted education is based on 4% of
the average daily membership of local school system.
21
Bibliography
Andrade, H.G. (2000). Using rubrics to promote thinking and learning. Educational
Leadership, 57(5), 13-18.
Assouline, S. (2003). Psychological and educational assessment of gifted children. In
N. Colangelo & G.A. Davis (Eds.) Handbook of gifted education (third edition). Boston:
Allyn & Bacon.
Barton, J.M. & Starnes, W.T. (1989). Identifying distinguishing characteristics of gifted
and talented/learning disable students. Roeper Review, 12, 23-29.
Boatman, T.A., Davis, K.G., & Benbow, C.P. (1995). Best practices in gifted education.
In A. Thomas & J. Grimes (Eds.), The academic acceleration of gifted children (P-102-
132). New York: Teachers College Press.
Borland, J.H. (1986b). I.Q. tests: Throwing out the bath water, saving the baby. Roeper
Review, 8, 163-168.
Burns, J.M., Matthews, N.F. & Mason, A. (1990). Essential steps in screening and
identifying preschool gifted students. Gifted Child Quarterly, 34, 102-107.
Colangelo, N. & Davis, G.A. (Eds.) (2003). Handbook of gifted education (third
edition). Boston: Allyn & Bacon.
Collins, M. & Amabile, T.M. (1999). Motivation and creativity. In R.J. Sternberg
(ED.), Handbook of creativity (pp. 297-312). New York: Cambridge University Press.
Costa, A.L. (ED.) (2001). Developing Minds: A resource book for teaching thinking.
Alexandria, VA: Association for Supervision and Curriculum Development.
Costa, A.L. & Kallick, B. (2000). Discovering and exploring habits of mind.
Alexandria, VA: Association for Supervision and Curriculum Development.
Feldman, D. (1992). Has there been a paradigm shift in gifted education? In N.
Colangelo, S.G. Assouline & D.L. Amronson (Eds.) Talent development: Proceedings
from the 1991 Henry B. and Joycelyn Wallace National Research Symposium on Talent
Development (pp.89-94). Unionville, NY: Trillium.
Fernandez, A.T., Gay, L.R., Lucky, L.F. & Gavilan, M.R. (1998). Teacher perceptions of
gifted Hispanic limited English proficient students. Journal for the Education of the
Gifted, 21, 335-351.
Gardner, H. (2000). The “Giftedness Matrix ”from a multiple intelligence perspective.
In R. Friedman & B. Shore (Eds.) Talents unfolding: Cognitive and developmental
frameworks (pp. 77-88). Washington, D.C.: American Psychological Association.
22
Gilmer, B. (2003). Empowering gifted minds: Educational advocacy that works.
Denver, CO: DeLeon Publishing, Inc.
Guskey, T.R., & Bailey, J. M. (2001). Developing grading and reporting systems for
learning. Thousand Oaks, CA: Corwin Press.
Guskey, T.R. (1996). Reporting on student learning: Lessons from the past-
Prescriptions for the future. In T.R. Guskey (ED.), Communicating student learning:
1996 Yearbook of the Association for Supervision and Curriculum Development (pp.13-
24). Alexandria, VA:ASCD.
Joyce, B. & Showers, B. (2002). Student achievement through staff development.
Alexandria, VA: ASCD.
Karnes, F.A. & Marquardt, R.G. (2000). Gifted children and legal issues: An update.
Scottsdale, AR: Gifted Psychology.
Lambros, A. (2002). Problem-based learning in k-8 classrooms. Thousand Oaks,
California: Corwin Press, Inc.
Lidz, C.S., & Macrine, S.L. (2001). An alternative approach to the identification of
culturally and linguistically diverse learners: The contributions of dynamic assessment.
School Psychology International, 22 (1), 74-96.
Mahoney, A.S. (1998). In search of gifted identity: From abstract concept to workable
counseling constructs. Roeper Review, 20(3), 222-226.
Mansilla, V.B. & Gardner, H. (1998). What are the qualities of understanding? In M.S.
Wiske (Ed.), Teaching for understanding: Linking research with practice. San
Francisco: Jossey-Bass.
Marquardt, R.G. & Karnes, F.A. (1994). Gifted education and discrimination: The role
of the Office for Civil rights. Journal for the Education of the Gifted, 18 (1), 87-94.
Marzano, R. J. (2000). Transforming classroom grading. Alexandria,VA: ASCD.
Valdes, Guadalupe. (1996). Bridging the distances between culturally diverse families
and schools-an ethnographic portrait. New York: Teachers College Press.
Mills, C. & Tissot, S. (1995). Identifying academic potential in students from
underrepresented populations: Is using the Ravens Progressive Matrices a good idea?
Gifted Child Quarterly, 39, 209-217.
23
National Academy Council. (1999). How people learn: brain, mind, experience and
school. Washington, DC: National Academy Press.
National Academy Council. (2002). Learning and understanding: Improving advanced
study of mathematics and science in U.S. high schools. Washington, DC: National
Academy Press.
Plucker, J.A., & Taylor, J.W.V. (1998). Too much too soon? Non-radical advanced
grade placement and self-concept of gifted students. Journal of Educational Psychology,
83, 528-538.
Renzulli, J.S., Leppien, J.H., & Hayes, T.S. (2002). The multiple menu model: A
practical guide for developing differentiated curriculum. Mansfield Center, CT:
Creative Learning Press.
Richert, E.S. (1990). Patterns of underachievement among gifted adolescents. In J.
Genshaft & M. Bireley (Eds.), The gifted adolescent: Personal and educational issues.
New York: Teachers College Press.
Richert, E.S. (1993). Richert teacher, parent and self-nomination forms. Ocean Grove,
NJ: Global Institute for Maximizing Potential.
Richert, E.S. (1994). Training handbook for maximizing student potential. Ocean
Grove, NJ: Global Institute for Maximizing Potential.
Richert, E.S. (2000). Maximizing gifted potential for the 21st century. Paper presented at
Arab Council for the Gifted and Talented Conference. Amman, Jordan.
Richert, E.S. (2003) Excellence with justice in identification and programming. N.
Colangelo & G.A. Davis (Eds.) Handbook of gifted education (third edition). Boston:
Allyn & Bacon.
Richert, E.S., Alvino, J.J., & McDonnel, R.C. (1982). National report on identification:
Assessment and recommendations for comprehensive identification of gifted and talented
youth (for the U.S. Department of Education). Sewell, NJ: Educational Information and
Resource Center.
Rogers, K.B. (2002). Re-forming gifted education: Matching the program to the child.
Scottsdale, AZ: Great Potential Press, Inc.
Silverman, L.K. (1986). The IQ controversy: Conceptions and misconceptions. Roeper
Review, 8(3), 136-140.
Silverman, L.K. (1998). Using test results to support clinical judgment. Gifted
Education Quarterly. Winter, 12(1), 2-5.
24
Silverman, L.K. (2002). Upside-down brilliance: The visual-spatial learner. Denver,
CO: DeLeon Publishing, Inc.
Smutny, J.F. (2003). Designing and developing programs for gifted students. Thousand
Oaks, California: Corwin Press, Inc.
Tallent-Runnels, M.K. & Chandler-Lotven, A.C. (1995). Academic competitions for
gifted students: A resource books for teachers and parents. Thousand Oaks,
California: Corwin Press.
Tomlinson, C.A., Kaplan, S.N., Renzulli, J.S. The parallel curriculum: A design to
develop high potential and challenge high ability learners. Thousand Oaks, California:
Corwin Press, Inc.
Valdes, Guadalupe. (1996). Bridging the distances between culturally diverse families
and schools-an ethnographic portrait. New York: Teachers College Press.
VanTassel-Baska, J. (1997). Excellence in educating gifted and talented learners (3rd
ed.). Denver: Love.
VanTassel-Baska, J., Avery, L.D., Little, C., & Hughes, C. (2000) An evaluation of the
implementation of curriculum innovation: The impact of the William and Mary units on
schools. Journal for the Education of the Gifted, 23, 244-270.
Van-Tassel-Baska, J. & Little, C.A. (2003). Content-based curriculum for high ability
learners. Waco, Texas: Prufrock Press, Inc.
VanTassel-Baska J., Leonhard, P., glenn, C. B., Poland, D. Brown, E. and Johnson, D.
(1999). Curriculum review as a catalyst for gifted education reform at the secondary
level. Journal of Secondary Gifted Education, 10, 173-183.
Wiggins, G. & McTighe, J. Understanding by design. Alexandria, VA: ASCD.
25
Appendix A
26
AIG Coordinators’ Institute
Implementing Article 9B: Research-Based Service delivery and
Effective Program Practices
October 3, 2003
Radisson Research Triangle Park
Citations From Dr. Linda Silverman’s Presentation
Identification and Assessment of Gifted Students
Citation for support of individual group IQ tests is the following book.
• Gilman, B. (2003). Empowering gifted minds: Educational advocacy that
works. Denver: DeLeon, Publishing, Inc..
Chapter 3, "Testing Considerations" P-71.
"Brief measures appear particularly inadequate for documenting the giftedness of twice
exceptional (both gifted and learning disabled) children and they frequently fail to
document high levels of giftedness. For example, we have seen profoundly gifted
children (175+) score in the 130s on the CogAT. Research as early as 1959, by Pegnato
and Birch, showed that highly capable children often expand the meaning of the
multiple-choice questions on brief tests beyond what the test writer had in
mind, and they may answer incorrectly."
• Assouline, S. (2003). Psychological and educational assessment of gifted
children. In N. Colangelo & G.A. Davis (Eds.) Handbook of gifted education
(third edition). Boston: Allyn & Bacon.
"Group intelligence tests are often used as a way of initially screening for
students of high academic ability, but beyond that their uses with
individual students are limited.
Despite improvements in the group-administered tests of general ability, an
individually administered test remains the best instrument for identifying
gifted children on the criterion of general ability." P-126.
• "Change 'and' to 'or'"
You may quote me from the presentation. If you look at the funnel screening in the
handout, "or" is implied between each of the methods of entering the talent pool. Every
"and" in an identification plan limits the identification of gifted students to high achievers
and leaves out the very children you are concerned about missing.
27
…It is certainly implied in the National Report that Susanne Richert put together and in
her chapter, “Excellence with justice in identification and programming.” From the
N. Colangelo & G.A. Davis (Eds.) Handbook of gifted education (third edition).
Boston: Allyn & Bacon.
You may also use this email (October 15 to Valorie Hargett) as a legitimate citation,
according to APA Guidelines.
Earlier IQ Scores
• Silverman, L.K. (1986). The IQ controversy: Conceptions and
misconceptions. Roeper Review, Vol. 8, Number 3, 136-140.
"Since developmental differences are more apparent during the preschool and
primary years than they are later, IQ tests are more sensitive to the differential rate of
development of younger children. For example, there are greater developmental
differences between four year olds and six year olds than there are between ten year olds
and twelve year olds; therefore, the tests are more effective in distinguishing differences
among four year olds. By junior high school, generalized abilities are channeled into
specific domains, and aptitude tests are considered better indicators of ability than
intelligence tests."
• Silverman, L.K. (1998). Using test results to support clinical judgment.
Gifted Education quarterly. Winter, 12(1), 2-5.
"It is impossible for a child to achieve beyond his or her capabilities. (This is why the
term 'overachiever' is an oxymoron.) Therefore, we recommend that the highest
indicator of a child's abilities at any age should be seen as the best estimate of the child's
giftedness."
Dr. Silverman wrote . . .
I don't believe anyone else is making that recommendation. They are all brainwashed
into believing that newer is better, without realizing the impact of "newer" on the IQ
scores of gifted children.
In 1989, I presented a paper at NAGC, entitled, "Lost: One IQ Point per Year for the
Gifted," in which I traced how the gifted have been penalized at THREE TIMES the level
of the Flynn Effect, since 1972. Now that we have 2002 norms, the picture has not
changed, but the cumulative effect is worse. The Flynn Effect is approximately 1/3 of an
IQ point per year. It indicates that the general population is increasing in intelligence at
about that rate. The average child achieved an IQ score of 100 in 1972. In 2002, the
same raw score would yield an IQ score of 90 (1/3 of an IQ point over a 30-year period is
10 IQ points). However, the Flynn Effect DOES NOT APPLY IN THE GIFTED OR
RETARDED RANGES, a piece of information overlooked by most test constructors and
psychologists. Yet, the gifted population is being
28
penalized in renorming at three times the rate of the general population. A raw score in
1972 that would have yielded an IQ score in the highly gifted range--145, would yield a
score of about 115 in 2003 on most of the newly normed tests, insufficient to gain
entrance into a gifted program! Is this child really 115 or actually a highly gifted child?
If the score had been 135, instead of 145, then the reduction would be in keeping with the
Flynn Effect. As no one else seems to be looking at these losses besides me, no
one else is writing about them, and I can provide no other citations.
This leads to the following recommendations:
1) Older test scores are likely to be better estimates of gifted ability than newer test
scores, because new norms are prejudicial against the gifted.
2) Scores of younger children are likely to be better estimates of gifted ability than newer
test scores for gifted girls, children of minority groups, and children of low socio-economic
circumstances, because these children are socialized to hide abilities as they get
older and the test items are more loaded on crystallized (environmentally loaded) abilities
than on fluid abilities in the upper age ranges.
3) Given the fact that the means for the gifted groups in the validation studies on the
WISC-IV and Binet 5 were 123.5 and 124, respectively, it is important to lower cut-off
scores for gifted programs to 120 IQ. ALL of the children in the validation studies tested
above 130 on other IQ tests, and some tested in the 200 IQ range. Even if the mean of
the gifted groups was 130 (a VERY conservative estimate), that represents a loss of 6.5
IQ points from 1991 to 2002--twice the Flynn Effect.
• Borland, J. H., & Wright, L. (1994). Identifying young, potentially gifted,
economically disadvantaged students. Gifted Child Quarterly, 38, 164-171.
"Working Principles” (page 164)
1. The potential for academic giftedness is present in roughly equal proportions in all
groups in our society.
Dickinson, R. M. (1970). Caring for the gifted. North Quincy, MA: Christopher
2. Gifted education is a form of special education.
(p. 165) 3. Identifying economically disadvantaged, potentially gifted students differs
from identifying other gifted students with respect to the goal of identification.
Placing potentially gifted economically disadvantaged students in special classes for
gifted students can be problematic. ... Such placements may become sink-or-swim
situations. This illustrates the need for what we call transitional services, interventions
29
designed to help potentially gifted students develop their latent abilities. ... We thus seek
evidence of undeveloped potential, not necessarily realized ability.
4. Knowledge needed to identify disadvantaged gifted students is to be found in school
classrooms, not the research literature.
5. The human being is the identification instrument of choice.
...Attempts to overcome this discrepancy [the gap between the mean IQs of African-
American and Caucasian children] by manipulating test scores (for example, by using a
matrix) or creating "culture-fair" or "culture-free" tests have not been promising. The
answer, we believe, is to use "the human instrument" (Lincoln & Guba, 1985, p. 39),
trading the objectivity of standardized tests for the sensitivity and adaptability of human
observation and judgment.
(This supports Annemarie Roeper's Qualitative Assessment Method that I discussed in
the presentation. I added the bracketed section from the previous sentence. The
parenthetical phrases are in the original.)
6. The concept of "best performance" is valid in identifying giftedness in young
economically disadvantaged children.
As Roedell, Jackson and Robinson argue,
very young children...rarely...can be relied upon to demonstrate the best
performance of which they are capable during all phases of a test session.
...[T]he most meaningful aspect of a young child's test performance is not
the child's average level of performance across a wide range of tasks, but
the most advanced performance demonstrated (1980, p. 38)
This is even more true for young children whose intellectual and experiential stimulation
has been minimal and inconsistent. Thus, for identification purposes, we look for any
sign of advanced performance that might represent untapped potential. This precludes
the use of a matrix or any process involving the averaging of data and requires using
many vaired indicators to increase the probability of uncovering an area of advanced
performance or potential."
References:
• Lincoln, Y.S., & Guba, E.G. (1985). Naturalistic inquiry. Beverly Hills,
CA: Sage.
• Roedell, W.C., Jackson, N.E., & Robinson, H.B. (1980). Gifted young
children. Perspectives on Gifted and Talented Education. New York:
Teachers College Press.
30
Working Principle Number 6 supports my assertion about the use of early IQ scores:
"Therefore, we recommend that the highest indicator of a child's abilities at any age
should be seen as the best estimate of the child's giftedness. It also supports the use of
multiple criteria. And it explains why matrices are not helpful in looking for giftedness
in disadvantaged populations.”
31
Appendix B
32
Appendix C