• by Bradley Scott, MA • IDRA Newsletter • February 2001
The Office for Civil Rights (OCR) of the U.S. Department of Education receives many requests for advice and technical assistance regarding testing. In response, it has created a document entitled, The Use of Tests When Making High-Stakes Decisions for Students: A Resource Guide for Educators and Policy-Makers. The need for a publication such as this is clear:
Educational stakeholders at all levels have approached the OCR requesting advice and technical assistance in a variety of test-use contexts, particularly as states and districts use tests as part of their standards-based reforms. Also, increasingly, OCR is addressing testing issues in a broader and more extensive array of complaints of discrimination that have been filed with OCR. These corresponding developments confirm the need to provide a useful resource that captures legal and test measurement principles and resources to assist educators and policy-makers (US Department of Education, 2000).
The publication is available for distribution in hard copy form and can be accessed in its entirety through the OCR web site at http://www.ed.gov/about/offices/list/ocr/index.html?src=mr. It is a guide that is intended for use at both the elementary and secondary school levels.
The guide addresses standardized measures generally recognized by the Standards for Educational and Psychological Testing (Joint Standards). The guide does not address teacher-made tests, but the basic principles to which it adheres are still applicable even to classroom teachers in individual classrooms. Among those principles cited in the guide are:
- A test should not be used as the sole criterion for making high-stakes decisions unless it is validated for such use.
- A high-stakes decision should not be made on the basis of a single test score. Other relevant information should be taken into account if it will enhance the overall validity of the decision.
- As the stakes for testing increase for individual students, the importance of considering additional evidence to document the validity of score interpretations and fairness in testing increases accordingly.
- Educators should carefully monitor over time all of the information and other inputs that go into the decision-making process and the outcomes to ensure that there is no discrimination arising from the use of any criteria. Should any criterion be found to be discriminatory, it should be eliminated.
In addition to these general principles are a set of test measurement and use principles and legal principles that create a framework for all of the information that is contained in the two chapters of the guide. These principles are grouped under “Test Use Principles,” and “Legal Principles.” Under both major groups of principles are several subgroups. Detailed explanations of the two groups follow, with statements directly from the guide.
Test Use Principles
Educational Objectives and Content
“Before any state, school district, or educational entity administers a test, the objective for using it should be clear.”
- Placement Decisions
“Decisions concerning the appropriate educational program for a student with a disability, placement in gifted and talented programs, and access to language services are examples of placement decisions. The joint standards state there should be adequate evidence documenting the relationship among test scores, appropriate instructional programs, and beneficial student outcomes.”
- Promotion Decisions
“When a test given for promotion purposes is being used to certify mastery, it is important that there be evidence that the test adequately covers only the content and skills that students have actually had an opportunity to learn.”
- Graduation Decisions
“When large-scale standardized tests are used in making graduation decisions, there should evidence that the tests adequately cover only the content and skills that students have had an opportunity to learn.”
Can the following question be answered in a way that benefits the students: Is there sufficient confidence in the test results at issue to allow for informed decisions to be made that will have specified consequences for the students taking the test? Following are criteria that should help educators and policy-makers to answer this question.
- Measurement Validity
“Is a test valid for a particular purpose, and does it accurately measure the test taker’s knowledge in the content area being tested?”
- Attribution of Cause
“Does the student’s performance on the test reflect knowledge and skills based on appropriate instruction, or is it attributable to poor instruction or to such factors as language barriers unrelated to the skills being tested?”
- Effectiveness of Treatment
“Do test scores lead to placement and other consequences that are educationally beneficial?”
The guide states that constitutional, statutory, and regulatory principles “form the federal legal nondiscriminatory framework applicable to the use of tests for high-stakes purposes.” Clearly, Title VI, Title IX, Section 504, Title II (ADA), and the 14th Amendment to the US Constitution prohibit intentional discrimination on the basis of race, national origin, color, sex and disability. The guide also points to specific sections of Section 504 and the Individuals with Disabilities Education Act (IDEA) that contain specific provisions relative to the use of high-stakes tests for individual with disabilities.
These legal bases allow three things to occur. They provide (1) a framework for the analysis of discrimination; (2) principles related to inclusion and accommodations; and (3) federal constitutional questions related to testing of elementary and secondary school students for high-stakes purposes. Details of the these topics follow.
Framework for the Analysis of Discrimination
- Different Treatment
This refers to practices applied consistently to similarly situated individuals or groups regardless of their race, national origin, sex or disability.
- Disparate Impact
This is when the criteria or methods of administration have the effect of subjecting individuals or groups to discrimination.
Principles Related to Inclusion and Accommodations
- Limited-English-proficient students
“States or school districts using tests for high-stakes purposes must ensure that, as with all students, the tests effectively measure limited-English-proficient students’ knowledge and skills in the content area being measured… It may be necessary in some situations to provide accommodations [regarding language differences] so that the tests provide accurate and valid information about the knowledge and skills intended to be measured.”
- Students with disabilities
Federal law requires that students with disabilities be included in statewide and districtwide assessment programs with the necessary accommodations: “There must be an individualized determination of whether a student with a disability will participate in a particular test and the appropriate accommodations, if any, that a student with a disability will need.”
Federal Constitutional Questions Related to Testing of Elementary and Secondary School Students for High-Stakes Purposes
The due process and equal protection requirements of the Fifth and 14th Amendments to the US Constitution would apply to ensure appropriate decisions were made by public schools and states. The courts have examined three questions in addressing due process claims related to the use of tests: (1) Is the purpose of the testing program legitimate and reasonable? (2) Have students received adequate notice of the test and its consequences? and (3) Have students actually been taught the knowledge and skills measured by the test?
All of these principles serve as a basis for the content of the remainder of the guide. The first chapter focuses specifically on an in-depth explanation of the measurement principles and their application. The second chapter explains the legal principles and the law with practical examples to facilitate understanding of how the law is applied.
Educators and policy-makers can gain tremendous insight into the measurement and legal principles that may guide the Office for Civil Rights in its investigation and compliance reviews under Title VI, Title IX, and Section 504 from The Use of Tests as Part of High-Stakes Decision-Making for Students: A Resource Guide for Educators and Policy-Makers. The document should be used as one of the tools for local internal examination of the use of tests for high-stakes purposes. Finally, the guide contains appendices that provide excellent resources in testing and high-stakes decision-making.
The Intercultural Development Research Association’s (IDRA) South Central Collaborative for Equity is the equity assistance center that serves Arkansas, Louisiana, New Mexico, Oklahoma, and Texas. It is available to assist school districts in such a self-examination. Contact the equity assistance center at IDRA, 210-444-1710.
US Department of Education Office for Civil Rights. The Use of Tests as Part of High-Stakes Decision-Making for Students: A Resource Guide for Educators and Policy-Makers (Washington, D.C.: US Department of Education Office for Civil Rights, December 2000).
Bradley Scott, MA, is a senior education associate in the IDRA Division of Professional Development and director of the IDRA South Central Collaborative for Equity. Comments and questions may be directed to him via e-mail at firstname.lastname@example.org.
[©2001, IDRA. This article originally appeared in the February 2001 IDRA Newsletter by the Intercultural Development Research Association. Permission to reproduce this article is granted provided the article is reprinted in its entirety and proper credit is given to IDRA and the author.]