Resource Center

• by Albert Cortez, Ph.D. • IDRA Newsletter • September 1998 • 

Over the last decade we have witnessed the emergence of a national movement to improve educational outcomes for all students. The movement was spurred by increasing public concern involving the quality of public education being provided to large numbers of U.S. students. This concern included government-sponsored reports such as A Nation at Risk, which was one of the first documents to trumpet the need for significant improvement in our public schools.

Following on the heels of these early efforts were national- and state-level reform initiatives that for the most part included the creation of “standards” of performance in certain core subject areas (reading, math, writing, science and, in some cases, social studies). These initiatives also included the development of corresponding testing programs to measure the extent to which those standards were being met.

The standards movement has been undergirded by some assumptions:

• A fundamental belief that the general public understands the performance standards and that the public concurs with the people who developed the standards on the levels of student or school performance that are eventually incorporated into such efforts.
• An assumption that data on a student’s and/or school’s progress toward reaching the prescribed standards is or can be made available to policy-makers, parents and the general public.
• An underlying belief that the data to be collected will actually include all (or most) pupils enrolled in schools.
• A faith that the availability of data on school and student performance will ultimately lead educators to develop and implement responses that result in improved student performance.

While standards offer many opportunities to achieve clarity of purpose and can contribute to improved school performance, the extent to which such improvement happens is significantly affected by the extent to which those standards are available to the various stakeholders in the system and the extent to which those stakeholders are vested in them.

Who Sets the Standards?

Many people consider it a good idea to develop common educational goals at the national level. A major problem with this concept, however, is that, unlike many other nations, US education evolved as primarily a state-level responsibility. As such, states developed distinct ideas about education. They have long-resisted national intrusion in the design or implementation of state and local educational policy.

As a result, the federal government and national-level standards groups have focused efforts on either providing some general guidance on standards or attempting to develop standards generic enough to support or provide for easy alignment with state-level standards. Recent congressional decisions to forego the development of national tests in reading and mathematics reflect the state and local resistance to the “nationalizing” of education activities.

In turn, language in federal education legislation (particularly the Elementary and Secondary Education Act [ESEA], which is the major vehicle for providing federal support for local education programs) emphasizes the importance of using federal resources to support ongoing or emerging state and local education reform efforts. At the state level, many states, including Kentucky, Connecticut and Texas, have developed very specific indicators outlining what it is they expect all students to know at every grade level. After developing their standards, the standards developers usually embark on campaigns to ensure that others (particularly school district and campus-level personnel) understand and buy into the standards.

Who Knows the Standards?

In reality most standards-based reform efforts are the product of a small handful of policy-makers. Very few of these legislators are aware of – let alone “understand” – what the student or school performance standards are or what constitutes the basis for them. Most of the educational standards that have been created for schools and students over the last decade have been developed by small groups of experts in specific areas who spend considerable time and energy on determining what it is all students should know or be able to do. These standards-setting efforts range from outlining a few basic ideas (as exemplified in the recent development of national education goals and subsequent educational priorities) to state-level activities that have created very specific expectations for students in every major subject area.

For many school-level personnel, the most direct encounter with national, state or local standards happens when they are briefed on student outcome expectations – that is, what their students are expected to know or be able to do by the year’s end. In Texas these expectations are outlined in the Texas Essential Knowledge and Skills (TEKS), which are based on standards that were first developed in the mid-1980s in the aftermath of a legislatively mandated comprehensive school reform. The TEKS are currently in the process of being updated. Many states around the country have comparable curricular frameworks that in essence dictate what it is their students will be expected to learn over the course of a year.

The clarity of purpose included in such curricular standards is useful to policy-makers, educators, parents and the general public. Specifying clear expectations has the potential of having everyone develop a common base of understanding about schooling and what is expected to result from that experience for individual pupils.

Unfortunately, experience has shown us that general understanding of state and local educational standards varies considerably within the various audiences noted. At the state level, most policy-makers do not delve into details of content mastery but tend to focus on aggregated data, such as the number or percent of pupils passing or demonstrating mastery on state testing measures.

On the other end of the spectrum, parents are concerned not with the specifics of their children’s learning but with whether or not they demonstrated enough knowledge or mastery of a concept to pass the subject. Generally it is the teachers and building-level administrators who focus on the subcomponents of detailed knowledge or skills that collectively result in doing well in reading, mathematics or science tests.

In recent work on standards-based reform, IDRA noted that even the school-level personnel’s knowledge of state and national standards varies extensively. While a small subgroup may be fairly well versed, other staff may have only a rudimentary grasp of critical standards targets. Moreover, community members are even less versed and need specially designed and targeted materials that would inform and educate them as a constituency.

Rather than worrying about what it is that students should know, conversations among many policy-makers ultimately tend to focus a lot of time and energy on determining who has passed or not passed the new standard. Texas was a notable exception, spending years convening groups of educators to help define what it was that students should be expected to learn and in what areas they should show competence. Even in Texas, however, only a handful of specialists ultimately grew to know and understand the rationale behind what it is students were expected to learn and demonstrate “mastery” in.

Who is Expected to Meet the Standards?

Fueling the creation of standards is a broad-based commitment to finding out how students are doing after spending nine or 10 months attending public schools. This expanded concern with knowing how schools are performing may have been caused by an array of large-scale research efforts that informed the public and policy leaders that many schools were not doing as well as most people had assumed. Or were they?

In the latter half of the century, many educators had become increasingly aware that while they had not changed much, the composition of the student populations they were serving were gradually changing. With the movement of the population from the small towns and rural communities to the larger cities and the shift of our economy from an agrarian to an industrial base, schools were faced with significant increases in the numbers of pupils who stayed in school beyond the elementary level.

Schools were also challenged by growing numbers of recent immigrants drawn in by promises of greater economic opportunities. And, unfortunately, the schools were often less than successful at effectively serving these more diverse populations. So why were we not concerned with standards, performance or accountability back then?

Well, the economies of those eras not only tolerated lower standards, in some cases they even required unskilled or nonliterate workforces. Thus, the schools’ abilities to sort out some and educate the few were well suited to their task. Problems for schools began to mount only as the needs of a post-industrial economy, emerging in the 1940s and 1950s and in full bloom by the 1960s and 1970s, required a much more effective public education system than what had been tolerated to date. These new expectations often took the form of standards, so as to clarify to all major constituencies – educators, policy-makers and the general public – what was expected to be achieved by schools and by students in exchange for the investment of public monies in those efforts.

The development of standards had broad-based appeal. Standards enabled policy-makers to “check up” on their charges (schools and students) and make sure they were doing what was expected. They also appealed to parents, for much the same reason. Standards appealed to business leaders who felt the need for the creation of some indicators of performance, a practice not unknown in many private sector operations.

The creation of standards sets in motion a series of actions and, eventually, reactions on the part of many school constituencies. While the beginning of the conversation tends to focus on individual student’s performances on specific measures aligned to state or local standards, it only takes a small step to begin compiling aggregates of pupils at the classroom, school and district level based on those measures of performance. Thus, what begins as an assessment of student performance often ends up including an assessment of performance by schools and school districts, as judgments are made on the relative effectiveness of specific sites.

It is entirely appropriate to compile and analyze the performance of classes and schools in order to gain insights into which schools and teachers are succeeding in meeting the stated standards, as well as those places that may be falling short of expected performance levels. Many states have developed rating systems and other accountability mechanisms that are designed to hold schools accountable for producing desired results.

The extent to which the progress of whole schools and important subgroups of students within schools can be measured in any standards-grounded effort can have a great impact on the extent to which such efforts improve performance for all students, rather than limiting their benefits to White, middle-class pupils in those systems. One result of such standards-driven school accountability can be a heightened awareness and concern for the collective performance of all pupils attending specific schools and districts. In some instances the development of uniform standards has fostered the development of high expectations for all students. This in turn has contributed to heightened efforts to better serve school populations that had been historically underserved, or ill-served, by the public school system.

In some instances, scores on standards-driven systems are often reported in the aggregate; that is, they are reported in a manner such that the performance of subgroups of students is not presented or is easily extracted from the numbers that are reported. Examples of composite scores include such practices as the reporting of the overall mean for a school or district, or the percentage of all students meeting selected standards (minimum, mastery or exemplary performance thresholds). While offering a good summary statistic of overall school performance, composite measures of school performance can also serve to mask underachievement among identifiable subgroups, including migrant, minority, low-income and limited-English-proficient students.

Many states address this issue by requiring the reporting of disaggregated data – data reported in a way that allows one to analyze the relative performance of an array of subgroups that may make up a total school’s enrollment. In Texas, the recognition that subgroup performance can lead to classifying a school or district as “low performing,” or impact a school’s rating as “recognized” or “high performing,” has contributed to increased efforts to raise the levels of academic achievement for all students – including those pupils who were not well served previously.

Thus even today, standards can serve as targets that are used to guide local, state and national efforts. They can serve as benchmarks, measures of where we are and where we need to go. The trouble is that, unless there is a broad-based understanding and people “buy into” what they are, how they are measured and what the consequences of not meeting them are, standards alone can really do little to actually improve school and student outcomes. In fact, their creation can and often does nontraditional pupils as much harm as good. Why does the creation of standards sometimes not work and at times result in unintended consequences?

Potential Negative Consequences of Standards

While the use of standards to frame performance objectives can result in school and student improvement, standards sometimes lead to negative consequences for pupils. Recently Texas’ governor announced a proposal that called for the retention of all third-grade pupils who did not pass the state’s assessment of basic skills. This sweeping proposal would have ignored the fact that many students had passed all their classes but happened to do poorly on the exam. It also ignored research indicating the ineffectiveness of in-grade retention. This proposal was widely criticized, and it triggered long-standing concerns about high-stakes assessment and the use of any single measure to determine a student’s standing in school.

Historically, students were the only entities held accountable for achieving certain outcomes. It is only recently that the public has demanded and acquired an accountability process that holds schools and districts accountable for the outcomes that they produce in students.

A large number of states around the country continue to use dysfunctional policies, wherein only the pupils pay any consequence for failing to meet certain performance expectations, while the personnel in schools and school districts are excluded from such consequences.

IDRA proposes that the achievement of the desired outcomes framed in state and national standards are a joint venture involving many parties, but particularly school staff, students and their families. The inability to perform at desired levels thus should also be considered a shared responsibility, including the responsibility to develop strategies to address any lack of achievement documented in the accountability process.

In addition to requiring the participation of numerous stakeholders, the success of standards-based efforts requires attention to the factors and conditions leading to the outcomes that are ultimately measured. Describing the components of a model for a balanced, inclusive accountability system, the Center for Educational Outcomes proposes a model composed of three interrelated components that include system accountability, individual student accountability, and input and/or process accountability. In addition to focusing on the traditional systemic and student accountability dimensions, the model acknowledges the need to assess the inputs that produce standards-based outcomes.

In such a comprehensive approach, stakeholders can examine information on factors such as teacher credentials and experience, financial resources available, and support mechanisms provided for both students and staff. Access to such critical input data provides the detail needed to address schools that require improvement, and to support and sustain those critical elements in schools that may be producing desired results.

Merely documenting that schools or students are not succeeding without isolating the related causal factors does an injustice to schools and students. Failure to acknowledge, target and support improvement in local school sites can lead to the evolution of dysfunctional systemic responses.

Exemptions and Exclusion of Students and Schools from Assessment of Performance

While one obvious response to meeting state and local standards is an increased amount of attention paid to having all pupils achieve to high standards, an unfortunate alternative too often pursued by some people involves finding ways to exclude harder-to-educate pupils from participating in the local or state accountability process.

A not-uncommon conversation that follows the development of state standards and related processes for assessing achievement involves ways to exclude pupils (or even whole schools) from the standards measurement system. While the exclusion of a handful of students for whom some uniform standard may be inappropriate is acceptable (and even in those cases, the process should involve the participation of an informed group of stakeholders, as in the ARD process used in special education), history has proven that the number of exclusions is commensurate with the extent to which key decision makers believe that those standards should be expected of and achieved by all students and schools. Unfortunately, in too many cases, determination of who achieves state and national standards can become a shell game where the claims are overstated by the exclusion of large numbers from the measurement process.

In Texas in the aftermath of the development of the TEKS, that spell out performance expectations for all Texas pupils in grades kindergarten through 12, the state tried to determine which types of students should be excluded from the state assessments that were aligned to those standards. After significant numbers of pupils of different types (migrant, limited-English-proficient and others) were found to have been excluded from the state testing process, state lawmakers adopted policy reforms that provided for greater inclusion of such special groups in the assessment system. These steps included providing flexibility for when pupils were assessed (in the case of migrant pupils), developing measures of content mastery in Spanish for students served in the state’s bilingual education programs, and triggering state education agency reviews in those schools or districts with excessive numbers of exemptions. A more recent incentive includes the consideration of the number or percent of student exemptions from the state testing process in determining whether selected schools would be accorded special recognition and/or designation by the state agency.

The recent explosion in the number of students referred to alternative education programs – disciplinary programs for students who have violated school codes of conduct – has also created a new venue for excluding selected pupils from state and local performance measures. In Texas, the alternative education programs have even developed their own “alternative accountability” systems that allow them to set different (and lower standards) for pupils referred to such programs. The referrals to disciplinary alternative education programs in Texas are approaching 100,000 pupils, and involve disproportionate numbers of students who are poor or minority.

The extent to which the assessment and accountability systems allow for the significant exclusion of pupils or schools from a uniform performance assessment system determines the extent to which those standards are perceived as uniform and applicable to all pupils and schools, rather than as a sham accountability system sold to the public. Furthermore, the extent to which it is real or merely looks good on the surface depends on the extent to which key stakeholders – particularly parents – are willing to hold all schools accountable. They must also be willing to look beyond the standards-related outcomes to the inputs made available to schools and communities. They must be willing to hold themselves and the state accountable for the inputs that contribute to producing the outcomes desired. Our students deserve no less than attention to both.

Timeline of the Modern Standards-Based Reform Movement

– compiled by Anna Alicia Romero, Intercultural Development Research Association

Challenges of Standards-Based School Reform

Resources

Marzano, R.J. and J.S. Kendall. The Fall and Rise of Standards-Based Education (Alexandria, Virginia: National Association of State Boards of Education, 1996).

Wheelock, A. Standards-Based Reform: What Does it Mean for The Middle Grades? (New York, N.Y.: Edna McConnell Clark Foundation, September 1995).

US Department of Education. Improving America’s Schools: A Newsletter on Issues in School Reform (Washington, D.C.: US Department of Education, Spring 1996).

American Federation of Teachers. Making Standards Matter 1996 (Washington, DC: American Federation of Teachers, August 1996).

Albert Cortez, Ph.D., is director of the IDRA Institute for Policy and Leadership. Comments and questions may be sent to them via e-mail at feedback@idra.org.

[©1998, IDRA. This article originally appeared in the September 1998 IDRA Newsletter by the Intercultural Development Research Association. Permission to reproduce this article is granted provided the article is reprinted in its entirety and proper credit is given to IDRA and the author.]