• by Adela Solís, Ph.D. • IDRA Newsletter • January 2000 • Dr. Adela Solis

The evaluation of educational programs is a critical, non-negotiable activity in schools that implement programs with federal, state and other external sources of funding. Evaluation is the way to discover strengths and weaknesses in program operations. It is the way by which practices – such as teaching, lesson planning and program managing – are assessed and re-directed to cause the program to have a better impact on student learning. Evaluation is also a way to determine whether what a school is doing through its various funded programs has worth or merit.

To the government and agencies that supply financial resources, this determination informs their decisions for possible continued funding of programs. School districts that continually operate programs through these types of external funds are keenly interested in providing evidence of worth and merit. Yet many struggle to understand and manage the evaluation process.

Numerous schools in Texas receive financial assistance from the federal government through the various provisions of the Improving America’s Schools Act (IASA) to serve minority and disadvantaged students. Title I program funds are the most common source of assistance. An increasing number of school districts have also been receiving funds through competitive grants under Title VII of the IASA.

The STAR Center has provided technical assistance to such school districts in Texas to aid their efforts to understand and manage evaluation requirements. The STAR Center is the comprehensive regional assistance center funded by the U.S. Department of Education to serve Texas. It supports Texas educators in creating systemic change that leads to achievement for all students, particularly students considered to be in at-risk situations. The center provides support and technical assistance services to regional service centers, to the Texas Education Agency, and to local school districts that are implementing state and local education reform efforts funded under the IASA. The STAR Center is a collaboration of the Intercultural Development Research Association (IDRA), the Charles A. Dana Center at the University of Texas at Austin, and RMC Research Corporation.

A few months ago, the STAR Center at IDRA sponsored a Title VII evaluation institute for school districts operating Title VII bilingual education grants. The purpose of the institute was to assist Title VII project staff and evaluators in developing evaluations that are meaningful, useful and compliant with federal requirements. Participants had an opportunity to review principles of program evaluation, discuss strategies for conducting the evaluation and writing evaluation reports, and hear the federal government?s Office of Bilingual Education and Minority Languages Affairs’ (OBEMLA) perspective on Title VII evaluation.

Participants included administrators in districts and campuses who were relatively new to Title VII projects and others who were more experienced Title VII project managers still in need of information about grant requirements under the IASA. Participant feedback concerning this two-day technical assistance event was overwhelmingly positive, and many requested that the institute be offered annually, particularly since not all grant recipients were able to attend.

Key Questions to Explore

The following topics were discussed by institute presenters and participants.

  • What qualifies as a significant and worthwhile change in educational evaluation,
  • Reasons for Title VII requiring evaluations,
  • Important strengths and weaknesses in Title VII evaluation reports,
  • Challenges involved in evaluating different types of programs,
  • Components and essential elements required in Title VII evaluations,
  • How to plan and conduct evaluations that meet Title VII requirements and provide useful information for program improvement,
  • Common challenges and solutions in conducting evaluations,
  • Key players in program evaluation, and
  • Information that should be reported in the biennial evaluation and the format for the report.

Evaluators who could not participate in the institute should explore these topics in the context of their evaluation plans.

Evaluation Reporting under IASA Title VII

An important part of the evaluation component of a Title VII project is the performance report. The federal government requires grantees to submit an annual performance report that provides background information about the program, demonstrates progress toward meeting the goals and objectives of the project, explains why activities or objectives have not been implemented as planned, furnishes information about current budget expenditures, and provides any other information requested by OBEMLA.

A biennial evaluation report is also required for Title VII projects. This report must provide information for program improvement, define additional goals and objectives, determine program effectiveness, and fulfill the requirements of the U.S. Department of Education. For further details on requirements for the reports, refer to IASA Title VII: Writing the Biennial Evaluation Report (see box below).

Some important points to know while conducting the evaluation under IASA Title VII projects are included in the box below.

Conducting Evaluation under IASA Title VII

As with any other federally-funded project, Title VII projects have specific guidelines for evaluation. The following three points in the act are particularly important.

  • An evaluation in the form prescribed by the U.S. Secretary of Education is required every two years.
  • The purpose of the evaluation is:
    • to improve the program,
    • to further define the program’s goals and objectives, and
    • to determine program effectiveness.
  • Evaluation shall included these components:
    • student achievement – show how students are achieving the state and student performance standards;
    • program implementation indicators, including data on appropriateness of curriculum in relationship to grade and course requirements, appropriateness of program management, appropriateness of program staff development, and appropriateness of language instruction;
    • program context indicators that describe the relationship of activities funded under the grant to the overall school program and other federal, state or local programs serving children and youth of limited English proficiency;
    • other information as the U.S. Secretary of Education may require.

– Improving America’s Schools Act of 1994, Title VII Section 7123

Resources on Evaluation

Data Analysis for Comprehensive School Improvement
by V.L. Bernhart (1998)
Eye on Education
Larchmont, New York
914/833-0551

IASA Title VII: Writing the Biennial Evaluation Report
by Center for the Education and Study of Diverse Populations (1998)
Rio Rancho, New Mexico
505/891-6111

A Model for School Evaluation
by J.R. Sanders, J.L. Horn, R.A. Thomas, D.M. Tuckett, and H. Yang (1995)
Center for Research on Educational Accountability and Teacher Evaluation
Kalamazoo, Michigan
616/387-5895

“Significant and Worthwhile Changes in Educational Evaluation: Putting Value in Evaluation”
by J.D. Supik (September 1999)
IDRA Newsletter
5835 Callaghan Road, Suite 350
San Antonio, Texas 78228-1190
210/444-1710
www.idra.org

The Program Evaluation Standards, Second Edition
The Joint Committee on Standards for Educational Evaluation (1994)
Sage Publications
Thousands Oaks, California

Evaluation Handbook
Educational evaluation for federal education programs, particularly Title VII of IASA of 1994
by J. Wilde and S. Sockey (1995)

Resources

For information about STAR Center services call 1-888-FYI-STAR.


Adela Solís, Ph.D., is a senior education associate in the IDRA Division of Professional Development. Comments and questions may be directed to her via e-mail at feedback@idra.org.


[©2000, IDRA. This article originally appeared in the January 2000 IDRA Newsletter by the Intercultural Development Research Association. Permission to reproduce this article is granted provided the article is reprinted in its entirety and proper credit is given to IDRA and the author.]

Share