Overview
An important truth is that regardless of how successful a well-researched, results-based model appears to be, it may not work in your own school for a multitude of reasons. That is why an ongoing evaluation of your own program and strategies is imperative.
Here are suggested guidelines to develop the evaluation component of your truancy reduction effort.
I. Goals and Objectives: All evaluation begins with goals (long term 1-3 years) and objectives (short term steps to meet those goals). Programs that clearly delineate goals and objectives are much more likely to successfully achieve those outcomes. Remember that evaluation is a TOOL to enable your program to evaluate effectiveness in order to improve and become more successful over time. Evaluation provides documentation of areas of needed improvement and success, however, use of the evaluation material and process is critical for programs to benefit from the evaluative process.
Goals and objectives may be developed across a number of areas including:
A. Program Development and Implementation
1. Internal communication, process & procedures
2. Communication & collaboration with community agencies
3. School level buy in and support
4. Safe school climate
B. Data Collection and Use
1. Targeted data identified for each goal/objective
2. Data collection methods and procedures
3. Communication of data to project staff
4. Use of data-based decision making
5. Implementation of program changes based on data
6. Data reporting to necessary and involved agencies
7. Data collection is ongoing and used to monitor project goals
C. Outcome Measures
1. Effects of project strategies on attendance & on-time arrival of referred students
2. Procedures to target groups of students or individual students whose attendance does not improve
3. Development of research based support plans (involving students and parents) to improve attendance for students whose attendance is not improving
4. Data collected and reviewed to monitor the progress of targeted students
II. Data Identification and Collection: Effective program evaluation should include both qualitative and quantitative data collection strategies depending on the specific goals and objectives, the sources of data and the questions to be answered by the evaluation. Qualitative data (word based data) might include data from interviews (project personnel, students, parents, collaborating agencies), observation and field notes about processes, samples of student attendance support plans, notes from planning meetings and internal memos, documents or other forms of communication. Quantitative data (number based data) might include percent of days of attendance or on time arrival, number of meetings between agencies, percent of project goals met per year or scores on questionnaires or client (student or parent) satisfaction surveys. Many other sources of data are available, the critical factor is that data answers the questions asked and speaks to the project's progress toward stated goals.
III. Sample Questions for Evaluation Efforts:
A. Project Development and Implementation
1. What roles do various project staff, parents, students and outside agencies play in the truancy reduction effort? How are roles and responsibilities communicated?
2. How is information communicated? Is communication effective? Do key players, including clients, feel supported by the project and the projects procedures?
3. Who is responsible for individual project goals or objectives? Do these goals and objectives speak to the mission of the project?
4. Which agencies are involved in the project? What is the level of communication and collaboration (administrative, direct care staff)? Is the level of communication appropriate, i.e., does it result in effective project implementation?
5. What strategies or activities have been developed to meet project goals and activities?
6. How is implementation being monitored? How is implementation integrity being measured?
7. What systems are in place for performance review both of employees and of leaders/supervisors by their employees?
B. Data Collection and Use
1. How and who decides what data is targeted to measure goals, objectives, implementation and project activities?
2. Once identified, how is data collected, organized and stored?
3. How is data analyzed? Who is involved in data analysis? Are the project staff responsible for implementing project activities involved in data analysis?
4. How is data collection and analysis used to evaluate projects, to inform practice and to make modifications to future implementation and evaluation efforts?
5. Are data analysis and data-based decision-making sessions truly problem solving and project improvement opportunities?
6. How is data displayed and disseminated to project staff, funding agents and project constituents? Are data displays clear and concise? Do they represent the data accurately?
C. Outcome Measures
1. What effect are project activities having on school attendance, attachment and achievement?
2. Which activities are most effective at achieving our goals? What aspects of those activities are most related to better project outcomes?
3. Which activities are not effective at achieving project goals? What can be done to alter or improve those activities? Should they be continued or new activities developed?
4. What leadership styles and behaviors are most related to achieving goals and objectives? Which leadership behaviors are least effective?
5. How effective are the support plans developed to target challenging students? What aspects of those plans are effective and which are not? What ideas are generated for improvement of those plans or processes?
6. How bought in to project outcomes are the students and their parents?
See also a list of process and outcome measures to consider when evaluating truancy programs from the Juvenile Justice Evaluation Center at http://www.jrsa.org/jjec/programs/truancy/performance-measures.html