Last Friday, the federal Office of Planning, Research, and Evaluation (OPRE) published an evaluation of the Office of Head Start’s (OHS) Designation Renewal System (DRS). Given the complex nature of the study, from the recruitment of the sample to the numerous quality measures, we thought it would be helpful to put the findings in context and begin to address the important questions raised in this report. We also are pleased to provide this snapshot summary of the research on the CLASS® involving thousands of classrooms and tens of thousands of students across the age levels, from infant care through secondary education. In collaboration with practitioners, researchers, and policymakers across the field, we are learning and building on our commitment to ensuring outstanding early childhood education for every child in every classroom.
On November 4, 2016, the federal Office of Planning, Research, and Evaluation (OPRE) within the Administration for Children and Families (ACF) released Early Implementation of the Head Start Designation Renewal System: Volume I, a report describing results from the first evaluation of the initial implementation of the Designation Renewal System (DRS). This report includes information relevant for understanding how classroom assessments, such as the Classroom Assessment Scoring System® (CLASS®) and the Early Childhood Environment Rating Scale (ECERS), can be utilized in accountability and program development systems. Given the role of the CLASS in the DRS and program improvement, Teachstone thought it would be helpful to the field to provide additional perspectives on the report and our plans to improve and ensure the best possible use of the CLASS and associated professional development on behalf of children and teachers.
One finding from the DRS study was that classroom quality scores as measured by the CLASS and ECERS were not significantly different for grantees who were and were not designated to compete for funding, suggesting that programs above and below re-competition cut score lines were not differentiated by the CLASS. In other analyses, CLASS scores were found to be significantly higher when collected by Office of Head Start (OHS) observers than when collected by the study’s evaluation team. These results raise appropriate questions about how best to use classroom observations, considered essential for describing the features of quality most salient for children’s development and learning, in the context of accountability systems.
High-stakes accountability systems are relatively new in early childhood education; as a result, the field still has much to learn about how to most effectively use observational measures within these systems. Teachstone is confident that the results of this study point to directions for improving the use of CLASS in a system such as the DRS. Using a validated measure of teacher-child interactions as part of the DRS has helped to transform the field’s definition of high-quality early childhood programs. Teachstone is deeply committed to working with OHS and others in the field to help ensure that the systems using the CLASS realize the intended benefits. The DRS report provides useful and actionable information to help inform these efforts.
In part as a result of what we have learned from the use of CLASS in research and practical applications, the field has refined what is meant by the term quality. Programs around the country are investing to ensure that teachers engage children with stimulating and responsive interactions essential to improving academic and social-emotional outcomes. The importance of effective teacher-children interactions is reinforced through findings from over 100 peer-reviewed research studies, most using CLASS®, indicating strong associations between features of teachers’ interactions with students and positive changes in learning and development. These studies, some from causal inference designs, contributed to redefining the term “quality” from a focus on structural or environmental factors, such as teacher-child ratios or the number of books in the classroom, to the actual “point of impact” — the effectiveness with which teachers interact with children. Because of this evidence base, and evidence showing targeted professional development can foster improved interactions, the CLASS has been used in the DRS as an indicator of quality.
At Teachstone, we hold a steadfast belief in the necessary connection between observation and professional development to improve quality. This link, and the focus on research-based indicators of quality, was confirmed with the new Head Start Performance Standards and continues to be a focus in the work we do.
A primary purpose of the DRS evaluation was to test whether grantees designated and not designated for recompetition differed on the quality of their Head Start classrooms. It compared CLASS domain scores as collected by the OHS observers with those collected by the evaluation team to assess agreement and potential bias. Randomly selected classrooms in 71 randomly selected grantees (35 designated for competition, 36 not designated) were assessed using the CLASS. Researchers tested for differences between designated and not designated grantees on these classroom quality measures, as well as center-level quality measures. As noted above, no differences were detected.
As with all research, there are important challenges and limitations to this study, from the study design to the difficulty in describing specifics of the assessment tools used and protocol followed. Understanding this is important as it allows us to most clearly articulate the evidence and how the field can best respond. It highlights the complexity of conducting a rigorous study such as this and the need to review all new systems for improvement. Teachstone is committed to draw on these results to improve the current system with necessary refinements to ensure the original goal of DRS is met: to ensure all Head Start children have access to high-quality classrooms.
At Teachstone, these results are an important reminder to us that classroom observation systems must be ever-evolving, with improvement and refinement of the training and support of observers and the system as a whole. We are committed to ensuring the highest quality training, scoring, and implementation with the CLASS and associated professional development and will continue to collaborate with partners in the field to ensure effective and appropriate systems are implemented.
Teachstone is proud of our work with Head Start over the years and unwavering in our commitment to ensuring all Head Start children are in high-quality classrooms that support their social and cognitive development. We are dedicated to the use of the CLASS® to help teachers understand and improve the daily interactions in their classrooms, as well as its use as a measure of quality when supported by a rigorous reliability and support system.
We are working with the field to determine the needed improvements to high-stakes accountability systems to refine the training, scoring, and implementation procedures already in place. Many of these refinements are used by observation teams, such as the OHS observers. These enhancements may include:
Accurately measuring classroom quality can be complex, and key to the accuracy of the CLASS is appropriate implementation. This is something Teachstone has always taken seriously, and we remain committed to ensuring that all trainings and services are delivered with the quality we know is needed. This is not easy – it requires resources, time, and expertise. We believe in the process we have put in place, and in the spirit of Head Start, we will work with our partners to engage in continuous quality improvement to support stronger implementation of the CLASS as part of the monitoring process.
All children deserve access to a high-quality early education experience to ensure a solid foundation for future learning. Strong teacher-child interactions, as measured by the CLASS, are a fundamental resource for children’s learning and development. We look forward to working together with partners in the field to strengthen teachers and their work with all children.
Since I joined Teachstone as Chief Impact Officer a few months ago, I’ve been thinking and reading a lot about opportunity gaps in education. In a great op-ed from 2013, Prudence Carter and Kevin Welner, co-editors of Closing the Opportunity Gap: What America Must Do to Give Every Child an Even Chance, wrote:
The violence of the past days has reminded us all that we must be vigilant in denouncing hate and speaking out against it. I encourage everyone who cares about our courageous country and experiment in democracy to get out and vote on November 6th! Paraphrasing from an editorial in the Hartford Courant today, the strength of our united character is being put to the test. We must speak up, and no better way to do so than voting or encouraging others to do so. Truly great educators are those who never give up. I am not ready to give up on our country as a land that holds such great promise of freedom and justice for all, beginning with our youngest ones who cannot vote but will live in the world we are creating with every action or inaction.
Take heart! For too long, one of the least heartening perspectives on the federal government budget came from early childhood education advocates, who, year in and year out, felt left out of the political dialogue during budget talks. They were either ignored or, worse, the recipients of unwanted attention as federal spending on education was slashed or level-funded as costs increased. However, during a time when we see much division in our country, especially coming out of Washington, D.C., it actually is a bit –just a bit – encouraging to look at early childhood education programs which are garnering more and more bipartisan support.
The statistics around exclusionary discipline practices, like suspension or expulsion, are grim. Kids who get kicked out, especially repeatedly, are often already behind academically, become less engaged in school, and are monumentally more likely to drop out of high school. And while exclusionary discipline affects all students, it’s essential to keep in mind that children of color are suspended and expelled at rates disproportionate to their white peers.