Last Friday, the federal Office of Planning, Research, and Evaluation (OPRE) published an evaluation of the Office of Head Start’s (OHS) Designation Renewal System (DRS). Given the complex nature of the study, from the recruitment of the sample to the numerous quality measures, we thought it would be helpful to put the findings in context and begin to address the important questions raised in this report. We also are pleased to provide this snapshot summary of the research on the CLASS® involving thousands of classrooms and tens of thousands of students across the age levels, from infant care through secondary education. In collaboration with practitioners, researchers, and policymakers across the field, we are learning and building on our commitment to ensuring outstanding early childhood education for every child in every classroom.
On November 4, 2016, the federal Office of Planning, Research, and Evaluation (OPRE) within the Administration for Children and Families (ACF) released Early Implementation of the Head Start Designation Renewal System: Volume I, a report describing results from the first evaluation of the initial implementation of the Designation Renewal System (DRS). This report includes information relevant for understanding how classroom assessments, such as the Classroom Assessment Scoring System® (CLASS®) and the Early Childhood Environment Rating Scale (ECERS), can be utilized in accountability and program development systems. Given the role of the CLASS in the DRS and program improvement, Teachstone thought it would be helpful to the field to provide additional perspectives on the report and our plans to improve and ensure the best possible use of the CLASS and associated professional development on behalf of children and teachers.
One finding from the DRS study was that classroom quality scores as measured by the CLASS and ECERS were not significantly different for grantees who were and were not designated to compete for funding, suggesting that programs above and below re-competition cut score lines were not differentiated by the CLASS. In other analyses, CLASS scores were found to be significantly higher when collected by Office of Head Start (OHS) observers than when collected by the study’s evaluation team. These results raise appropriate questions about how best to use classroom observations, considered essential for describing the features of quality most salient for children’s development and learning, in the context of accountability systems.
High-stakes accountability systems are relatively new in early childhood education; as a result, the field still has much to learn about how to most effectively use observational measures within these systems. Teachstone is confident that the results of this study point to directions for improving the use of CLASS in a system such as the DRS. Using a validated measure of teacher-child interactions as part of the DRS has helped to transform the field’s definition of high-quality early childhood programs. Teachstone is deeply committed to working with OHS and others in the field to help ensure that the systems using the CLASS realize the intended benefits. The DRS report provides useful and actionable information to help inform these efforts.
In part as a result of what we have learned from the use of CLASS in research and practical applications, the field has refined what is meant by the term quality. Programs around the country are investing to ensure that teachers engage children with stimulating and responsive interactions essential to improving academic and social-emotional outcomes. The importance of effective teacher-children interactions is reinforced through findings from over 100 peer-reviewed research studies, most using CLASS®, indicating strong associations between features of teachers’ interactions with students and positive changes in learning and development. These studies, some from causal inference designs, contributed to redefining the term “quality” from a focus on structural or environmental factors, such as teacher-child ratios or the number of books in the classroom, to the actual “point of impact” — the effectiveness with which teachers interact with children. Because of this evidence base, and evidence showing targeted professional development can foster improved interactions, the CLASS has been used in the DRS as an indicator of quality.
At Teachstone, we hold a steadfast belief in the necessary connection between observation and professional development to improve quality. This link, and the focus on research-based indicators of quality, was confirmed with the new Head Start Performance Standards and continues to be a focus in the work we do.
A primary purpose of the DRS evaluation was to test whether grantees designated and not designated for recompetition differed on the quality of their Head Start classrooms. It compared CLASS domain scores as collected by the OHS observers with those collected by the evaluation team to assess agreement and potential bias. Randomly selected classrooms in 71 randomly selected grantees (35 designated for competition, 36 not designated) were assessed using the CLASS. Researchers tested for differences between designated and not designated grantees on these classroom quality measures, as well as center-level quality measures. As noted above, no differences were detected.
As with all research, there are important challenges and limitations to this study, from the study design to the difficulty in describing specifics of the assessment tools used and protocol followed. Understanding this is important as it allows us to most clearly articulate the evidence and how the field can best respond. It highlights the complexity of conducting a rigorous study such as this and the need to review all new systems for improvement. Teachstone is committed to draw on these results to improve the current system with necessary refinements to ensure the original goal of DRS is met: to ensure all Head Start children have access to high-quality classrooms.
At Teachstone, these results are an important reminder to us that classroom observation systems must be ever-evolving, with improvement and refinement of the training and support of observers and the system as a whole. We are committed to ensuring the highest quality training, scoring, and implementation with the CLASS and associated professional development and will continue to collaborate with partners in the field to ensure effective and appropriate systems are implemented.
Teachstone is proud of our work with Head Start over the years and unwavering in our commitment to ensuring all Head Start children are in high-quality classrooms that support their social and cognitive development. We are dedicated to the use of the CLASS® to help teachers understand and improve the daily interactions in their classrooms, as well as its use as a measure of quality when supported by a rigorous reliability and support system.
We are working with the field to determine the needed improvements to high-stakes accountability systems to refine the training, scoring, and implementation procedures already in place. Many of these refinements are used by observation teams, such as the OHS observers. These enhancements may include:
Accurately measuring classroom quality can be complex, and key to the accuracy of the CLASS is appropriate implementation. This is something Teachstone has always taken seriously, and we remain committed to ensuring that all trainings and services are delivered with the quality we know is needed. This is not easy – it requires resources, time, and expertise. We believe in the process we have put in place, and in the spirit of Head Start, we will work with our partners to engage in continuous quality improvement to support stronger implementation of the CLASS as part of the monitoring process.
All children deserve access to a high-quality early education experience to ensure a solid foundation for future learning. Strong teacher-child interactions, as measured by the CLASS, are a fundamental resource for children’s learning and development. We look forward to working together with partners in the field to strengthen teachers and their work with all children.
Greetings! One of my New Year’s resolutions is to blog more than last year. While I’m not the most prolific, when I do post, please know it comes from the heart. And, there’s nothing I’m more passionate about than Head Start and its mission to support young children and families through a program of comprehensive services that can move mountains for our most vulnerable young children.
Is this your program’s first year conducting CLASS observations? Do you have new teachers who have never been observed? Implementing any kind of change in an organization can be challenging, so it’s important to provide many opportunities to discuss the factors behind the change and allow your staff to engage in open-ended discussions.
Here are some conversation points to help your team feel at ease before CLASS observations begin.
Welcome to our newest blog series dedicated to the research we're reading and thinking about.
For our first post in this series, we’re looking at exclusionary disciplinary practices with new eyes as states are submitting their ESSA plans. The Every Student Succeeds Act requires states to discuss how they will help local education agencies reduce their overuse of exclusionary discipline practices. These are actions like suspensions or expulsions that send students out of classrooms. Not only do exclusionary discipline practices negatively affect school climate (something we care a lot about here at Teachstone!), evidence shows that students of color, particularly Black students, are disproportionately on the receiving end.
This post was originally published by the McCormick Center for Early Childhood Leadership.
I often think about my time working as a director in a child care program and wonder how different things would have been if I had known then, what I know now. As time passes and I gain new experiences and insights on leadership in early childhood education, I frequently ask myself what I would do differently if I could relive that period of time. In my reflection, I have realized that my conclusions are from my point of view. Recognizing that the experience I had as a program administrator affected so many, I thought it would be interesting to learn what my team would like for me to have known.