Last Friday, the federal Office of Planning, Research, and Evaluation (OPRE) published an evaluation of the Office of Head Start’s (OHS) Designation Renewal System (DRS). Given the complex nature of the study, from the recruitment of the sample to the numerous quality measures, we thought it would be helpful to put the findings in context and begin to address the important questions raised in this report. We also are pleased to provide this snapshot summary of the research on the CLASS® involving thousands of classrooms and tens of thousands of students across the age levels, from infant care through secondary education. In collaboration with practitioners, researchers, and policymakers across the field, we are learning and building on our commitment to ensuring outstanding early childhood education for every child in every classroom.
On November 4, 2016, the federal Office of Planning, Research, and Evaluation (OPRE) within the Administration for Children and Families (ACF) released Early Implementation of the Head Start Designation Renewal System: Volume I, a report describing results from the first evaluation of the initial implementation of the Designation Renewal System (DRS). This report includes information relevant for understanding how classroom assessments, such as the Classroom Assessment Scoring System® (CLASS®) and the Early Childhood Environment Rating Scale (ECERS), can be utilized in accountability and program development systems. Given the role of the CLASS in the DRS and program improvement, Teachstone thought it would be helpful to the field to provide additional perspectives on the report and our plans to improve and ensure the best possible use of the CLASS and associated professional development on behalf of children and teachers.
One finding from the DRS study was that classroom quality scores as measured by the CLASS and ECERS were not significantly different for grantees who were and were not designated to compete for funding, suggesting that programs above and below re-competition cut score lines were not differentiated by the CLASS. In other analyses, CLASS scores were found to be significantly higher when collected by Office of Head Start (OHS) observers than when collected by the study’s evaluation team. These results raise appropriate questions about how best to use classroom observations, considered essential for describing the features of quality most salient for children’s development and learning, in the context of accountability systems.
High-stakes accountability systems are relatively new in early childhood education; as a result, the field still has much to learn about how to most effectively use observational measures within these systems. Teachstone is confident that the results of this study point to directions for improving the use of CLASS in a system such as the DRS. Using a validated measure of teacher-child interactions as part of the DRS has helped to transform the field’s definition of high-quality early childhood programs. Teachstone is deeply committed to working with OHS and others in the field to help ensure that the systems using the CLASS realize the intended benefits. The DRS report provides useful and actionable information to help inform these efforts.
In part as a result of what we have learned from the use of CLASS in research and practical applications, the field has refined what is meant by the term quality. Programs around the country are investing to ensure that teachers engage children with stimulating and responsive interactions essential to improving academic and social-emotional outcomes. The importance of effective teacher-children interactions is reinforced through findings from over 100 peer-reviewed research studies, most using CLASS®, indicating strong associations between features of teachers’ interactions with students and positive changes in learning and development. These studies, some from causal inference designs, contributed to redefining the term “quality” from a focus on structural or environmental factors, such as teacher-child ratios or the number of books in the classroom, to the actual “point of impact” — the effectiveness with which teachers interact with children. Because of this evidence base, and evidence showing targeted professional development can foster improved interactions, the CLASS has been used in the DRS as an indicator of quality.
At Teachstone, we hold a steadfast belief in the necessary connection between observation and professional development to improve quality. This link, and the focus on research-based indicators of quality, was confirmed with the new Head Start Performance Standards and continues to be a focus in the work we do.
A primary purpose of the DRS evaluation was to test whether grantees designated and not designated for recompetition differed on the quality of their Head Start classrooms. It compared CLASS domain scores as collected by the OHS observers with those collected by the evaluation team to assess agreement and potential bias. Randomly selected classrooms in 71 randomly selected grantees (35 designated for competition, 36 not designated) were assessed using the CLASS. Researchers tested for differences between designated and not designated grantees on these classroom quality measures, as well as center-level quality measures. As noted above, no differences were detected.
As with all research, there are important challenges and limitations to this study, from the study design to the difficulty in describing specifics of the assessment tools used and protocol followed. Understanding this is important as it allows us to most clearly articulate the evidence and how the field can best respond. It highlights the complexity of conducting a rigorous study such as this and the need to review all new systems for improvement. Teachstone is committed to draw on these results to improve the current system with necessary refinements to ensure the original goal of DRS is met: to ensure all Head Start children have access to high-quality classrooms.
At Teachstone, these results are an important reminder to us that classroom observation systems must be ever-evolving, with improvement and refinement of the training and support of observers and the system as a whole. We are committed to ensuring the highest quality training, scoring, and implementation with the CLASS and associated professional development and will continue to collaborate with partners in the field to ensure effective and appropriate systems are implemented.
Teachstone is proud of our work with Head Start over the years and unwavering in our commitment to ensuring all Head Start children are in high-quality classrooms that support their social and cognitive development. We are dedicated to the use of the CLASS® to help teachers understand and improve the daily interactions in their classrooms, as well as its use as a measure of quality when supported by a rigorous reliability and support system.
We are working with the field to determine the needed improvements to high-stakes accountability systems to refine the training, scoring, and implementation procedures already in place. Many of these refinements are used by observation teams, such as the OHS observers. These enhancements may include:
Accurately measuring classroom quality can be complex, and key to the accuracy of the CLASS is appropriate implementation. This is something Teachstone has always taken seriously, and we remain committed to ensuring that all trainings and services are delivered with the quality we know is needed. This is not easy – it requires resources, time, and expertise. We believe in the process we have put in place, and in the spirit of Head Start, we will work with our partners to engage in continuous quality improvement to support stronger implementation of the CLASS as part of the monitoring process.
All children deserve access to a high-quality early education experience to ensure a solid foundation for future learning. Strong teacher-child interactions, as measured by the CLASS, are a fundamental resource for children’s learning and development. We look forward to working together with partners in the field to strengthen teachers and their work with all children.
As fall breezes sweep in and the smell of freshly sharpened pencils fill the air, we know lots of teachers are making their own new year’s resolutions. After all, when better to reflect on your practice and set new intentions than at the beginning of the academic calendar? Maybe teachers are creating goals that center on social-emotional learning, or they’re recommitting to building strong relationships with their students. Whatever the goal, a 2013 study shows us the value in getting an early start.
Welcome to our newest blog series dedicated to the research and reports Teachstone is reading and thinking about.
For our first post in this series, we’re looking at exclusionary disciplinary practices with new eyes as states are submitting their ESSA plans. The Every Student Succeeds Act requires states to discuss how they will help local education agencies reduce their overuse of exclusionary discipline practices. These are actions like suspensions or expulsions that send students out of classrooms. Not only do exclusionary discipline practices negatively affect school climate (something we care a lot about here at Teachstone!), evidence shows that students of color, particularly Black students, are disproportionately on the receiving end.
This spring we released a new research paper, "Effective Teacher-Child Interactions and Child Outcomes." It summarizes what 150 peer review studies have to say about CLASS. In conjuction, we designed a handy infographic with a few key points and stats from the paper. You'll also find a few facts about Teachstone. Enjoy!
If you’ve ever attended a CLASS Observation Training, you’ve heard the trainer state that the CLASS is a valid tool for measuring the efficacy to teacher-child interactions: that classroom quality, as measured by the CLASS, predicts positive developmental and academic outcomes for children (predictive validity). Specifically, children who attend classrooms with higher CLASS scores demonstrate better social and academic outcomes than their peers in classrooms that were not rated as highly.