Yes, you read that correctly. The CLASS isn’t reliable in scientific terms. But, it is valid: you can count on the CLASS to measure what is predictive of positive developmental and academic outcomes for children, and it has been validated through more than a hundred studies. In research terms, however, tools aren’t reliable on their own. People can be trained to sufficient reliability with successful completion of CLASS observation training and reliability testing. To maintain high levels of reliability, it’s important to think about your own professional development plan to maintain the skills you recently acquired.
So, once you complete training and pass your certification test, you’re reliable, right? Well, yes and no. You are considered reliable because you demonstrated sufficient reliability on that test. You demonstrated the capacity to collect data reliably, but it’s critical that each time you observe in a classroom, you observe and code with the same amount of precision as you did for the certification (or recertification) test.
The extent to which observers need to maintain high levels of reliability is dependent on the purpose of CLASS observations. Teachstone certifies observers who have reached 80% agreement with master codes for a one-year period from certification testing. This minimum requirement allows for broad access to the CLASS for those who demonstrate reasonable accuracy.
The way CLASS data are being used should drive decisions about the type and frequency of observer supports provided throughout the year. For example, when CLASS is being used for accountability or other high-stakes purposes, programs should consider setting higher levels of reliability, and/or implementing frequent reliability checks, to help increase accuracy in CLASS observations. This can be achieved in a variety of ways:
Teachstone recommends that new observers double-code their first observation with an experienced team member (someone who has been coding with CLASS for at least one year) or a highly experienced CLASS affiliate trainer. After the observation, the two coders debrief about each cycle, coming to consensus. If the new coder and experienced coder are within one point of each other on at least 80% of the dimensions, the new coder is considered reliable. If they are in agreement on less than 80% of the scale, then it is recommended that the new coder participate in double coding again until reaching a minimum of 80% agreement with the experienced coder. Observers can also receive double-coding support from Teachstone. Double coding involves a Teachstone expert observer traveling to the site, coding alongside the client observer, and holding a debriefing session afterward.
Calibration provides opportunities for observers to code a sample classroom and receive feedback on their codes from a Teachstone master coder. Calibrations are recommended six months after becoming certified on the CLASS and six months after each successful recertification. For individuals who conduct frequent observations, typically for accountability or research, calibration is recommended every 2–4 weeks or every 10–15 live observations. Calibration involves coding Teachstone videos that have been master coded and determining reliability against the master codes. If a coder fails to achieve at least 80% agreement, they should receive extra support from Teachstone or another veteran coder. After individuals watch and code the calibration videos, it is recommended that groups meet to discuss the master codes, dimension by dimension, focusing on dimensions where observers coded with insufficient reliability. Observers can also purchase access to webinars with their calibration.
Double Coding 10–20% of all video observations. Many CLASS users also decide to double code videos to provide ongoing insight into the reliability of the data being collected. Although the amount of double coding that should occur depends on the way the CLASS is being used, we typically recommend double coding 10–20% of observations. This allows users to monitor for problems in data collection. For example, it is possible to identify particular coders who may consistently be scoring quite differently than a second coder. These issues can then be addressed with additional calibration support in ways that will help ensure higher quality data.
The ultimate goal of all of these efforts (training, certification, calibration, and double coding) is to ensure that the CLASS data collected offer the most accurate assessment of teacher-child interactions. Although taking these steps is an investment in time and resources, it is an essential part of high-quality CLASS implementation.
There are things you need to do to make the CLASS (or any measurement tool, for that matter) useful for you. That’s why training, certification, and ongoing professional development are so important. We focus on teacher professional development with the CLASS, so doesn’t it make sense that observers and coaches using the CLASS also have their own professional development plans? Here are some resources that you can use to support your observers and coaches as they use the CLASS to support teachers.
Across the country and around the globe, schools/programs will soon reopen after extended closures due to COVID-19. Those that have remained open are instituting new health and safety practices.. Localities will determine whether to provide in-person, online, or hybrid teaching. Regardless of the model that schools/programs adopt, classrooms will look different now and for the foreseeable future.
CLASS allows us to quantify the quality of teacher-child interactions—and that is a powerful thing. But improving child outcomes takes more than just data collection; it’s what you do with the data that really matters.
Here are 4 things you should know about using data to improve student outcomes.
It’s been a great year. You have just conducted some professional development trainings for the group of teachers you are coaching. You got the opportunity to visit their classrooms and see them in action, do formal and informal CLASS observations, and had countless coaching conversations. You see that it’s all beginning to click. You have the teachers’ buy-in, and the motivation is high.