Yes, you read that correctly. The CLASS isn’t reliable in scientific terms. But, it is valid: you can count on the CLASS to measure what is predictive of positive developmental and academic outcomes for children, and it has been validated through more than a hundred studies. In research terms, however, tools aren’t reliable on their own. People can be trained to sufficient reliability with successful completion of CLASS observation training and reliability testing. To maintain high levels of reliability, it’s important to think about your own professional development plan to maintain the skills you recently acquired.
So, once you complete training and pass your certification test, you’re reliable, right? Well, yes and no. You are considered reliable because you demonstrated sufficient reliability on that test. You demonstrated the capacity to collect data reliably, but it’s critical that each time you observe in a classroom, you observe and code with the same amount of precision as you did for the certification (or recertification) test.
The extent to which observers need to maintain high levels of reliability is dependent on the purpose of CLASS observations. Teachstone certifies observers who have reached 80% agreement with master codes for a one-year period from certification testing. This minimum requirement allows for broad access to the CLASS for those who demonstrate reasonable accuracy.
The way CLASS data are being used should drive decisions about the type and frequency of observer supports provided throughout the year. For example, when CLASS is being used for accountability or other high-stakes purposes, programs should consider setting higher levels of reliability, and/or implementing frequent reliability checks, to help increase accuracy in CLASS observations. This can be achieved in a variety of ways:
Teachstone recommends that new observers double-code their first observation with an experienced team member (someone who has been coding with CLASS for at least one year) or a highly experienced CLASS affiliate trainer. After the observation, the two coders debrief about each cycle, coming to consensus. If the new coder and experienced coder are within one point of each other on at least 80% of the dimensions, the new coder is considered reliable. If they are in agreement on less than 80% of the scale, then it is recommended that the new coder participate in double coding again until reaching a minimum of 80% agreement with the experienced coder. Observers can also receive double-coding support from Teachstone. Double coding involves a Teachstone expert observer traveling to the site, coding alongside the client observer, and holding a debriefing session afterward.
Calibration provides opportunities for observers to code a sample classroom and receive feedback on their codes from a Teachstone master coder. Calibrations are recommended six months after becoming certified on the CLASS and six months after each successful recertification. For individuals who conduct frequent observations, typically for accountability or research, calibration is recommended every 2–4 weeks or every 10–15 live observations. Calibration involves coding Teachstone videos that have been master coded and determining reliability against the master codes. If a coder fails to achieve at least 80% agreement, they should receive extra support from Teachstone or another veteran coder. After individuals watch and code the calibration videos, it is recommended that groups meet to discuss the master codes, dimension by dimension, focusing on dimensions where observers coded with insufficient reliability. Observers can also purchase access to webinars with their calibration.
Double Coding 10–20% of all video observations. Many CLASS users also decide to double code videos to provide ongoing insight into the reliability of the data being collected. Although the amount of double coding that should occur depends on the way the CLASS is being used, we typically recommend double coding 10–20% of observations. This allows users to monitor for problems in data collection. For example, it is possible to identify particular coders who may consistently be scoring quite differently than a second coder. These issues can then be addressed with additional calibration support in ways that will help ensure higher quality data.
The ultimate goal of all of these efforts (training, certification, calibration, and double coding) is to ensure that the CLASS data collected offer the most accurate assessment of teacher-child interactions. Although taking these steps is an investment in time and resources, it is an essential part of high-quality CLASS implementation.
There are things you need to do to make the CLASS (or any measurement tool, for that matter) useful for you. That’s why training, certification, and ongoing professional development are so important. We focus on teacher professional development with the CLASS, so doesn’t it make sense that observers and coaches using the CLASS also have their own professional development plans? Here are some resources that you can use to support your observers and coaches as they use the CLASS to support teachers.
At Teachstone, we talk to a lot of educators. From coast to coast and around the globe, there’s a common thread that unites them: wanting to be better for their students.
Even when things are tough in education, even in years made even more challenging by the pandemic and its effects on teaching and learning, educators are striving to be their best. That dedication to equitable, ongoing development is what inspires Teachstone’s work. To reach the day when all children are afforded excellent education and care, it’s going to take a systematic, data-driven approach, and we are enthusiastic partners in getting there.
Hey there, Teachstone community! My name is Stephanie Lewandowski, and I am the Senior Product Manager for myTeachstone. Before joining Teachstone, I built digital products for education companies, financial institutions, and government agencies. I’m passionate about delivering impactful products, particularly the tools that make the everyday work of teaching and learning a little bit easier. As a parent, and as a product manager, I know how invaluable early childhood education is, and I’m inspired by the teachers in both my personal and professional life.
Did you know that over 12 million children in the United States (and more every year!) speak a language other than English at home? While the education workforce does not exactly parallel its students’ demographics, we know that many educators are also multilingual. That’s why Teachstone has resources available in both English and Spanish. All children deserve the individualized support and care that best fosters learning - and so do their caregivers and educators.
At Teachstone, our driving vision is to ensure every child experiences life-changing teaching. This mission is why we’re making a commitment to restabilize and improve education for every child, and every educator. And, we know that bringing this commitment to life requires providing education leaders with the support they need to not only face the current challenges, but that will propel towards the future of quality and equity.