Recently, I conducted a CLASS Train-the-Trainer. During the training, as we discussed going over the certification and re-certification process with their trainees, a question arose that often comes up during Observation training. Or rather in this case, it was a statement. “I wish we could know the master codes for all of our reliability test videos. I feel like I would learn more about my mistakes. The feedback just isn’t helpful to me, I want to know how I really did.”
Since this is a recurring comment in Observation trainings as well or really, an FAQ, let’s explore this question. And while we are at it, one more question that often happens during the same discussion is, “Why can’t I know how I did directly after I code each video?”
Last question first; “Why can’t I know how I did after each video?”
Here is how I address that question as a trainer. I share that the point of reliability testing is to mimic a CLASS observation. That means I’m going to be out in the field conducting 4-6 cycles of observing and coding. I’m going to need to be reliable in each cycle, and I have no one there to give me feedback between those cycles. I then go on to say, “...if I had that feedback, I may change my observation and coding behavior for each subsequent cycle. This would not be a consistent, reliable CLASS observation.”
Why don’t I get to know the [actual] master code for each reliability video after I have completed my testing?
Here is what I share after that question: The point of reliability testing is just that; we have moved beyond teaching CLASS and into testing your CLASS knowledge. A participant has attended an extensive Observation training, and now the purpose is to test the outcome. Post reliability testing feedback, while not master-code specific, does give (you) the tester important and helpful information about the over-all attempt. If a trainee has passed the reliability test, that person has demonstrated that they are ready to go out into the field and conduct observations to fidelity.
The next thing I usually say is “That’s what the training videos are for. During observation training, you are provided the opportunity to view and code 5 videos, as well as having access to a practice video. After viewing each of the videos, the actual codes are provided. This is your chance to evaluate your progress in comparison to actual master codes.”
Another way to address the question is to say, "We do not share the master codes because people who are testing won't see any of those videos again. It far better for an individual to understand his or her scoring tendencies and biases; that is what will help that person understand the tool and how to code. Furthermore, because the data is used to inform a variety of decisions ranging from how to target PD to high stakes decisions about programs, it is important to maintain a high degree of reliability in the testing process. Sharing codes with the larger user population has the potential to compromise the integrity of data being collected."
And there you have it, some helpful answers to support you in the field as you train participants to be reliable CLASS observers. What other questions about the reliability testing process come up during your trainings? How do you address those?
As a CLASS Group Coaching (MMCI) instructor, the sections of any given two-hour session may feel, at times, very goal driven. These sections titled "Know," "See," and "Do” are interconnected. In particular, it is possible to consider "Do" within "Know," and "See." When an instructor supports in-the-moment experiences that connect new knowledge to current practice, they make the CLASS dimensions more relevant to the educators' daily work. But how can we infuse more “Do” into “Know” and “See?” First, let's re-cap what happens in each section.
I have a confession to make. Recently, I used vacation time to stay home and watch Season 6 of The Walking Dead. I know, I know. How could I have let myself miss a whole season? Oh, and I feel a little bad about taking the time off from work too, but this was very nearly an emergency! I mean it was only weeks before Season 7 of the season premiere. I had to do something. Don’t judge.
While I was watching, I had the strangest feeling of deja vu. I felt like I had actually walked through a herd of zombies, but couldn’t quite place why it felt so familiar. Then it hit me—I had unknowingly created zombie-like participants during at least two of my previous CLASS trainings.
As we head into elections, I've been crafting a story to share with my local legislators. I want to let them know the many glorious reasons why they need to fund early childhood education.
Everyone knows stories matter, so as I stared at my blank piece of paper I found myself wondering: