“When is a test not a test?” is not a trick question: tests must measure what they are intended to measure. In providing the general TCI curriculum, it is a challenge for some staff to require the course content so that they become proficient enough to perform satisfactorily at work and on the written and physical exams presented at the end of the coursework.
If the pragmatic goal of the TCI written exam is to review learning and assure competency rather than to assign grade scores of passing or failing, the test review process and an awareness of how individuals acquire and produce information becomes even more meaningful.
Errors reflected during training or upon taking the written exam are often attributed to problems in learning the extensive content; an apparently logical connection. However, the “real” problem may lie with the interaction between the nature of the testing process and the participant’s individual learning style, rather than a lack of knowledge or competence.
At Sweetser, TCI is delivered in the 24 hour+/4 day process. For new, inexperienced employees, a great deal of important information is delivered quite rapidly over the first few days. Even with all of the excellent training components and experiential exercises TCI incorporates as part of its training process, many staff still find the course work a bit overwhelming. Over the past ten years or so of providing TCI training, we have observed that there are many alternative learners, including many learning disabled individuals in the behavioral healthcare professions.
One picks up “patterns of performance” over time, and there are distinct response patterns represented by the staff with whom I work. These have more to do with how people learn and how the test is constructed (literally its structure/process) and less to do with the content (memorizing and acquiring the concepts, meaning, and application of learning material the questions are intended to measure). It is the intent of any well-constructed test that the test questions as closely and accurately as possible reflect and “test” the course material to be learned or “content validity.”
Some staff just cannot easily respond to certain questions, question types (i.e., multiple choice), or instructions given the manner in which the questions or directions are constructed. And some individuals demonstrate difficulty extracting the meaningfulness (intent) or subtle nuances of words used in the questions which causes them confusion. Again, this does not necessarily reflect competence in applying the information.
*TCI: Therapeutic Crisis Intervention: a Crisis Prevention and Management System
- 2 -
There are also participants who are “unsophisticated test takers” who jump to identify and “check off” an initial correct response without continuing to read on to (perhaps) check out (and off) the “all of the above” answer.
We follow up our written exams with the correct answers being reviewed to assure overall comprehension and provide the staff with the opportunity to make an appointment following the completion of the TCI course (often the next week) to review the exam. However, we require staff scoring below 80% to make an appointment with their TCI facilitator (if they are on another campus) or with me to review their written exam in person and demonstrate competency.
In my experience, I would say that the larger percentage of those few who fail (under 80%) are able to perform satisfactorily orally, in person, when they cannot seem to do so during the test process.
It is important to realize how the structure of the testing process itself presents something of a roadblock for certain alternative learners. Interestingly enough, it also validates the multi-experiential approach that TCI utilizes in its training process; the information just needs to be tapped into using a slightly different process to extract the meaning (and the correct answer…)! And if we employ this process in training, why not do so in testing.
It is important to distinguish why individuals may have some difficulty in acquiring or demonstrating the TCI course content, and to provide them with opportunities to acquire and demonstrate this information using alternative approaches. When we present initial instructions for TCI participants, we ask “those who may have experienced learning difficulties in their past” to come speak with me confidentially. By so doing, we can ascertain and keep an eye out for better methods to employ with them during the course, or better monitor how their in-class performance and participation represents how much of the course material they are actually acquiring in the manner in which it is being presented.
This has proven to be a valuable strategy, resulting in decreased “test” anxiety. It has allowed us to differentiate between some very small percentage of participants who just do not do the work from those who are trying and failing due to circumstances beyond their control. The results are better trained TCI practitioners.
Article reprinted from “refocus,” The Residential Child Care Project Newsletter, Vol. 12, 2007, Family Life Development Center, College of Human Ecology, Cornell University.
Mr. Melnick is a certified TCI Instructor