PIAE - III

Assessment - There comes a time when we need to measure what we have done. But measuring somethings can prove to be quite challenging - psychological constructs are perfect examples. Psychological constructs are things we know exist, but cannot prove by measuring directly. Examples are intelligence, love, happiness, etc.  Since we cannot measure them directly, we look for behaviors we think are related to the construct.  If someone is happy, we would expect to see certain indications like smiling, laughing, being carefree, etc.  Likewise, we measure intelligence by looking for "smart" behaviors.  So, when we deal with intelligence, learning, or thinking, we will never truly measure the source.  For those philosophers amongst us, Plato's Allegory of the Cave is an interesting parallel to this dilemma.

So we have this issue of not being able to measure the "academic growth" of our students.  We have other issues as well, for instance we can revisit the "transmission vs construction" debate, and that would add even more complexity here.  If we simply transmit information into their brains, we could measure it much more simply than if they indeed construct it on their own. If they do construct, it will have unique and idiosyncratic features, often due to the diversity in their schemata.  Perhaps this is why we give so many true/false, multiple choice, and yes/no questions, to force them back into the right model. You can feel this diverse knowledge construction more when you give open-ended questions like short answer and essays, as you see the places they will push the information given a chance. And if they do construct their knowledge, and we force them into very tight narrow assessment windows, what are we cutting off, what new creative information is lost to us?

Finally, one of the greatest challenges in measuring these constructs is that the students do not think and construct in a language (one we speak anyway), they use other tools, but when they have to be measured, the students must mediate these other tools into words and into confining test mechanisms. Then have them perform in a second language (or even third or fourth) and imagine how far you really are from what is in those beautiful heads.  Ever wonder why people hate tests? Why they fear them? We think it is because of the consequences of the exercise, but it might be that it is so darn unnatural.

I will have a lot to say about testing and test construction later, but I want to talk a little bit about testing now - what a test does and doesn't do, and what a student should look forward to when taking one. If you read the chapter on the Three Responsibilities of a Student, you may remember there are actually four stages initially - Comprehension, Storage, Retrieval, and Expression. I contend that tests are stupid, they really give us very little information, and they are very poor at the "whys."  You might remember this series of scenario, if so, I apologize: Thinking about the four stages, CSRE, remember a time when you had a difficult time with a subject. Did it ever happen that you didn't comprehend it all before the test? When you took the test there was no comprehension, nothing to store or to retrieve, and nothing to express - the test said you didn't know it. Maybe you comprehended it but failed to store it,there was still nothing to to retrieve or express, and according to the test you didn't know it just like the person who never understood it. If you have test anxiety, perhaps you comprehended it, stored it, but couldn't retrieve it due to your stress - nothing to express, you don't know it. Finally, think of a very difficult essay test - you comprehend the material, stored it, retrieved it, but then butchered it trying to express it. You don't know it!  You can begin to see how uninsightful tests can be and how important language is in this whole process.

A student's prime goal when coming into a test should be to get the information from her head onto the test paper. No more, no less.  We need to teach them how to do this fairly. I will talk a great deal about this in a few weeks :)  One last note on assessment: TS = OS + E where a student's True Score (actual ability on the construct) equals his Observed Score (the test grade or result) plus or minus error.  We can make mistakes and/or guess luckily on some questions.  The point is there is always some sort of error on a test. We never see the true score, let alone the original construct - now we are even further away than earlier.  Phew.

Think over these things, let me know what you think (if you think for those of you not posting :)