Recently, I have been updating the official syllabi for some of my courses, and I have been experiencing some cognitive dissonance in that process. At my university, we are supposed to write out objectives using “action verbs” and we are specifically prohibited
from using the the words “know” and “understand.” The goal is to make objectives that are measurable. But that is where my dissonance is coming from: I think this approach is conflating the goals and the measurement.

For example, for the intro stat class I would write an objective as

  • understands the mean, median and mode and the strengths and weakness of each measure.

Naturally, as the key verb is understands, it cannot be directly measured. That’s okay, though, I can just apply evidence-centered design and ask the question about what kind of observations could provide evidence for this objective. Some examples
might be

  • Can calculate the mean, median and mode of a small data set using only a calculator.
  • Can predict the effects of outliers on each of the three measures.
  • Can recognize situations in which the median is a better measure of center than the mean.

Naturally, this list of not exhaustive.

One part of my cognitive dissonance arises because the Curriculum Committee which set up the rules for the syllabus objectives are coming at assessment from a different perspective than I am. In referring to a paper
of Bob Mislevy’s where he lays out four psychological perspectives for educational arguments: a trait perspective, a behaviorist perspective, a information-processing perspective, and a sociocultural perspective. The Curriculum committee seems to favor a behaviourist perspective where the focus is on specific behaviors we can observe
the students do. I, on the other hand, am coming at the problem of course design from a information-processing perspective. My concern is whether or not the students acquire basic statistical tools they can use to process information in the course of the class.

Even though I can write out a list of behaviours which correspond to the list of knowledge objectives I want to measure, the list of knowledge objectives is more compact. Can I really properly write out everything I want the students to be able to do with the concept of mean? This results in a list of course objectives that is several pages long. Furthermore, I like the idea of being to make the list of behaviours open ended, I’m not sure I could list out every way I could eventually think about that a student in an introductory stat class could use a mean. On the other hand, the students probably strongly prefer a finite list of objectives, so that they can be sure that they have studied every type of problem that is likely to appear on the test.

Curiously, if I came at this from a sociocultural perspective, I would not have the same difficulty with the action words. One of the objects for my intro stat class is for the student

  • to be able to working from computer (SPSS) output describe the results of the analysis in the style of the results section of a research paper.

The active verb here is “describe”, which is just fine with the Curriculum Committee. However, the phrase “style of the results section of a research paper” hides a multitude of details. In fact, I don’t think I could write out all of the rules for good style
(although I have made some attempts), all I can really say is that I know it when I see it. I have a remarkable good agreement rate with other faculty members on this topic, but even senior graduate students helping me provide feedback to students in the stat class miss many of the stylistic details I think are important.

To be fair, I think the Curriculum Committee’s rule is at least moving in the right direction. It forces the course designer to think about how the objectives will be measured, which is a good thing. I’m just not sure that we have a consensus on what the best way to write measurable objectives is.

Advertisements