Its time to mark my beliefs to market on my earlier statements about Behaviourist and Cognitive perspectives on assessment.

I’m now involved in a small consulting project where Betsy Becker, myself and some of our students are helping an agency review their licensure exam.  Right now we have our students working on extracting objectives from the documents of the requirements.  We are trying this out with the cognitive approach rather than the behaviourist one, so our students are asking questions about how to write the objectives.  I thought I would share our advise, and possibly get some feedback from the community at large.

I don’t have permission from our client to discuss their project, so let me use something from my introductory statistics class.  Reading the chapter on correlation, I find the following objective:

  • Understand the effect of leverage points and outliers on the correlation coefficient.

The behaviourists wouldn’t like that one, because it uses an unmeasurable verb, understands.  They would prefer met to substitute an observable verb in the statement, something like:

  • Can estimate the effect of a leverage point or outlier on the correlation coefficient.

This is measurable, but it only captures part of what I had in mind in the original objective.  The solution is to return to the original objective, but add some evidence:

  • Understand the effect of leverage points and outliers on the correlation coefficient.
    1. Given a scatterplot, can identify outliers and leverage points.
    2. Given a scatterplot and with a potential leverage point identified, can estimate the effect of removing the leverage point on the correlation coefficient.
    3. Can describe the sensitivity of the correlation coefficient to leverage points in words.
    4. Given a small data set with a leverage point, can estimate the effect of the leverage point on the correlation.
    5. Can add a high leverage point to a data set to change the correlation in a predefined direction.

The list is not exhaustive.  In fact, a strong advantage of the cognitive approach is that it suggest novel ways of thinking about measuring and perhaps then teaching the objective.

Listing at least a few examples of evidence when defining the objective helps make the objective more concrete.  It also ties it to relative difficulty, as we can move these sources of evidence up and down Bloom’s taxonomy to make the assessment harder or easier.  For example:

  1. [Easier] Define leverage point.
  2. [Harder] Generate an example with a high leverage point.

We are instructing our team to write objectives in this way.  Hopefully, I’ll get a chance later to tell you about how well it worked.