Separating the assessment from the task?

I have several projects this summer that have lead me to think deeply about assessing student learning.  Partly this time has been spent trying to understand how the structural factors surrounding the learning and assessment (class structure, assigned task, time of day, etc) effect the outcome of the assessment.  I am starting to wonder if one can control for those variables or even if it is advisable to try.  Does trying to set up a controlled assessment destroy the learning process?  I think it is a fair question.  As of yet I have no answers but I am interested in what my fellow IC-bGers might have to add to this conversation.  I ran across this quote from an excellent book on assessment written by the assessment gurus at the National Research Council and published by the National Academies Pres; Knowing what students know: The science and design of educational assessment (pdf).

“Every assessment is also based on a set of  beliefs about the kinds of tasks or situations that will prompt students to say, do, or create something that demonstrates important knowledge and skills. The tasks to which students are asked to respond an assessment are not arbitrary.“

All too often, I fail to think how the task that students do impacts the assessment and fall into the trap of thinking that using a single instrument to assess very different tasks will produce comparable data. Recently, I was lucky enough to see Charles Blaich and Kathleen Wise’s keynote at the 2013 Institute for Pedagogy in the Liberal Arts where they discussed what results from the Wabash National Study and the Center of Inquiry at Wabash College.  This multi-year study follow student cohorts from a variety of institutions through their college experiences, collecting student and institutional data with a wide-ranging set of assessment instruments.  The results are somewhat disappointing in that students don’t show great gains in many criteria and in some they even demonstrate losses in some areas (e.g., openness to diversity).

The message from this keynote and the NRC book I find compelling is the idea that assessing student learning is best done directly as a part of the learning process.  Secondary measures of learning are second best. Institutions of higher education tend to rely very heavily on national surveys such as the NSSE and similar indirect, self-reported surveys.  The problem with these assessments is that they don’t directly measure student learning but rather demonstrate student’s attitudes about their own learning.

So that leaves us with a quandary…

How will we get to comparable measures of learning that are conducted in the classroom.  Any ideas out there?

Julia

Advertisements

About Dr. J. Metzker

I believe in the power of a liberal education to transform individuals and society. I am currently the Executive Director of the Brown Center for Faculty Innovation and Excellence. Formerly, I led a community engagement initiative and held a full professorship in chemistry at a public liberal arts university. I am a proud product of The Evergreen State College.
This entry was posted in notes. Bookmark the permalink.

3 Responses to Separating the assessment from the task?

  1. I think this is an interesting topic and perhaps something that could drive a teaching circle or two and a reading circle or two. I imagine CETL+CEL (the Provost has yet to approve a name for our new Center) would welcome proposals addressing the quandry posed by Julia.
    Steve

  2. klmk says:

    This means that we have to ask philosophical & moral questions and seek their answers. To me there isn’t much danger in using an ill-formed or inappropriate assessment when it will not be used for grading or evaluation. But since almost all assessments used at the college level are for grading and/ or evaluation the assessor should certainly have examined his/her beliefs about learning, success, etc. and probably should be included on her/his syllabus.

    • klmk, Interesting idea. I wonder how I might include these beliefs on my syllabus? Over the past few years, I’ve tried some new approaches to assessing student work AND communicating the meaning of a grade rather than a number or letter. I think it is a start.

      I’ve also begun thinking that if we really need to compare student learning across different situations, perhaps it isn’t the process of using an instrument to evaluate the work that is comparable but the way in which we report the results by using work samples to demonstrate gains in a particular criteria. Of course, this is much more complex than comparing scores from a survey or multiple choice exam.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s