I have several projects this summer that have lead me to think deeply about assessing student learning. Partly this time has been spent trying to understand how the structural factors surrounding the learning and assessment (class structure, assigned task, time of day, etc) effect the outcome of the assessment. I am starting to wonder if one can control for those variables or even if it is advisable to try. Does trying to set up a controlled assessment destroy the learning process? I think it is a fair question. As of yet I have no answers but I am interested in what my fellow IC-bGers might have to add to this conversation. I ran across this quote from an excellent book on assessment written by the assessment gurus at the National Research Council and published by the National Academies Pres; Knowing what students know: The science and design of educational assessment (pdf).
“Every assessment is also based on a set of beliefs about the kinds of tasks or situations that will prompt students to say, do, or create something that demonstrates important knowledge and skills. The tasks to which students are asked to respond an assessment are not arbitrary.“
All too often, I fail to think how the task that students do impacts the assessment and fall into the trap of thinking that using a single instrument to assess very different tasks will produce comparable data. Recently, I was lucky enough to see Charles Blaich and Kathleen Wise’s keynote at the 2013 Institute for Pedagogy in the Liberal Arts where they discussed what results from the Wabash National Study and the Center of Inquiry at Wabash College. This multi-year study follow student cohorts from a variety of institutions through their college experiences, collecting student and institutional data with a wide-ranging set of assessment instruments. The results are somewhat disappointing in that students don’t show great gains in many criteria and in some they even demonstrate losses in some areas (e.g., openness to diversity).
The message from this keynote and the NRC book I find compelling is the idea that assessing student learning is best done directly as a part of the learning process. Secondary measures of learning are second best. Institutions of higher education tend to rely very heavily on national surveys such as the NSSE and similar indirect, self-reported surveys. The problem with these assessments is that they don’t directly measure student learning but rather demonstrate student’s attitudes about their own learning.
So that leaves us with a quandary…
How will we get to comparable measures of learning that are conducted in the classroom. Any ideas out there?