From "Assessment" to "Scholarship of Teaching and Learning"
Jo Michelle Beld, St. Olaf College
David Van Wylen, St. Olaf College
Matt Richey, St. Olaf College
The subject of this conference in general, and of this session in particular, could not be more timely for our professional interests and responsibilities. Beld has just assumed the directorship of the Office of Academic Research and Planning at St. Olaf College, a residential liberal arts college in the Midwest. She is a faculty member in the Department of Political Science but brings a long-standing interest in educational policy and research, coupled with experience in evaluating both educational and public programs, to her new role. The decision to appoint a faculty member to the directorship of the Office (rather than hiring a staff professional in educational research) was quite intentional on the part of the College administration, for precisely the reasons being addressed by this session. Assessment will never achieve its full potential unless it is organic to the institution and endemic to faculty roles and rewards. This, in turn, requires faculty leadership, rather than external or top-down mandates. Beld is seeking concrete strategies she can use to help make assessment part and parcel of the delivery of educational services and programs at St. Olaf, and in her view, this means decentralizing both the kinds of questions that are asked and the design and implementation of assessment plans.
This is a particularly pressing concern at this point in the life of the College, because our re-accreditation process is currently underway. We have applied to the North Central Association to conduct a special-emphasis self-study; one of the five areas of emphasis is the improvement of student learning through the scholarship of teaching and learning. Beld is responsible for authoring this chapter; equally important, the College has committed itself to having made progress in each of the five areas of the self-study by the time the external visiting team conducts its on-site investigation in the spring of 2003. She has prepared a prospectus for this chapter which directly addresses the role and impact of assessment practices at the institutional level. The excerpt below indicates where her thinking is on these issues at this point in time:
Assessment at St. Olaf College is not yet organic. It is not sufficiently embedded in the educational programs offered by the College; it is not clearly linked to faculty roles and rewards; and it is not adequately supported by institutional infrastructure. All too often, faculty continue to see assessment as an externally-imposed and not-very-useful mandate, rather than as an activity which both springs from and supports the College's educational mission.
One way to move forward is to reconceptualize assessment more broadly, in a way which better fits the mission, culture, and resources of the College. A promising direction is to focus this chapter on the scholarship of teaching and learning, of which assessment is one, but not the only, dimension. According to Barbara Cambridge, Vice President for Programs for the American Association for Higher Education (AAHE) and Director of the Carnegie Academy Campus Program, The scholarship of teaching and learning is an endeavor that involves serious, designed inquiry. An investigator chooses a question or issue that is intellectually interesting and important pedagogically, and then designs experiments [or other means of systematic data collection] to explore that.
Pat Hutchings and Lee S. Shulman, respectively vice president and president of the Carnegie Foundation for the Advancement of Teaching, suggest that faculty engaged in the scholarship of teaching and learning "frame and systematically investigate questions related to student learning--the conditions under which it occurs, what it looks like, how to deepen it, and so forth--… with an eye not only to improving their own classroom but to advancing practice beyond it." The scholarship of teaching and learning, like other kinds of scholarship, is thus "public, subject to critical evaluation, and usable by others in both the scholarly and the general community." This characterization of the scholarship of teaching and learning is rooted in the multi-dimensional nature of scholarship articulated in the 1990 report, Scholarship Reconsidered, by former Carnegie Foundation President Ernest Boyer.
Reconceptualizing "assessment" as "the scholarship of teaching and learning" will enable the members of the St. Olaf community to more readily identify its potential benefits, not only for student learning but for faculty learning as well. The scholarship of teaching and learning, like other kinds of scholarship, springs from questions or puzzles originating in the lived professional experiences and disciplined reflections of committed teacher-scholars. It is most likely to be meaningful, sustained, and supported over time if it is undertaken to:
- Inform decision-making about a specific program or educational activity -- a course, a major, a concentration, a General Education requirement, an off-campus or experiential learning program
- Satisfy genuine intellectual curiosity on the part of those responsible for delivering the program and/or designing and carrying out the inquiry
- Strengthen faculty capacity for disciplined inquiry
- Improve student learning, not simply through analyzing and applying the results of the inquiry, but ideally through the process of conducting it.
Put simply, the better the fit between the scholarship of teaching and learning and the roles, responsibilities, and rewards of faculty and staff, the more likely it is that such scholarship will be undertaken and used to improve student learning.
A final reason to focus on the scholarship of teaching and learning in this chapter is to enable the College community to recognize ways in which disciplined pedagogical inquiry is already occurring across the campus in a wide variety of contexts -- program reviews, grant-supported initiatives, faculty publications, and so forth. This portion of the self-study will thus allow us to identify, evaluate, and disseminate internally an array of ongoing activities, which hold promise for the improvement of student learning. A clear understanding of what we are already doing successfully is essential to the development of a successful long-term plan.
Beld's most significant experience with assessment to date derives from her role as Project Director of a major pedagogical innovation funded in part by the Fund for the Improvement of Postsecondary Education (FIPSE). In the fall of 1994, St. Olaf implemented a new General Education curriculum which included a new requirement in oral communication. The College committed itself to offering courses across the curriculum to meet this requirement, modeling its instructional and faculty development program on analogous programs in writing across the curriculum. (As an aside, the program has attracted a small but committed number of faculty from the Faculty of Natural Sciences and Mathematics, and several courses in the Departments of Biology and Chemistry have been revised to include instruction, practice, and feedback in effective oral communication in the scientific community.) Beld authored the grant, developed and co-facilitated the faculty development program, and in collaboration with an interdisciplinary leadership team, developed and implemented the evaluation plan, which included the assessment of student learning outcomes. She anticipates that the evaluation plan will continue under the auspices of her new responsibilities as Director of Academic Research and Planning, and that it will serve as a model for inquiry into the effects of other disciplinary and inter-disciplinary programs. Equally important, she hopes that what she learns from other departments and programs with respect to the evaluation of student learning can be used to improve both the design and the use of the oral communication evaluation plan.
Like Beld, Van Wylen has assumed new responsibilities which place him squarely in the conversation about assessment in the service of student learning. As the newly-appointed Associate Dean of Natural Sciences and Mathematics, Van Wylen is a principal user of the data generated by assessment at the departmental and interdisciplinary program level. One of the particular challenges he faces is that the disciplines for which he is responsible for providing leadership are, perhaps even more than some other disciplines, constantly changing. In a sense, the "institutionalization" of assessment practices runs directly counter to the dynamic nature of the disciplines he leads. The question for him might not be simply "How can assessment be institutionalized?" but more fundamentally, "Why should assessment be institutionalized, when the findings may be so rapidly outdated and therefore of little utility to the educational program being assessed?" Given Van Wylen's responsibility for effective use of scarce institutional resources (including faculty time), this is a pressing and difficult question.
Richey's institutional responsibilities have also engaged him with designing and implementing assessment practices at the institutional level. As chair of the Department of Mathematics, he typifies one of the major "clients" for Beld's Office of Academic Research and Planning. -- namely, department chairs and program directors. His department is currently completing a major self-study as part of its five-year program review process. In addition, the Computer Science concentration staffed by several members of the Mathematics Department is also undergoing review. Beld and Richey are currently working together to pilot an instrument in several sections of the department's sequence of Calculus courses to determine whether some textbooks and pedagogical approaches are more effective than others in motivating and enhancing student learning in these courses. This project exemplifies the department's hope that the program review process will do more than inspire a one-shot effort to document student learning outcomes. Instead, the department intends to use the program review process as an opportunity to integrate inquiry into student learning as part and parcel of the delivery of its educational program. The Department is also considering developing a proposal to the National Science Foundation's new grant program in Evaluative Research and Evaluation Capacity-Building, perhaps in collaboration with one or more of the other departments in the Faculty of Natural Sciences and Mathematics. Participation in this roundtable would serve as invaluable preparation for the development of an effective proposal.