Preparing Faculty Members To Do Assessment
Janice E. Thornton, Oberlin College
Patricia Ann deWinstanley, Oberlin College
Background. Oberlin College recently applied for and received an Award for the Integration of Research and Education (AIRE) from the National Science Foundation. This award has been used primarily to sponsor curriculum development fellowships for faculty members so that they can develop/modify their courses to integrate more research-like activities into them. As part of the award we also said that we would develop ways to institutionalize mechanisms for evaluating these curricular innovations and developments. Our assessment strategy has been multifaceted. First of all, we developed an Assessment Workshop to train faculty in the principles and practice of assessment. Additionally, we developed a pre/post questionnaire of students in AIRE-sponsored courses to assess the impact of these new research-like experiences across the curriculum, and a survey of AIRE sponsored faculty regarding the design and the impact of their course innovations. Moreover, we have designed an experimental study on whether research-like experience vs a lecture differentially impact the various aspects of learning. The present discussion will describe our efforts to develop an Assessment Workshop to teach faculty the basic principles and practice of assessment. We ran this workshop at Oberlin College and it was so successful that we were invited to run a similar workshop at Colby College. That too was successful. We had approximately 20 participants at each workshop.
Reasons for the Assessment Workshop. The main purpose of Oberlin College's AIRE was to stimulate effective curricular innovation, i.e. innovation that accomplishes the educational goals of the course. Obviously, this is a purpose that is shared by faculty members whenever they develop new course materials. However, many faculty members do not realize that this requires that curriculum development and assessment be intertwined. Faculty members are very diverse in their level of knowledge regarding assessment principles and practice, and the level of motivation to do assessment. We wanted to bring faculty members to an understanding of the importance of doing assessment and help them understand some of the basic principles involved. We also wanted to teach them some basic techniques and help them develop tools they could use to assess their own curricular innovations. Many faculty members are unfamiliar with assessment procedures and are reluctant to try them or resistant to the additional work created by them. We also realized that assessment may pose threats to faculty members when the success of their curricular innovation is not certain (particularly since many faculty members are only familiar with summative assessment that is used to evaluate faculty members). Therefore we wanted to enhance faculty members' commitment to assessment by affirming their feelings as natural while demonstrating the benefits of formative assessment work.
We developed a one day workshop for faculty members, to prepare them to do assessment. We centered our examples around the formative assessment of curricular innovations but also indicated how many of same principles apply to both formative and summative assessment, and to instances other than curricular assessment.
Goals. We had a number of goals for the workshop. Specifically, we wanted participants to increase their knowledge about assessment principles and resources, increase their skill in doing assessment, increase the perceived value of assessment, and the perception of it as practical and doable, increase their comfort level in doing assessment, and increase the likelihood that they would do assessment in the future. Lastly, one of our goals was to foster within and across departmental communication by having the participants learn more about what their colleagues are doing in their respective classes. To assess how well we reached our goals we developed a pre-workshop survey and a post-workshop survey for these items and administered it to workshop participants.
Overview Of The Process. Prior to the workshop day we encouraged each participant to develop and bring to the workshop a set of course goals for a course they wanted to work on. We conceptualized the workshop as having two parts. In the morning, we covered "The What And The Why of Assessment". In the afternoon, we covered "The How-To of Assessment".
The What and The Why of Assessment. This covered the more qualitative aspects of assessment. We talked about the kinds of things that are assessed, the reasons for doing assessment, and some of the obstacles. We then talked about why to set educational goals. Afterwards we assigned them to small cross-disciplinary groups (of 3-4 people) and asked them to describe their course goals to each other and to come up with 3 shared goals. Then as a large group we discussed the shared goals and looked at the commonalities. We also used these as examples to talk about the categories of goals including knowledge, skills, attitude, and intentions. We also described the differences between goals, observable objectives, and activities. We described how to formalize goals/objectives/activities, and introduced Bloom's taxonomy of knowledge objectives. With this as background we then had them again break into small groups, this time within a discipline. In those groups they re-examined their course goals. They then chose one goal and developed written objectives and activities to meet it. We then came back together as a large group and discussed what they came up with and how the process went. At the end of the morning we talked more specifically about assessment and what it is (i.e. formative vs. summative) and why do it.
The How-To of Assessment. This covered the more quantitative aspects of assessment. We began the afternoon by describing types of assessment tools (e.g. self-reflective, instructor, or third-party assessment). We then gave them some written goals for a hypothetical Intro to Science course (using some of the shared goals they had generated in the morning) and had them work in small groups to develop some assessment tools for the course. Each group had 2 goals to work with, and each goal had 2 groups that worked on it. Then, as a large group we discussed the tools developed. With this foundation, we then had each person begin to develop assessment tools for their own course. Although they worked individually, they sat next to someone working on a similar course/problem so they had someone with whom to confer. Once they had gotten this hands-on experience in developing their own assessment tools, we discussed a specific example of how one can analyze the results of a course assessment and what it can tell the instructor. We then did a wrap-up of the Workshop and had them fill out a post-workshop survey.
Final Comments. As faculty members in the Social and Natural Sciences, we understand many of the issues that face other faculty members, particularly at liberal arts colleges. We feel that we understand a lot of the roadblocks that faculty perceive in regards to doing assessment. We also believe that relatively small assessment efforts can yield useful information about a course innovation's effectiveness and ways to improve course. This workshop was designed to help faculty members easily gain some basic knowledge about assessment principles and practice. In the long run, if we can get faculty to understand and appreciate the usefulness of assessment, we anticipate that they will be more willing to participate in future assessment efforts.