There are scores of
books, articles, and websites that offer examples of case studies
which use easy, sound assessment methods. Three popular resources
that we have found useful are Linda Suskie’s Assessing Student
Learning: A Common Sense Guide (Jossey-Bass 2009); Thomas A.
Angelo and K. Patricia Cross’s Classroom Assessment Techniques:
A Handbook for College Teachers (Jossey-Bass 1993); and
Barbara E. Walvoord’s Assessment Clear and Simple: A Practical
Guide for Institutions, Departments, and General Education,
2nd ed. (Jossey-Bass 2010). The menu below (by no
means comprehensive) references passages in these books where
faculty can find additional information.
Listening to
Students
Anonymous
Feedback
Collecting short, anonymous feedback at
different points in the semester can be a great means of finding
out from students not only what they think of a course, but where
they would prefer greater clarity, more time, or a chance to review
certain information. Faculty can ask students to submit a page of
typed, anonymous feedback at mid-semester, and then if necessary
modify their teaching in response for the second half of the
semester. Then, faculty could ask students to submit another
one-page feedback at the end of the semester, reflecting upon
whether or not they felt their concerns had been addressed in the
second half of the semester. Even if a faculty member is unable to
fully address every concern that students have, the simple act of
asking for students’ opinions, and making an honest effort to
address some of them, speaks volumes. Students appreciate such
efforts.
One-minute
Paper
Faculty can ask students to take a minute at
the beginning or at the end of class to offer feedback, summarize
an issue under discussion in the course, or answer a problem.
Faculty can then quickly skim through these responses in order to
identify any outstanding questions, issues, or problems that
students identify.
Conferences
If the size of one’s class permits, faculty
can conduct one-on-one or small group conferences with students,
asking them questions related to the course or soliciting their
feedback. Many faculty find that students who are silent observers
in class tend to open up more freely in such encounters outside of
the classroom.
Online
Feedback
Using an online discussion board, through
Blackboard or any other course management system, is an excellent
way to gather student comments and to assess their ongoing
understanding of material. Many students who would not speak up in
class, nor feel comfortable in one-on-one conferences, will freely
write feedback in an online format.
See Angelo and
Cross’s Classroom Assessment Techniques for more
information on such related assessment techniques as focused
listing, memory matrices, analytic memos, one-sentence summaries,
concept maps, directed paraphrasing, student-generated test
questions, diagnostic learning logs, electronic mail feedback,
assignment assessments, etc.
Focus
Groups
An ideal way for departments to gather
information from their majors and minors is to hold focus groups
where a manageable group of students are brought together to answer
a series of questions that the department has identified. In focus
groups, it is important that the questions asked in each group
remain the same, and that the people asking the questions remain
neutral, and as much as possible do not contribute their own
opinions to the conversation. A good incentive to bring students
together for focus groups is to offer food (free pizza is an
important assessment tool that all departments should have in their
arsenals). It can help to have two people conduct these focus
groups—one to ask questions and gather input from all students
assembled—and to ask follow-up questions in order to further
clarify students’ input—and a second person to record those
responses. After a series of focus groups, faculty can report their
findings back to their departments, and have a discussion as
to what they should change or maintain as a result of that student
feedback.
Focus groups have the added benefit of sending
a message to students that a department values their input, and
regularly considers their opinions as it conducts its business of
designing curricula and evaluating the program.
For more on Focus
Groups see Suskie, pp 195, 196.
Pre-Post
These “before and after” measures show how
students have and haven’t progressed from one period to another. A
professor might collect writing samples from her students early in
a semester, then compare one or two variables later in the semester
to see if they have changed. For example, how are students using
evidence to back up their written claims? As with much assessment,
the key here is to be focused one the variable one has chosen to
assess: if one is looking at how students are using evidence, one
does not (for the purposes of this assessment) focus on other
issues as well, such as, say, presence of lively description. The
more that faculty keep their assessment question simple and direct,
the greater their success in following through on the project.
A Department can also conduct pre-post
assessment across courses. A variable in the writing of students in
a sophomore course for majors can be compared to that of students
in a senior capstone course in order to obtain evidence on the
degree to which students are demonstrating progress. If a
department discovers that their senior students are not responding
to that variable as successfully as they would like, then this can
lead to changes in how that sophomore or senior course are taught.
(Such changes in teaching are the all important “feedback loop”
that needs to happen if assessment is to be successful.) Here it is
important to note that it is not a black mark on the department to
identify that their students are not performing up to certain
standards in their senior year; in fact, the department’s diagnosis
of this problem via assessment—and their resultant efforts at
addressing this problem—would be examples of an engaged faculty
conducting good assessment.
For more on getting
students to reflect at the beginning and end of courses, see Suskie
190-191. Pre-Post or Before-and-After approaches to assessment are
implicit in many of the methods described on this page.
Rating
Scales
A popular means of gathering information from
students on a range of issues is via rating scales. The First-Year
Writing program and the Writing Center have worked together to give
all first-year writing students a Likert Rating Scale at the
beginning and again at the end of the semester during which they
take ENG 1000c. The purpose of this scale is not to measure student
writing ability, but rather their engagement and self-assessment as
writers.
For more on rating scales, see Suskie pp 195-198.
Rubrics
Many faculty use rubrics as a means of
clarifying their values, and defining the characteristics of what
they might describe as successful, moderate, and less successful
performance. Many value rubrics because they make evaluation less
mysterious and seemingly arbitrary.
See Suskie, pp
137-153 for more on scoring guides and rubrics. See Barbara
Walvoord’s Appendix on “Sample Rubrics” in Assessment Clear and
Simple (107-14). Also, one can find rubrics designed for our
University core courses
here.
Some faculty and departments, however, prefer
methods other than rubrics. Bob Broad’s concept of Dynamic Criteria
Mapping, however, originated in part from a critique of rubrics
that found them to be too narrow and not descriptive enough to
respond to the rich and complex range of variables that can exist.
As a result, some faculty and programs come up with their own
“homegrown” maps or other forms of capturing and defining the
characteristics of student work they wish to promote.
For faculty
interested in examples of these alternatives to rubrics, we
recommend Bob Broad (ed.), Organic Writing Assessment: Dynamic
Criteria Mapping in Action (Utah State University Press,
2009).
Surveys
Similar in spirit to focus groups, surveys
allow faculty to solicit feedback from students in response to a
variety of questions drawn up by a faculty member or department.
Surveys can be distributed online, or passed out and collected
(anonymously) in class. One advantage of surveys over focus groups
is that a department can get input from a greater number of
students. One possible drawback however (especially with online
surveys) is that students sometimes suffer from “survey fatigue,”
as they are continually asked to fill out surveys and
evaluation forms for so many other programs and events.
There is an art to
writing good survey questions. For more information on surveys, see
Suskie pp 198-200.
Values
Inventory
One of the most important early steps any
department can do to approach assessment logically and efficiently
is to first spend time identifying the shared values within that
department. The goal is to develop an inventory of a department’s
shared values that will then serve as the foundation for the
learning goals to be established for a course or program. Many
departments would do well to consider this approach before
embarking on other more detailed assessment projects. This can be
an ideal way to gather faculty together in a spirit of mutual
inquiry, discussion, and exploration.
Individually, faculty can spend time writing
down all of the learning outcomes they want their students to
have—either at the end of a particular course, or at the end of
their program. Then faculty would come together to share what each
had written down privately in order to see where their opinions are
in sync. They might rank their desired outcomes, then see where
different members of a department prioritize certain skills or
qualities. Such a departmental values inventory quickly becomes an
exercise in vocabulary analysis: for example, if a faculty member
says she wants her students to write “strong claims,” whereas
another privileges “rhetorical risk-taking,” faculty would then
spend time trying to articulate and carefully define what, exactly,
these terms mean to them. Such clarification of terms—and the
inevitable intradepartmental faculty debates that result from such
conversations—is an essential component of good assessment. Until
faculty can carefully and accurately define their own vocabulary of
learning outcomes terms, any assessment of students’ ability to
meet those outcomes could lack validity. Much of assessment is a
matter of moving beyond the general to the specific: fine-tuning,
clarifying, and articulating with precision what, exactly faculty
are looking for in student performance.
See “Developing Learning Goals” in
Suskie (115-133) and Walvoord (81-85).
Portfolios
Portfolios offer a rich means of collecting
artifacts students have created in their courses, as well as their
own self-assessment. Portfolios can include both early and final
drafts, and all manner of artifacts. Beginning Fall 2011, the
University will begin implementing electronic portfolios for
certain students. For more on our electric portfolios, go
here.
For more on
portfolios see Suskie (202-213) and Walvoord (50-54).
IRB
Many assessment methods do not require
permission from an Institutional Review Board. Federal policy
exempts the following:
(1) Research conducted in established or
commonly accepted
educational settings, involving normal
educational practices, such as (i) research on regular and special
education instructional strategies, or (ii) research on the
effectiveness of or the comparison among instructional techniques,
curricula, or classroom management methods.(2) Research involving
the use of educational tests (cognitive, diagnostic, aptitude,
achievement), survey procedures, interview procedures or
observation of public behavior.
Faculty and departments must be very careful,
however, that individual students can’t be identified, nor harmed
by disclosing their opinions or feedback outside of the assessment
research. For more see U.S. Department of Health and Human
Services, Title 45, Part 46, “Protection of Human Subjects,”
Section 101, “To What Does This Policy Apply,” found
here. Finally, consult our University’s IRB
here for further information.
WEAVE
All of the above assessment methods are
legitimate approaches, and can be summarized and entered into WEAVE
online as examples of a department’s ongoing efforts to assess its
courses and programs.