One of the most difficult aspects of assessment is identifying and articulating measurable outcomes. Before you can proceed on this topic, it is important to define and clearly express the mission and objectives of the program or process that you want to assess. In other words, before you can proceed with writing outcomes, you must have already asked and answered the questions, What does my program or process do? Why does it exist? In addition, you will need to determine the following:

• why you are conducting an assessment;
• who the assessment is for;
• how the results will be used;
• to whom the results will be given and in what format;
• when and how often follow-up assessments should be completed; and
• what decisions you expect to make from the assessment findings.

The following pointers assume that a coherent mission and accompanying objectives have already been written. In addition, the process of writing your outcomes assumes that you have attempted to answer the who, what, and when of assessment and that you have identified your target readers. Your intended outcomes will stem from all of that preliminary work and particularly from your program objectives.

What are the Components of a Good Outcome?
First, clarify what you are trying to assess. If you can't define what it is you are trying to learn about your program or process or what learning or development aspect is present, then you certainly can't measure it. Ask yourself, What do I want to assess?Questions that can help you get started with clarifying what you are assessing include the following: Do you want to assess what your program is accomplishing and the degree to which it is being accomplished (program outcomes)? Do you want to assess what students are learning or what staff is learning as a result of the curriculum or training your program is offering (learning outcomes)? Do you want to assess how students are or staff is developing as a result of an intervention you are applying within your program (developmental outcomes)?

Second, the outcome must be measurable. In other words, you must be able to provide evidence that the outcome was or was not achieved. By selecting verbs that specify the trait, ability, behavior, or habit of mind that you wish to assess such as create, compose, calculate, build, develop, or evaluate (Maki, 2001), the outcome is more likely to become measurable. The trait you are assessing can be measured with quantitative or qualitative data. The question to ask here is "Is the outcome measurable?"

Third, the intended outcome must measure something useful and meaningful. According to Nine Principles of Good Practice for Assessing Student Learning produced for the American Association of Higher Education Assessment Forum (1998), "Assessment makes a difference when it begins with issues of use and illuminates questions that people really care about. [I]nformation must be connected to issues or questions that people really care about. This implies assessment approaches that produce evidence that relevant parties will find credible, suggestive, and applicable to decisions that need to be made." For instance, higher education (like every other profession) is replete with examples of very nice-sounding language that simply doesn't hold up under examination. Therefore, one of the critical steps in writing good outcomes calls attention to language and, especially, to claims of value for student learning. Being able to defend, re-define, or delete language that makes such claims is necessary when writing measurable and meaningful outcomes.

Finally, you may want to include in the outcome statement the major criteria for which you will assess the trait, ability, behavior, or habit of mind. That is, on what basis, will you be able to ascertain how well is achieving or performing or using or integrating Maki, 2001). Some choose to place the particular strategy for showing evidence of the outcome in the evaluation methods section of the assessment plan, rather than writing it into the outcome. If you take the latter approach, then you will want to ask the question, "How will this outcome be measured?," thus helping to answer the question, "Is this a measurable outcome?"

Confused?
An illustration may provide some clarification. For example, if you want to know whether students who participated in your service learning course improved their critical thinking skills, you could write a learning outcome that looks similar to the following:

Students participating in the SL206 Service Learning through Music Therapy course will demonstrate an increase in critical thinking skills, as exhibited by an improvement in scores on the Chronicle Critical Thinking pre-test/post-test instrument.

Is this is an acceptable outcome? Check by posing the aforementioned questions:

• Is it clear what you are assessing? Yes, the writer states that she wants to know if students who participated in this particular course have increased their critical thinking skills.
• Is the intended outcome measuring something useful and meaningful? Yes, the writer is measuring the increased level of critical thinking.
• Is the outcome measurable? The writer is asking the participants to demonstrate or show evidence of rather than using a general descriptor such as know or understand.
• How will this outcome be measured? The writer will use, in this case, the fictitious Chronicle Critical Thinking Skills pre-test/post-test that measures improvement in critical thinking skills.

Here is another example of a learning outcome:

Students participating in SL206 Service Learning through Music Therapy will understand the importance of the application of historical methodology.

Is this is an acceptable outcome? Check by posing the aforementioned questions:

• Is it clear what you are assessing? Well, it looks like the writer wants to assess if students who participated in SL206 have an understanding of an application of some concept. However, it is not clear what she means by application. Application to what? And what historical methodology? Clarification is needed here.
• Is the intended outcome measuring something useful and meaningful? This question is difficult to answer, as it is difficult to know what is being measured.
• Is the outcome measurable? The writer wants to assess understanding, which is general. This outcome would become measurable if an active verb was selected such as articulate or demonstrate.
• How will this outcome be measured? While the writer didn't state how the outcome would be measured, it is evident that the outcome needs to be re-written as a measurable outcome before this question can be answered.

Types of Outcomes
Outcomes can be classified into different categories, but many experts in the assessment field differ about category classification. For simplicity, three categories are provided for discussion:

Program Outcomes
Program outcomes illustrate what you want your program to do. These outcomes differ from learning outcomes in that you discuss what it is that you want your program to accomplish. Program goals can often be measured in simple terms such as Was the task or activity completed or not. However, this kind of question may not be meaningful and will not provide you with information that is necessary for continuous improvement. Thus, it may be wiser to assess the effectiveness of what you want to accomplish in your program.

For example, a program goal may be stated as The Deciding Student Academic Advising Office will advise deciding students of all racial groups represented in the deciding student population. As you can see, there is no intention of assessing learning or development or quality of service in this outcome. In addition, this outcome doesn't attempt to assess the level of satisfaction of all races with advising; it only attempts to assess whether deciding students of all races were advised. Does this outcome contain all the components of a measurable outcome? Is it measuring something useful and meaningful?

Learning Outcomes
Learning outcomes illustrate the learning that you want to occur. Learning outcomes assess cognitive abilities, not affective dimensions or attitudes. Cognitive abilities refer to thinking skills, affective objectives refer to attitudes and values (Palomba & Banta, 1999). It is important to note that program outcomes are not learning outcomes unless you are assessing actual learning. The following example is not a learning outcome. Why not?

Three seminars on critical thinking will be presented and students who participate will be expected to write two essays on critical thinking skills—one prior to the start of the seminar and one upon completion of the seminar.

While this statement illustrates what will occur to enhance learning (i.e., what students will do, write essays), it does not illustrate the assessment of learning (i.e., how well they did and what their essays demonstrate). This statement would be classified as a program outcome (writing 2 essays) rather than a learning outcome (the learning that would be expected from writing two essays).

Developmental Outcomes
Developmental outcomes illustrate the affective dimensions you desire to instill or enhance. Developmental outcomes assess affective dimensions or attitudes, not cognitive abilities. Affective dimensions are directed toward a person, (including oneself), object, place, or idea, and predispose individuals to behave in certain ways. Examples of intended outcomes for affective dimensions include being sensitive to the values of others, becoming aware of one's own talents and abilities, and developing an appreciation for life-long learning. Practicing ethical behavior, exhibiting personal discipline, and providing leadership are other examples of intended outcomes that address attitudes and values (Palomba & Banta, 1999).

A specific example of a developmental outcome is Students participating in SL103 Service Learning through Latin American Studies will show evidence of increased civic responsibility as measured by increased civic responsibility correlation on the CSIPI pre-test/post-test. Does this outcome meet the criteria for a measurable outcome?

Practice, Practice, Practice
Writing good outcomes takes practice, and it takes a lot of time. Writing good outcomes is an iterative process. Continuing to ask these questions will help you in your thinking and, thus, in your writing. Asking others to evaluate the substance and value of your answers to those questions may reveal holes in your outcomes statements as well. Try not to be frustrated with the process. Be patient with yourself and seek assistance from others. Finally, enjoy the experience of seeking and documenting new information, knowledge, and understandings about your work.

References:
AAHE. (1998). Nine Principles of Good Practice for Assessing Student Learning. American Association of Higher Education Assessment Forum.

Maki, P. (March, 2001). Student outcomes assessment:Strategies for getting started, sustaining the momentum, and addressing obstacles. AAHE National Conference on Higher Education, Washington D.C.

Palomba, C.A. & Banta, T.W. (1999). Assessment essentials: Planning, implementing, and improving assessment in higher education. San Francisco: Jossey-Bass.

St. Cloud State University Assessment Office (1999). Developing an assessment plan for your program: Guidelines for program assessment. http://condor.stcloudstate.edu/~assess/guidelines.html

Marilee J. Bresciani, Ph.D., is Director of Assessment for the Division of Undergraduate Affairs at North Carolina State University. Her contact information:

126 Leazar Hall, Box 7105
Raleigh, North Carolina 27695-7105
Phone: 919-515-6433 Fax 919-515-4416
Marilee_Bresciani@ncsu.edu