Good Practices in Outcomes-Based Assessment Program Review

This study describes “good practices” in outcomes-based assessment program review in an effort to demonstrate how improvements in student learning and development are made based on a systematic institutional evaluation process.  Basic illustrative components of outcomes-based assessment program review are presented as well as criteria for identifying good practices within outcomes-based assessment program review.   Institutional case studies in outcomes-based assessment program review are interwoven throughout the study to illustrate principles of good practice.   Finally, suggested steps for implementing sustainable outcomes-based assessment program review are outlined.

This study is intended for faculty and administrators responsible for implementing and sustaining outcomes-based assessment program review at their respective campuses.  In addition, the study can help faculty and administrators understand the “what” and “why” of outcomes-based assessment program review.  Furthermore, it will explain the value of the outcomes-based assessment program review process as identified by the participating good practice institutions.

This study is not a “how-to-do-assessment” presentation.  Rather, this study utilizes the results of a grounded theory study of multiple institutions to posit answers to the questions:  what does good outcomes-based assessment program review practice look like from an institutional perspective?  Who is doing replicable work?  In short, it couples fine assessment practices with cyclical program review so that the single process of outcomes-based assessment informs many purposes: program review, strategic planning, professional accreditation, institutional accreditation, and possibly even the assessment of general education.

Assessment and Accountability
This study is a timely contribution to literature on outcomes-based assessment.  Because legislatures and government agencies are increasingly observant of institutional practices and are involved in institutional decision-making, accountability has become a prevalent concept and programmatic initiative (Allen & Bresciani, 2003; Ewell, 1997a; Ewell, 1997b; Ewell, 2003).  To respond to growing demands for institutional accountability with regard to student learning and related expenditures, higher education requires an articulated definition of “quality education,” rather than the ill-defined conceptualizations currently in use.  For example, those engaged in quality of education conversations traditionally have used persistence and graduation rates as reliable indicators of quality education (Pike, 2001; Ewell, 1997a).  Such indicators are easy to define and measure, but do they identify the quality of the education experienced by students?  Using these indicators, can one understand how well a student has learned the content and mechanics of her discipline or program?  How well do faculty know whether a student has mastered problem solving or application of theory to practice?  Indicators of persistence and graduation may indicate institutional type and mission, but they typically neglect more meaningful questions of quality learning outcomes.

Nationally standardized testing, whose purpose is ostensibly to evaluate student learning across institutional types, is another traditional medium through which to measure the quality of education.   Though they offer a relatively simple method of data collection, many tests are designed without consulting the curricula; thus, the tests are not measuring the content of the course or program being delivered (Lopez, 1997; Maki, 2004).  One solution is to require institutions to align their curricula with the content of the standardized test, a practice that typically reduces education to “teaching-to-the-test.”  Additionally, such an approach neglects the particularities of institutional culture and its unique student populations in order to satisfy a standardized approach to higher education.  Another consequence may be a decrease in the specialized information required for institutional and societal improvement. This trade-off may recreate the pattern of educating elites only, which practitioners have eschewed in favor of more equitable and accessible education (Anderson et alia, 2001).  Yet another consequence may be that the information contained in the standardized tests may remain behind the learning curve of the research that is taking place in higher education, therefore furthering the gap between postsecondary classroom learning and scholarly research.  Additionally, standardized tests may be unable to accommodate varying learning styles (Anderson et alia, 2001; Maki 2004). Finally, the question is raised as to whether using one standardized test to evaluate all students' learning is even responsible research.  What researchers do we know who engage in sound research methodology  but use only one means of evaluation to formulate a theory or an assumption of quality?

Despite their claims of promoting quality and standards, legislators who call for strict quality monitoring actually appear to promote “watering down” educational principles because their proposals take simplistic approaches to evidence of learning. Moreover, the proposals often prevent the expression of faculty expertise and varying institutional values and resources (Bresciani, 2003).  Some of these legislated indicators even distort conversations about quality.  Institutions may feel compelled to seek traditional types of students and promote ”cookie-cutter” institutional values in an effort to meet some standardized definition of a quality institution (Ewell, 2003).   In reality, of course, different types of institutions are needed to serve different types of students with varying educational goals, such as an institution that educates adult single parents to become more effective managers, a research extensive university where 60 percent of the student body leaves on a two-year religious mission, or a private liberal arts college whose students do not typically work more than 20 hours a week and who are most likely to obtain a bachelor’s degree in four years (Bresciani, 2003).  It is not difficult to see the obstacles in using any one test or set of indicators to assess the actual quality outcomes of the curriculum.

The challenge of finding meaningful quality indicators is exacerbated by a model of higher education that dates back to the mid- to late-1800s (Goodchild & Whecshler, 1989).  Patterns and habits have become entrenched.  Change in the delivery of higher education comes slowly, even when clearly necessary.  When government officials ask faculty and administrators to verify student outcomes, institutional responses are defensive, subversive or non-existent.  Such avoidance behaviors manifest themselves because the inquiries and required indicators come from outside the academic discipline and institution (Bresciani, 2003).  And, equally important, those stakeholders outside the academy—legislators, donors, parents, other educators, granting agencies, and so on—hold not only the purse strings for our endeavors, but also control the barometer of public opinion, prestige, and general well-being for the academy.  In short, refusing to answer questions from these stake holders about academic quality is no longer an option.

The scrutiny of quality in higher education is not diminishing.  In fact, educators face growing pressure to produce evidence of student learning, while faculty grow increasingly frustrated that their disciplinary expertise is under-appreciated or unrecognized (Bresciani, 2003).  How, then, do institutions (a) report in a meaningful way data on student learning and development?

The full summary of these findings can be found here: http://styluspub.com/Books/BookDetail.aspx?productID=113962